[jira] [Commented] (AVRO-2302) Invalid namespace importing Avro files generated from Protobuf

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (AVRO-2302) Invalid namespace importing Avro files generated from Protobuf

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/AVRO-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16763072#comment-16763072 ]

ASF subversion and git services commented on AVRO-2302:
-------------------------------------------------------

Commit e0ab6bd54ab632569782a577b7e9a1d419e08845 in avro's branch refs/heads/master from Pedro Carneiro
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=e0ab6bd ]

AVRO-2302 Fix invalid namespace on BigQuery import

This change fixes an issue when importing Avro files into BigQuery, when
such files have been generated using `ProtobufData`. The schema
namespace may end in a dollar sign, if using nested classes.

The correction piggybacks on the fix provided for AVRO-2143 (#283). In
it, `SpecificData` will attempt to load a class joining the namespace
and the name with a dot, and if that fails, it tries again with a dollar
sign.

The same behaviour is hereby reused for `ProtobufData`.


> Invalid namespace importing Avro files generated from Protobuf
> --------------------------------------------------------------
>
>                 Key: AVRO-2302
>                 URL: https://issues.apache.org/jira/browse/AVRO-2302
>             Project: Apache Avro
>          Issue Type: Bug
>    Affects Versions: 1.8.2
>            Reporter: Pedro Carneiro
>            Priority: Major
>             Fix For: 1.9.0
>
>
> An error occurs when importing Avro files into BigQuery.
>  The files were generated using a ProtobufData schema.
>  The error shown is similar to:
> {code}
> BigQuery error in load operation: Error processing job
> '<some_job_id>': Error while reading data, error message: The
> Apache Avro library failed to parse the header with the following error: Invalid namespace:
> some.package.SomeClassProto$
> {code}
> When using a modified version of the ProtobufData class, that yields a namespace like
> {noformat}
> some.package.SomeClassProto
> {noformat}
> (without the dollar), the import worked without errors.
> Potentially related to AVRO-2143?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)