[jira] [Commented] (AVRO-1724) avrogen.exe "import" handling

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (AVRO-1724) avrogen.exe "import" handling

Dave Cole (Jira)

    [ https://issues.apache.org/jira/browse/AVRO-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17345942#comment-17345942 ]

Tim te Beek commented on AVRO-1724:
-----------------------------------

Via this thread [https://github.com/confluentinc/confluent-kafka-dotnet/issues/1127#issuecomment-562966218]

I came upon [https://github.com/alfhv/avro] which seems to have added rudimentary support for schema imports.

Could that be used to restart this effort?

> avrogen.exe "import" handling
> -----------------------------
>
>                 Key: AVRO-1724
>                 URL: https://issues.apache.org/jira/browse/AVRO-1724
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: csharp
>            Reporter: Norman Fung JP
>            Priority: Major
>         Attachments: avro-csharp-1.7.7.tar.gz, simpleTest.7z
>
>
> I have two simple JSON Schema in SAMEfolder:
> Content in "simpleA.avsc" as follows:
> {
>  "namespace": "com.bearstearns",
>  "type": "record",
>  "name": "simpleA",
>  "imports": ["simpleB.avsc"],
>  "fields":
> [
> {"name":"appName","type":"string"},
> {"name":"userId","type":"string"},
> {"name":"nestedB","type":[{"type":"array","items":"simpleB"},"null"]}
> ]
> }
> Content in "simpleB.avsc" as follows:
> {"namespace": "com.bearstearns",
>  "type": "record",
>  "name": "simpleB",
>  "fields":
> [
> {"name":"ID","type":"string"},
> {"name":"SimpleName","type":"string"}
> ]
> }
> Avrogen can be downloaded from http://apache.communilink.net/avro/avro-1.7.7/avro-csharp-1.7.7.tar.gz
> After download, from command prompt set PATH environment variable:
>        set PATH=%PATH%;C:\...\avro\codegen\Release
> From Command Prompt,
>     avrogen -s simpleA.avsc outputDir
> The error message is:
>     "Exception occurred. Undefined name: simpleB"
> By the way, you cannot get around this by putting two schema in one file: Only one on top will be generated. The one in bottom won't. For example below if you put simpleB top, comma, then simpleA bottom. Only simpleB.cs will be generated (simpleA.cs won't)
> {"namespace": "com.bearstearns",
>  "type": "record",
>  "name": "simpleB",
>  "fields":
> [
> {"name":"ID","type":"string"},
> {"name":"SimpleName","type":"string"}
> ]
> },
> {"namespace": "com.bearstearns",
>  "type": "record",
>  "name": "simpleA",
>  "fields":
> [
> {"name":"appName","type":"string"},
> {"name":"userId","type":"string"},
> {"name":"nestedB","type":[{"type":"array","items":"simpleB"},"null"]}
> ]
> }
> However, a workaround is to nest simpleA and simpleB in same avsc - this will work:
> {
>  "namespace": "com.bearstearns",
>  "type": "record",
>  "name": "simpleA",
>  "fields":
> [
> {"name":"appName","type":"string"},
> {"name":"userId","type":"string"},
> {"name":"nestedB","type":{"type":"array","items":{
> "type": "record",
> "name": "simpleB",
> "fields":
> [
> {"name":"ID","type":"string"},
> {"name":"SimpleName","type":"string"}
> ]
> }
> }}
> ]
> }
> fyi, I also try using avrogen -p switch in combination with protocol file, no joy.
> For this method, I first define the protocol file "bearstearns.avdl"
> @namespace("com.bearstearns")
> protocol BearStearns {
> import schema "simpleA.avsc";
> import schema "simpleB.avsc";
> }
> Then from command prompt:
> avrogen -p bearstearns.avdl outputDir
> This time, the error message is,
> Exception occurred. Invalid JSON format: @namespace("com.bearstearns")
> protocol BearStearns {
>         import schema "simpleA.avsc";
>         import schema "simpleB.avsc";
> }
> Actually is Avro support for dotnet ready yet? Avro official doc for C# is completely empty: http://avro.apache.org/docs/current/api/csharp/index.html
> Their guys even created a ticket for this in DEC 2013. https://issues.apache.org/jira/browse/AVRO-1420
> As of August 2015, Avro doc for C# remains empty. And not much foot print in Internet (C# and Avro I meant).
> References:
> • Official: https://avro.apache.org/docs/1.7.7/index.html 
> • Download utility avrogen.exe: http://apache.communilink.net/avro/avro-1.7.7/ 
> • http://www.tutorialspoint.com/avro/serialization_by_generating_class.htm 
> • http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/ 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)