Avro error "org.apache.avro.AvroTypeException: Expected field name not found"

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Avro error "org.apache.avro.AvroTypeException: Expected field name not found"

Manish Mehndiratta
Hi Team,

I stripped out my avro schema file and json file to only one element and yet it continues to give me the same error. 

Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID

--- avsc file ---
{
  "type" : "record",
  "name" : "DataModel",
  "fields" : [ 
{ "name" : "TMS_ID", "type" : "string", "default" : "NONE" }
]
}

--- json file ---
{
    "DataModel" :{
"TMS_ID" : "asdf"
}
}

--- avro validation query ---
java -jar avro-tools-1.8.1.jar fromjson --schema-file TMSDataModelFlat.avsc 90573445988483.json > validated.avro

--- error ---
Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID
        at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:477)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:139)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:219)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:214)
        at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:430)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:99)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avro error "org.apache.avro.AvroTypeException: Expected field name not found"

Elliot West
Try:

{
  "TMS_ID" : "asdf"
}

On Wed, 9 Aug 2017 at 19:22, Manish Mehndiratta <[hidden email]> wrote:
Hi Team,

I stripped out my avro schema file and json file to only one element and yet it continues to give me the same error. 

Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID

--- avsc file ---
{
  "type" : "record",
  "name" : "DataModel",
  "fields" : [ 
{ "name" : "TMS_ID", "type" : "string", "default" : "NONE" }
]
}

--- json file ---
{
    "DataModel" :{
"TMS_ID" : "asdf"
}
}

--- avro validation query ---
java -jar avro-tools-1.8.1.jar fromjson --schema-file TMSDataModelFlat.avsc 90573445988483.json > validated.avro

--- error ---
Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID
        at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:477)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:139)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:219)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:214)
        at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:430)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:99)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avro error "org.apache.avro.AvroTypeException: Expected field name not found"

Manish Mehndiratta
Thank you, Elliot. This worked.
If my json needs to include "DataModel":, then what changes do I need to make in my avsc file?

Thanks,
Manish

On Wed, Aug 9, 2017 at 1:46 PM, Elliot West <[hidden email]> wrote:
Try:

{
  "TMS_ID" : "asdf"
}

On Wed, 9 Aug 2017 at 19:22, Manish Mehndiratta <[hidden email]> wrote:
Hi Team,

I stripped out my avro schema file and json file to only one element and yet it continues to give me the same error. 

Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID

--- avsc file ---
{
  "type" : "record",
  "name" : "DataModel",
  "fields" : [ 
{ "name" : "TMS_ID", "type" : "string", "default" : "NONE" }
]
}

--- json file ---
{
    "DataModel" :{
"TMS_ID" : "asdf"
}
}

--- avro validation query ---
java -jar avro-tools-1.8.1.jar fromjson --schema-file TMSDataModelFlat.avsc 90573445988483.json > validated.avro

--- error ---
Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID
        at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:477)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:139)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:219)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:214)
        at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:430)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:99)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avro error "org.apache.avro.AvroTypeException: Expected field name not found"

Elliot West
Hi,

In that case you will need a field called 'DataModel' whose type is 'DataModel':

{
  "type" : "record",
  "name" : "ParentType",
  "fields" : [ 
    "name" : "DataModel",
    "type" : {
      "type" : "record",
      "name" : "DataModel",
      "fields" : [ 
  { "name" : "TMS_ID", "type" : "string", "default" : "NONE" }
  ]
    }
  ]
}

On 9 August 2017 at 20:46, Manish Mehndiratta <[hidden email]> wrote:
Thank you, Elliot. This worked.
If my json needs to include "DataModel":, then what changes do I need to make in my avsc file?

Thanks,
Manish

On Wed, Aug 9, 2017 at 1:46 PM, Elliot West <[hidden email]> wrote:
Try:

{
  "TMS_ID" : "asdf"
}

On Wed, 9 Aug 2017 at 19:22, Manish Mehndiratta <[hidden email]> wrote:
Hi Team,

I stripped out my avro schema file and json file to only one element and yet it continues to give me the same error. 

Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID

--- avsc file ---
{
  "type" : "record",
  "name" : "DataModel",
  "fields" : [ 
{ "name" : "TMS_ID", "type" : "string", "default" : "NONE" }
]
}

--- json file ---
{
    "DataModel" :{
"TMS_ID" : "asdf"
}
}

--- avro validation query ---
java -jar avro-tools-1.8.1.jar fromjson --schema-file TMSDataModelFlat.avsc 90573445988483.json > validated.avro

--- error ---
Exception in thread "main" org.apache.avro.AvroTypeException: Expected field name not found: TMS_ID
        at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:477)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:139)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:219)
        at org.apache.avro.io.JsonDecoder.readString(JsonDecoder.java:214)
        at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:430)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:99)
        at org.apache.avro.tool.Main.run(Main.java:87)
        at org.apache.avro.tool.Main.main(Main.java:76)




Loading...