Quantcast

Apache Avro : UnresolvedUnionException when using union data type

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Apache Avro : UnresolvedUnionException when using union data type

andy36
I am using [Apache Avro][1] for object serialization.

I have one Avro schema for `School` object:

    {"namespace": "com.my.model",
     "type": "record",
     "name": "School",
     "fields": [
         {"name": "sid",  "type": "int"},
         {"name": "size",  "type": "long"},
         {"name": "other", "type": ["null", "Teacher", "Student"]}
       ]
    }

As you see above, the `"other"` field holds an **union data type**, it could be either `null` or a `Teacher` instance or a `Student` instance.

The **Teacher** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Teacher",
      "fields": [
    {"name": "isMale", "type": "boolean"}
    ]
    }

The **Student** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Student",
      "fields": [
    {"name": "age", "type": "int"}
    ]
    }

I compiled above schemas with Avro tool & Avro generated all the Java classes for me automatically.

Then, in my java program I am creating `School` instance in the following way:

    School school = new School();
    school.setSid(3);
    school.setSize(2000);
   
    //create a student object
    Student student = new Student();
    student.setAge(18);
   
    //set student into school instance
    school.setOther(student);

As you see above, the school instance's `other` field holds a student object. When I compile my code, however I got the **UnresolvedUnionException** . It complains about the **union data type** for the `other` field of `School` schema. Seems it can not resolve the `student` I set to `school` in my Java code. Why is this exception ? Stacktrace is:

    org.apache.avro.UnresolvedUnionException: Not in union ["null",{"type":"record","name":"Student","namespace":"com.my.model","fields":[{"name":"age","type":"long"}]},{"type":"record","name":"Teacher","namespace":"com.model","fields":[{"name":"isMale","type":"boolean"}]}]: false
    at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:561)
    at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:144)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
    at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
    at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:131)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
    at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
    at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)


  [1]: http://avro.apache.org/docs/1.7.4/gettingstartedjava.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Avro : UnresolvedUnionException when using union data type

Lewis John Mcgibbney
Hi,
I've never experienced this before but I would suggestb setting a default value for the UNION  {"name": "other", "type": ["null", "Teacher", "Student"]}
Lewis


On Mon, Dec 2, 2013 at 8:00 PM, andy36 <[hidden email]> wrote:
I am using [Apache Avro][1] for object serialization.

I have one Avro schema for `School` object:

    {"namespace": "com.my.model",
     "type": "record",
     "name": "School",
     "fields": [
         {"name": "sid",  "type": "int"},
         {"name": "size",  "type": "long"},
         {"name": "other", "type": ["null", "Teacher", "Student"]}
       ]
    }

As you see above, the `"other"` field holds an **union data type**, it could
be either `null` or a `Teacher` instance or a `Student` instance.

The **Teacher** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Teacher",
        "fields": [
                {"name": "isMale", "type": "boolean"}
        ]
    }

The **Student** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Student",
        "fields": [
                {"name": "age", "type": "int"}
        ]
    }

I compiled above schemas with Avro tool & Avro generated all the Java
classes for me automatically.

Then, in my java program I am creating `School` instance in the following
way:

    School school = new School();
    school.setSid(3);
    school.setSize(2000);

    //create a student object
    Student student = new Student();
    student.setAge(18);

    //set student into school instance
    school.setOther(student);

As you see above, the school instance's `other` field holds a student
object. When I compile my code, however I got the
**UnresolvedUnionException** . It complains about the **union data type**
for the `other` field of `School` schema. Seems it can not resolve the
`student` I set to `school` in my Java code. Why is this exception ?
Stacktrace is:

    org.apache.avro.UnresolvedUnionException: Not in union
["null",{"type":"record","name":"Student","namespace":"com.my.model","fields":[{"name":"age","type":"long"}]},{"type":"record","name":"Teacher","namespace":"com.model","fields":[{"name":"isMale","type":"boolean"}]}]:
false
        at
org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:561)
        at
org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:144)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
        at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:131)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
        at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)


  [1]: http://avro.apache.org/docs/1.7.4/gettingstartedjava.html



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Apache-Avro-UnresolvedUnionException-when-using-union-data-type-tp4028641.html
Sent from the Avro - Users mailing list archive at Nabble.com.



--
Lewis
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Avro : UnresolvedUnionException when using union data type

Martin Kleppmann
Hi Andy,

Can you check that you are using namespaces consistently? I noticed in your exception message that the Teacher record is in namespace com.model, but your schema above declares it as being in namespace com.my.model.

This suggests that perhaps something is broken in your build process — for example, perhaps you are using code generated from an old version of the schema?

Martin


On 2 December 2013 20:06, Lewis John Mcgibbney <[hidden email]> wrote:
Hi,
I've never experienced this before but I would suggestb setting a default value for the UNION  {"name": "other", "type": ["null", "Teacher", "Student"]}
Lewis


On Mon, Dec 2, 2013 at 8:00 PM, andy36 <[hidden email]> wrote:
I am using [Apache Avro][1] for object serialization.

I have one Avro schema for `School` object:

    {"namespace": "com.my.model",
     "type": "record",
     "name": "School",
     "fields": [
         {"name": "sid",  "type": "int"},
         {"name": "size",  "type": "long"},
         {"name": "other", "type": ["null", "Teacher", "Student"]}
       ]
    }

As you see above, the `"other"` field holds an **union data type**, it could
be either `null` or a `Teacher` instance or a `Student` instance.

The **Teacher** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Teacher",
        "fields": [
                {"name": "isMale", "type": "boolean"}
        ]
    }

The **Student** object schema:

    {"namespace": "com.my.model",
    "type": "record",
    "name": "Student",
        "fields": [
                {"name": "age", "type": "int"}
        ]
    }

I compiled above schemas with Avro tool & Avro generated all the Java
classes for me automatically.

Then, in my java program I am creating `School` instance in the following
way:

    School school = new School();
    school.setSid(3);
    school.setSize(2000);

    //create a student object
    Student student = new Student();
    student.setAge(18);

    //set student into school instance
    school.setOther(student);

As you see above, the school instance's `other` field holds a student
object. When I compile my code, however I got the
**UnresolvedUnionException** . It complains about the **union data type**
for the `other` field of `School` schema. Seems it can not resolve the
`student` I set to `school` in my Java code. Why is this exception ?
Stacktrace is:

    org.apache.avro.UnresolvedUnionException: Not in union
["null",{"type":"record","name":"Student","namespace":"com.my.model","fields":[{"name":"age","type":"long"}]},{"type":"record","name":"Teacher","namespace":"com.model","fields":[{"name":"isMale","type":"boolean"}]}]:
false
        at
org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:561)
        at
org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:144)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71)
        at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at
org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:131)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
        at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:106)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
        at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)


  [1]: http://avro.apache.org/docs/1.7.4/gettingstartedjava.html



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Apache-Avro-UnresolvedUnionException-when-using-union-data-type-tp4028641.html
Sent from the Avro - Users mailing list archive at Nabble.com.



--
Lewis

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Avro : UnresolvedUnionException when using union data type

andy36
Hi,Martin,
THanks for the suggestion . However, that is just my typo in my post, I am using consistent name space in my schema.

Andy
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Apache Avro : UnresolvedUnionException when using union data type

Martin Kleppmann
Can you post a minimal example program / test case which reproduces the problem? Hard to diagnose otherwise.

Martin


On 4 December 2013 07:54, andy36 <[hidden email]> wrote:
Hi,Martin,
THanks for the suggestion . However, that is just my typo in my post, I am
using consistent name space in my schema.

Andy



--
View this message in context: http://apache-avro.679487.n3.nabble.com/Apache-Avro-UnresolvedUnionException-when-using-union-data-type-tp4028641p4028667.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Loading...