Quantcast

Python Union Behavior not consistent

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Python Union Behavior not consistent

Gaurav
Hi,

I was using following schema to encode some data and found that while python integers are written as double, which is not correct. Atleast it should respect the order in which union is written.
{"type":"map","values":["int","long","float","double","string","boolean"]}


A small change in io.py can fix this:

  def write_union(self, writers_schema, datum, encoder):
    """
    A union is encoded by first writing a long value indicating
    the zero-based position within the union of the schema of its value.
    The value is then encoded per the indicated schema within the union.
    """
    # resolve union
    index_of_schema = -1
    for i, candidate_schema in enumerate(writers_schema.schemas):
      if validate(candidate_schema, datum):
        index_of_schema = i
        break
    if index_of_schema < 0: raise AvroTypeException(writers_schema, datum)

Please include this, if it makes sense.

Thanks,
Gaurav Nanda
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Union Behavior not consistent

Doug Cutting
Gaurav,

This looks like a good change.  Can you please file an issue in Jira?

Thanks,

Doug

On 12/08/2011 06:07 AM, Gaurav wrote:

> Hi,
>
> I was using following schema to encode some data and found that while python
> integers are written as double, which is not correct. Atleast it should
> respect the order in which union is written.
> {"type":"map","values":["int","long","float","double","string","boolean"]}
>
>
> A small change in io.py can fix this:
>
>   def write_union(self, writers_schema, datum, encoder):
>     """
>     A union is encoded by first writing a long value indicating
>     the zero-based position within the union of the schema of its value.
>     The value is then encoded per the indicated schema within the union.
>     """
>     # resolve union
>     index_of_schema = -1
>     for i, candidate_schema in enumerate(writers_schema.schemas):
>       if validate(candidate_schema, datum):
>         index_of_schema = i
> *        break*
>     if index_of_schema < 0: raise AvroTypeException(writers_schema, datum)
>
> Please include this, if it makes sense.
>
> Thanks,
> Gaurav Nanda
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Python-Union-Behavior-not-consistent-tp3570352p3570352.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Union Behavior not consistent

Gaurav
Logged AVRO#973.

On Sat, Dec 10, 2011 at 6:10 AM, Doug Cutting <[hidden email]> wrote:

> Gaurav,
>
> This looks like a good change.  Can you please file an issue in Jira?
>
> Thanks,
>
> Doug
>
> On 12/08/2011 06:07 AM, Gaurav wrote:
>> Hi,
>>
>> I was using following schema to encode some data and found that while python
>> integers are written as double, which is not correct. Atleast it should
>> respect the order in which union is written.
>> {"type":"map","values":["int","long","float","double","string","boolean"]}
>>
>>
>> A small change in io.py can fix this:
>>
>>   def write_union(self, writers_schema, datum, encoder):
>>     """
>>     A union is encoded by first writing a long value indicating
>>     the zero-based position within the union of the schema of its value.
>>     The value is then encoded per the indicated schema within the union.
>>     """
>>     # resolve union
>>     index_of_schema = -1
>>     for i, candidate_schema in enumerate(writers_schema.schemas):
>>       if validate(candidate_schema, datum):
>>         index_of_schema = i
>> *        break*
>>     if index_of_schema < 0: raise AvroTypeException(writers_schema, datum)
>>
>> Please include this, if it makes sense.
>>
>> Thanks,
>> Gaurav Nanda
>>
>> --
>> View this message in context: http://apache-avro.679487.n3.nabble.com/Python-Union-Behavior-not-consistent-tp3570352p3570352.html
>> Sent from the Avro - Users mailing list archive at Nabble.com.
Loading...