Using optional fields (nullable unions) with the C-API

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Using optional fields (nullable unions) with the C-API

Markus Resch
Hey avro users,

I'm using a record like this

{
    "type" : "record",
    "name" : "TestPacket",
    "fields" : [
        { "name" : "Id" , "type" : "int"},
        { "name" : "Value" , "type" : ["string","null"]}
        ]
}

But if I just not write anything to the 'Value' filed,
avro_writer_memory() refuses to validate it. How do you normally use the
optional fields with the C-API?

Regards,

        Markus


--
Markus Resch
Software Developer
P: +49 6103-5715-0 | F: +49 6103-5715-111 |
ADTECH AG | Robert-Bosch-Str. 32 | 63303 Dreieich | Germany
www.adtech.com<http://www.adtech.com/>


ADTECH | A Division of Advertising.com Group - Residence of the Company:
Dreieich, Germany - Registration Office Offenbach HRB 34657
Chairman of the Board: Erhard Neumann - Chairman of the Supervisory
Board: Ned Brody

This message contains privileged and confidential information. Any
dissemination, distribution, copying or other use of this
message or any of its content (data, prices...) to any third parties may
only occur with ADTECH's prior consent.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using optional fields (nullable unions) with the C-API

Douglas Creager
> I'm using a record like this
>
> {
>    "type" : "record",
>    "name" : "TestPacket",
>    "fields" : [
>        { "name" : "Id" , "type" : "int"},
>        { "name" : "Value" , "type" : ["string","null"]}
> ]
> }
>
> But if I just not write anything to the 'Value' filed,
> avro_writer_memory() refuses to validate it. How do you normally use the
> optional fields with the C-API?

Hi Markus

In Avro, an “optional” field is really just a common pattern where you've got a union, with a “null” for one branch, and some other type for the second branch.  And with all union types, you have to explicitly tell the Avro library which branch is active for a particular value.  (Think of the union as a container that must always contain exactly one value, but where the type of that value can change.)

So, if you're using the new value API, you'd use something like:

  avro_value_t  rec = /* given */;
  avro_value_t  field, branch;
  avro_value_get_by_name(rec, "Value", &field, NULL);
  avro_value_set_branch(&field, 1, &branch);
  avro_value_set_null(&branch);

and if you're using the old datum API:

  avro_datum_t  rec = /* given */;
  avro_datum_t  field, branch;
  avro_record_get(rec, "Value", &field);
  avro_union_set_discriminant(field, 1, &branch);
  /* no avro_null_set in the old API */

In set_branch/set_discriminant, the 1 refers to the index of the "null" type in the union.  I tend to use branch 0 for the null type in these kinds of unions, but that's just a matter of aesthetics.

cheers
–doug


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using optional fields (nullable unions) with the C-API

adityanahan
This post has NOT been accepted by the mailing list yet.
Thanks for this example.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using optional fields (nullable unions) with the C-API

adityanahan
This post has NOT been accepted by the mailing list yet.
This post was updated on .
In reply to this post by Douglas Creager
Hi Doug,

I thought i was using it correctly, but running into issues of using the union with nullable fields.
Doing something like:

   acos_avro_check_i(avro_value_get_by_name(&writer_value, "port_number", &field, NULL));
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch));
    acos_avro_check_i(avro_value_set_null(&branch));
    acos_avro_check_i(avro_value_set_int(&branch, 80));
    /*protocol*/
    acos_avro_check_i(avro_value_get_by_name(&writer_value, "protocol", &field, NULL));
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch));
    acos_avro_check_i(avro_value_set_null(&branch));
    acos_avro_check_i(avro_value_set_string(&branch, "tcp"));

Posting it.

But getting error when i try to set the value for say set_int or set_string.
I can't seem to understand why it is failing, however it did work before.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using optional fields (nullable unions) with the C-API

adityanahan
Posting again.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Using optional fields (nullable unions) with the C-API

adityanahan
In reply to this post by Markus Resch
Hi guys,

I am facing problem in encoding the nullable fields.


Hi Doug,

I thought i was using it correctly, but running into issues of using the union with nullable fields.
Doing something like:

Schema: This is just snippet, the whole schema is not posted.

{
            "name": "protocol",
            "type": [
                "string",
                "null"
            ]
        },
        {
            "name": "port_number",
            "type": [
                "long",
                "null"
            ],
            "doc": "Port"
        },
}
   acos_avro_check_i(avro_value_get_by_name(&writer_value, "port_number", &field, NULL));
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch));
    acos_avro_check_i(avro_value_set_null(&branch));
    acos_avro_check_i(avro_value_set_int(&branch, 80));
    /*protocol*/
    acos_avro_check_i(avro_value_get_by_name(&writer_value, "protocol", &field, NULL));
    acos_avro_check_i(avro_value_set_branch(&field, 1, &branch));
    acos_avro_check_i(avro_value_set_null(&branch));
    acos_avro_check_i(avro_value_set_string(&branch, "tcp"));

Posting it.

But getting error when i try to set the value for say set_int or set_string.
I can't seem to understand why it is failing.
Is this correct way to encode a union field or are there some better examples i can look at ?
Loading...