[jira] [Resolved] (AVRO-3037) py3 ipc.Responder swallows schema errors

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Resolved] (AVRO-3037) py3 ipc.Responder swallows schema errors

Dave Cole (Jira)

     [ https://issues.apache.org/jira/browse/AVRO-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael A. Smith resolved AVRO-3037.
    Resolution: Won't Fix

The avro-python3 implementation is unmaintained and avro (the implementation in lang/py) fully supports Python 3. Please open a new ticket if there are issues with the lang/py implementation.

> py3 ipc.Responder swallows schema errors
> ----------------------------------------
>                 Key: AVRO-3037
>                 URL: https://issues.apache.org/jira/browse/AVRO-3037
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: python
>    Affects Versions: 1.10.1
>            Reporter: Owen Healy
>            Priority: Minor
>              Labels: ipc, python3
> In the Python3 Avro IPC library, if the responder returns a response that doesn't match the schema, this error is swallowed and the response data is truncated. This results in a malformed response and a parsing error at the client end.
> An [example showing the problem is on GitHub|https://github.com/ellbur/avro-error-example/blob/main/main.py].
> The problem appears to come down to [these lines in ipc.py|https://github.com/apache/avro/blob/branch-1.10/lang/py3/avro/ipc.py#L393]:
> {code:java}
> except schema.AvroException as exn:
>     error = AvroRemoteException(str(exn))
>     buffer_encoder = avro_io.BinaryEncoder(io.StringIO())
>     META_WRITER.write(response_metadata, buffer_encoder)
>     buffer_encoder.write_boolean(True)
>     self._WriteError(SYSTEM_ERROR_SCHEMA, error, buffer_encoder)
> {code}
> I can think of two ways to fix this:
>  # Error on the server side; or
>  # Pass the error through to the client.
> Choice (1) seems to make logical sense since sending a non-conforming response is a bug in the server, not the client, but (2) might be easier to detect in real applications.
> The way to do (2) would be something like the following, which I am happy to make a pull request for if it would be helpful:
> {code:java}
> # write response using local protocol
> META_WRITER.write(response_metadata, buffer_encoder)
> saved_pos = buffer_writer.tell()
> buffer_encoder.write_boolean(error is not None)
> if error is None:
>     writers_schema = local_message.response
>     try:
>         self.write_response(writers_schema, response, buffer_encoder)
>     except schema.AvroException as e:
>         buffer_writer.seek(saved_pos)
>         buffer_writer.truncate(saved_pos)
>         buffer_encoder.write_boolean(True)
>         writers_schema = local_message.errors
>         self.write_error(writers_schema, e, buffer_encoder)
> else:
>     writers_schema = local_message.errors
>     self.write_error(writers_schema, error, buffer_encoder)
> {code}

This message was sent by Atlassian Jira