[jira] [Commented] (AVRO-3146) Test scale validation when encoding decimal logical schema

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (AVRO-3146) Test scale validation when encoding decimal logical schema

Dave Cole (Jira)

    [ https://issues.apache.org/jira/browse/AVRO-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17350132#comment-17350132 ]

ASF subversion and git services commented on AVRO-3146:

Commit c8f953bc5e6599b90283992b643bdc91adbef84b in avro's branch refs/heads/master from Michael A. Smith
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=c8f953b ]

AVRO-3146: Fix Decimal Scale Exception Handling (#1228)

Avro is supposed to check if the scale for a decimal logical schema is large enough for the datum to be written. But Python provides the exponent as negative, so the exponent is always less than the scale. This change corrects the comparison by ensuring scale and exponent have the same sign.

In testing the above, avro attempts to raise AvroTypeException with improper arguments, leading to a secondary exception within the first. This change also fixes AvroTypeException so that it is flexible and won't crash when invoked with different arguments.

It also adds a special AvroTypeException subtype to provide useful debugging details when a decimal schema tries to write a datum with an exponent that is too large.

Finally, many test cases in Avro consist of running the same code over variations on pairs of schema and datum. But unittest exits at the first test failure! To get a comprehensive view of tests, even when some fail, I have refactored test_io so that individual test methods are generated over the set of test cases.

> Test scale validation when encoding decimal logical schema
> ----------------------------------------------------------
>                 Key: AVRO-3146
>                 URL: https://issues.apache.org/jira/browse/AVRO-3146
>             Project: Apache Avro
>          Issue Type: Test
>          Components: python
>    Affects Versions: 1.10.2
>            Reporter: Michael A. Smith
>            Assignee: Michael A. Smith
>            Priority: Major
> Based on [a CodeQL finding|https://github.com/apache/avro/security/code-scanning/464], it seems decimal logical types are probably broken and need testing with various encoding and decoding steps. I think:
> 1. Python decimal exponents are always negative, and avro schema scales are always positive, so the {{if exp > scale: raise}} check does not raise an exception.
> 2. If it did, it would raise the wrong exception, because it would fail to instantiate the exception class.

This message was sent by Atlassian Jira