Quantcast

Type cast exception

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Type cast exception

Jaydeep Ayachit

Hello,

 

I am trying to read from avro data file and want to use specific object (exa sample.LogMessage)

The following code gives exception mentioned below. Reader.next() when collected in Object/Record, no exception is thrown which is expected.

 

Any help is appreciated.

 

public void testMethod5() throws Exception {

            DatumWriter<LogMessage> writer = new GenericDatumWriter<LogMessage>(LogMessage.SCHEMA$);

            DataFileWriter<LogMessage> out = new DataFileWriter<LogMessage>(writer);

 

            out.create(LogMessage.SCHEMA$, <File>);

            for (int i = 0; i < 100; i++) {

                  LogMessage msg = new LogMessage();

                  msg.applicationName = new Utf8("app" + i);

                  msg.userid = new Utf8("user" + i);

                  msg.message = new Utf8("this is test log message " + i);

                 

                  out.append(msg);

            }          

            out.close();

           

            DataFileReader<LogMessage> reader;

            GenericDatumReader<LogMessage> din = new GenericDatumReader<LogMessage>(LogMessage.SCHEMA$);

            reader = new DataFileReader<LogMessage>(new File(<>), din);

           

            while (reader.hasNext()) {

                  LogMessage msg = reader.next();

                  System.out.println(msg.toString());

            }

            reader.close();

      }

 

Exception

 

java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to sample.LogMessage

      at com.ibm.clas.ReaderWriter.testMethod5(ReaderWriter.java:313)

      at com.ibm.clas.ReaderWriter.main(ReaderWriter.java:322)

 

 

Thanks

Jaydeep

 

DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

Scott Carey
<base href="x-msg://67/">Try 

SpecificDatumReader<LogMessage> din = new SpecificDatumReader<LogMessage>(LogMessage.class);

instead of the GenericDatumReader below.


On Apr 26, 2010, at 7:59 AM, Jaydeep Ayachit wrote:

Hello,
 
I am trying to read from avro data file and want to use specific object (exa sample.LogMessage)
The following code gives exception mentioned below. Reader.next() when collected in Object/Record, no exception is thrown which is expected.
 
Any help is appreciated.
 
public void testMethod5() throws Exception {
            DatumWriter<LogMessage> writer = new GenericDatumWriter<LogMessage>(LogMessage.SCHEMA$);
            DataFileWriter<LogMessage> out = new DataFileWriter<LogMessage>(writer);
 
            out.create(LogMessage.SCHEMA$, <File>);
            for (int i = 0; i < 100; i++) {
                  LogMessage msg = new LogMessage();
                  msg.applicationName = new Utf8("app" + i);
                  msg.userid = new Utf8("user" + i);
                  msg.message = new Utf8("this is test log message " + i);
                 
                  out.append(msg);
            }          
            out.close();
           
            DataFileReader<LogMessage> reader;
            GenericDatumReader<LogMessage> din = new GenericDatumReader<LogMessage>(LogMessage.SCHEMA$);
            reader = new DataFileReader<LogMessage>(new File(<>), din);
           
            while (reader.hasNext()) {
                  LogMessage msg = reader.next();
                  System.out.println(msg.toString());
            }
            reader.close();
      }
 
Exception
 
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to sample.LogMessage
      at com.ibm.clas.ReaderWriter.testMethod5(ReaderWriter.java:313)
      at com.ibm.clas.ReaderWriter.main(ReaderWriter.java:322)
 
 
Thanks
Jaydeep
 

DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

TrevniUser
I am facing this same issue while using SortedKeyValueFile. Could you please provide some pointers why in the first place is GenericRecord being picked up and not SpecificRecord?

@Test
    public void testSortedKeyValueFile() throws IOException {
        Path filePath = new Path(test.getMethodName());

        Schema keySchema = ReflectData.get().getSchema(Long.class);
        Schema valSchema = ReflectData.get().getSchema(DataSinkChangeLogContent.class);

        SortedKeyValueFile.Writer.Options opts = new Options();
        opts.withKeySchema(keySchema);
        opts.withValueSchema(valSchema);
        opts.withConfiguration(conf);
        opts.withPath(filePath);
        SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer = new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
                opts);
        for (long i = 0; i < 10; i++) {
            writer.append(i, new DataSinkChangeLogContent(eType.toString(), eKey.toString(), i, 15L));
        }
        writer.close();

        SortedKeyValueFile.Reader.Options readerOpts = new SortedKeyValueFile.Reader.Options();
        readerOpts.withKeySchema(keySchema);
        readerOpts.withValueSchema(valSchema);
        readerOpts.withConfiguration(conf);
        readerOpts.withPath(filePath);
        SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader = new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
                readerOpts);
        Iterator<AvroKeyValue<Long, DataSinkChangeLogContent>> i = null;
        if (reader.get(5L) != null) {
            i = reader.iterator();
        }
        while (i != null && i.hasNext()) {
            AvroKeyValue<Long, DataSinkChangeLogContent> keyValue = i.next();
            System.out.println("---------------------------- Clock: " + keyValue.getValue());
        }
        DataSinkChangeLogContent e = reader.get(5L); // FAILS AT THIS LINE
        reader.close();
    }

EXCEPTION:

java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
        at com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

Doug Cutting
Could this be AVRO-1240?  What version of Avro are you using?

https://issues.apache.org/jira/browse/AVRO-1240

Doug

On Tue, Jul 2, 2013 at 10:05 AM, TrevniUser <[hidden email]> wrote:

> I am facing this same issue while using SortedKeyValueFile. Could you please
> provide some pointers why in the first place is GenericRecord being picked
> up and not SpecificRecord?
>
> @Test
>     public void testSortedKeyValueFile() throws IOException {
>         Path filePath = new Path(test.getMethodName());
>
>         Schema keySchema = ReflectData.get().getSchema(Long.class);
>         Schema valSchema =
> ReflectData.get().getSchema(DataSinkChangeLogContent.class);
>
>         SortedKeyValueFile.Writer.Options opts = new Options();
>         opts.withKeySchema(keySchema);
>         opts.withValueSchema(valSchema);
>         opts.withConfiguration(conf);
>         opts.withPath(filePath);
>         SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent> writer =
> new SortedKeyValueFile.Writer<Long, DataSinkChangeLogContent>(
>                 opts);
>         for (long i = 0; i < 10; i++) {
>             writer.append(i, new DataSinkChangeLogContent(eType.toString(),
> eKey.toString(), i, 15L));
>         }
>         writer.close();
>
>         SortedKeyValueFile.Reader.Options readerOpts = new
> SortedKeyValueFile.Reader.Options();
>         readerOpts.withKeySchema(keySchema);
>         readerOpts.withValueSchema(valSchema);
>         readerOpts.withConfiguration(conf);
>         readerOpts.withPath(filePath);
>         SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent> reader =
> new SortedKeyValueFile.Reader<Long, DataSinkChangeLogContent>(
>                 readerOpts);
>         Iterator<AvroKeyValue&lt;Long, DataSinkChangeLogContent>> i = null;
>         if (reader.get(5L) != null) {
>             i = reader.iterator();
>         }
>         while (i != null && i.hasNext()) {
>             AvroKeyValue<Long, DataSinkChangeLogContent> keyValue =
> i.next();
>             System.out.println("---------------------------- Clock: " +
> keyValue.getValue());
>         }
> *        DataSinkChangeLogContent e = reader.get(5L);* // FAILS AT THIS LINE
>         reader.close();
>     }
>
> EXCEPTION:
>
> java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record
> cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
>         at
> com.abc.kepler.datasink.hdfs.HDFSDataSinkChangeLogScannerTest.testSortedKeyValueFile(HDFSDataSinkChangeLogScannerTest.java:926)
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027727.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

TrevniUser
Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what my project dependencies look like:

[INFO] +- org.apache.avro:avro:jar:1.7.3:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
[INFO] |  +- org.apache.avro:avro-ipc:jar:1.7.3:compile
[INFO] |  \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

Doug Cutting
This was fixed in Avro 1.7.4.  Please try upgrading and see if that helps.

Doug

On Tue, Jul 2, 2013 at 10:33 AM, TrevniUser <[hidden email]> wrote:

> Seems like it could be. Please confirm. I am using 'avro 1.7.3'. This is what
> my project dependencies look like:
>
> [INFO] +- org.apache.avro:avro:jar:1.7.3:compile
> [INFO] +- org.apache.avro:avro-mapred:jar:1.7.3:compile
> [INFO] |  +- org.apache.avro:avro-ipc:jar:1.7.3:compile
> [INFO] |  \- org.apache.avro:avro-ipc:jar:tests:1.7.3:compile
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Type-cast-exception-tp756884p4027730.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

TrevniUser
Updated but still facing the same issue. Any ideas why?

Updated project dependencies. Now pulling in:
[INFO] +- org.apache.avro:avro:jar:1.7.4:compile
[INFO] +- org.apache.avro:avro-mapred:jar:1.7.4:compile
[INFO] |  +- org.apache.avro:avro-ipc:jar:1.7.4:compile
[INFO] |  \- org.apache.avro:avro-ipc:jar:tests:1.7.4:compile

EXCEPTION:
java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to com.abc.kepler.datasink.DataSinkChangeLogContent
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

TrevniUser
For now, as a work-around, we are thinking of switching to directly using the schema instead of the generated class file. So, something like this:

Schema schema=Schema.parse(getClass().getResourceAsStream("ChangeLogContent.avsc"));

Its still unresolved why avro 1.7.4 couldn't fix this issue though.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

Doug Cutting
In reply to this post by TrevniUser
On Tue, Jul 2, 2013 at 10:45 AM, TrevniUser <[hidden email]> wrote:
> Updated but still facing the same issue. Any ideas why?

I just noticed that SortedKeyValueFile uses GenericDatumReader and
GenericDatumWriter, when it needs to use SpecificDatumReader and
SpecificDatumWriter to be able to return specific records.  I filed
AVRO-1351 to address this.

https://issues.apache.org/jira/browse/AVRO-1351

Doug
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Type cast exception

TrevniUser
Thanks, Doug. I will track that issue for further updates.
Loading...