Quantcast

how to Convert the input data to Avro files in HDFS

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

how to Convert the input data to Avro files in HDFS

rajharireddy
Create the avro schemas for inputs and outputs and Convert the below txt file to avro file using Python

txt file containing
1631225499|10185|0.998823
6631225541|10185|0.99606
6631225542|10185|0.99606
4630445085|23101|1
6630175441|33064|0.995986
7630175441|33064|0.996005
10631275999|42492|0.999987
8631275999|42492|0.978484
6631275999|42492|0.974801
13631275999|42492|0.974801
5631175441|43894|0.996875
3631175441|43894|0.996638
6630527399|53715|0.998816
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: how to Convert the input data to Avro files in HDFS

Harsh J-2
Are you requesting we do that for you or are you looking for tools
that will help you do that?

Either way, check out the older "avrocat" or the newer "avro" tool
included as part of Apache Avro from 1.6 onwards
(https://issues.apache.org/jira/browse/AVRO-836). It has sub-tools
letting you transform a CSV form of file into Avro formats.

On Fri, Mar 8, 2013 at 3:18 PM, rajharireddy <[hidden email]> wrote:

> Create the avro schemas for inputs and outputs and Convert the below txt file
> to avro file using Python
>
> txt file containing
> 1631225499|10185|0.998823
> 6631225541|10185|0.99606
> 6631225542|10185|0.99606
> 4630445085|23101|1
> 6630175441|33064|0.995986
> 7630175441|33064|0.996005
> 10631275999|42492|0.999987
> 8631275999|42492|0.978484
> 6631275999|42492|0.974801
> 13631275999|42492|0.974801
> 5631175441|43894|0.996875
> 3631175441|43894|0.996638
> 6630527399|53715|0.998816
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/how-to-Convert-the-input-data-to-Avro-files-in-HDFS-tp4026548.html
> Sent from the Avro - Developers mailing list archive at Nabble.com.



--
Harsh J
Loading...