Spark: DataFrames And JDBC

Spark: DataFrames And JDBC

For Spark 1.3 onward, JdbcRDD is not recommended as DataFrames have support to load JDBC. Let us look at a simple example in this recipe. Using JdbcRDD with Spark is slightly confusing, so I thought about putting a simple use case to explain the functionality. Most...
Spark: DataFrames And Parquet

Spark: DataFrames And Parquet

This recipe works with Spark 1.3 onward. Apache Parquet as a file format has garnered significant attention recently. Let’s say you have a table with 100 columns, most of the time you are going to access 3-10 columns. In Row oriented format all columns are...
Top