SparkSql: Left Outer Join With DSL

SparkSql: Left Outer Join With DSL

Spark SQL is great at executing SQL but sometimes you want to stick to the RDD level. Then comes the role of DSL. Sometimes how exactly to use Spark with DSL becomes confusing. This recipe is an attempt to reduce that. Let’s get done with pleasantries first,...
Spark SQL: SchemaRDD: Programmatically Specifying Schema

Spark SQL: SchemaRDD: Programmatically Specifying Schema

This recipe is inspired by http://spark.apache.org/docs/latest/sql-programming-guide.html#programmatically-specifying-the-schema and all rights are owned by their respective owners. In Spark SQL, the best way to create SchemaRDD is by using scala case class. Spark...
Top