Spark Submit With SBT

Spark Submit With SBT

This recipe assumes sbt is installed and you have already gone over mysql with Spark recipe. I am a big fan of Spark Shell. Biggest proof is Spark Cookbook which has all recipes in the form of collection of single commands on Spark Shell. It makes it easy to...
Spark: Accumulators

Spark: Accumulators

Accumulators are Spark’s answer to MapReduce counters but they do much more than that. Let’s first start with a simple example which uses accumulators for simple count. scala> val ac = sc.accumulator(0) scala> val words =...
Top