Data Science and Machine Learning with Scala and Spark (Episode 01/03) | by MA Raza, Ph.D. | Towards Data Science

Scala is the core language to be used in writing the most popular distributed big data processing framework apache Spark. Big Data processing is becoming inevitable from small to large enterprises. The amount of data being produced is increasing by every second.
Extracting the valuable insights from data requires state of the art processing tools and frameworks. Researchers are working hard to develop application-specific tools, programming languages, and frameworks. Scala is one of those inventions to leverage big data processing.
Scala programming language is build to implement scale able solutions to crunch big data in order to produce actionable insights.
In this series, I will develop a series of tutorials in Google colab to learn Scala and apply for data science and machine learning applications. Python is considered to be the most common programming language for Data Science and machine learning. However, when it comes to processing large data, Data scientists should know alternatives such as Scala.
Who should follow these tutorials?
If you are a data scientist with a Pythonic approach and want to add another big data processing tool, this series is for you.
I have structured the series into three parts. In this story, I will cover the basics of the Scala programming language. We will cover spark with Scala in the next article and finish the series with Machine Learning with Spark and Scala. For each of the stories, there will be associated with google colab with information and practice codes.
In the first story, you will learn how to install or set up Scala in Google colab, most important data types, Scala expressions, Scala Functions, and Scala Objects.
While working with python, it is not required to specify the variables' types however Scala require you to specify the data type of variables. Therefore it makes more sense of understanding and practicing the data types implemented in Scala. Below are the most basic data types implemented in Scala. You can do practice using google Colab.
Scala Data Types: Int, Char, Float, Double, Long, Short, BooleanScala Collections: Arrays, VectorsScala Maps: Key-value pairs
Scala Data Types: Int, Char, Float, Double, Long, Short, Boolean
Scala Collections: Arrays, Vectors
Scala Maps: Key-value pairs
As with other programming languages, Scala expressions follow a similar approach with numerical, Boolean, and logical expressions. Some common and basic expressions are including in the Google colab notebook.
// Syntaxval <identifier>[: <type>] = <expression>var <identifier>[: <type>] = <expression>// Comment out one lineprintln("Hello Scala")// sample expressionvar x = 2 * Math.sqrt(10) / 5
Scala is an objected oriented programming language and you can write functions pretty much following the same style. below is the syntax of defining a function in Scala. You can practice the real examples through google colab notebook accompanying this blog.
def <identifier>(<identifier>: <type>[, ... ]): <type> = <expression>
One example of a function to multiply two integers with integer output is given below.
def multiplier(x: Int, y: Int): Int = { x * y }
Classes are considered to be very helpful in object-oriented programming and it is true for Scala too. Working with big data and complex code structures, classes make things simpler for programmers with Scala. Like other languages, you can write classes in Scala. Below are some examples. A minimal class example is below
class Userval user1 = new User
Tour of Scala Classes is a quick guide to write classes in Scala. More examples are given in google colab notebook.
Like other languages, you can import native as well as third party packages in Scala. the syntax is pretty much the same across java and python.
import Array.
The key objective of Scala is to work with large data sets. To achieve that Scala has a very clean solution of processing in parallel with minimal code change. If you are coming from Python, multiprocessing is hard to set up and often results in unintended results and failures. Scala has much better support of multiprocessing. Among other operations, filter and map are the most commonly used when processing in parallel. Follow some examples below.
I have prepared a functional Google colab notebook. Feel free to use the notebook for practice.
In this tutorial, we will learn data science and machine learning with scala.
SCALA is functional and objecte oriented progarmming running with JVM with aim to be scalable hence name as SCALA.
Spark is written in scala which makes scala a native choice for processing big data. We will also cover the basic of using scala with spark and machine learning using spark and scala.
Key Objectives

Install Scala to work with Google Colab
Introduction and basics of Scala
Scala for Data Science
Scala with Spark Introduction
Machine Learning with Scala

Install Scala to work with Google Colab
Introduction and basics of Scala
Scala for Data Science
Scala with Spark Introduction
Machine Learning with Scala
Above is the bigger scope of this tutorial. The aim is to provide a working google colab notebook for all above sections
I followed the tutorial Machine Learning with Scala in Google Colaboratory by Shadad Laddad to start Scala kernel in the google colab.
If you get a "scala" kernel not recognized warning when loading up the notebook for the first time, start by running the two cells below. Once you are done reload the page to load the notebook in the installed Scala kernel.
%%shell
SCALA_VERSION=2.12.8 ALMOND_VERSION=0.3.0+16-548dc10f-SNAPSHOT
curl -Lo coursier https://git.io/coursier-cli
chmod +x coursier
./coursier bootstrap \
    -r jitpack -r sonatype:snapshots \
    -i user -I user:sh.almond:scala-kernel-api_$SCALA_VERSION:$ALMOND_VERSION \
    sh.almond:scala-kernel_$SCALA_VERSION:$ALMOND_VERSION \
    --sources --default=true \
    -o almond-snapshot --embed-files=false
rm coursier
./almond-snapshot --install --global --force
rm almond-snapshot

%%shell
echo "{
  \"language\" : \"scala\",
  \"display_name\" : \"Scala\",
  \"argv\" : [
    \"bash\",
    \"-c\",
    \"env LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libpython3.6m.so:\$LD_PRELOAD java -jar /usr/local/share/jupyter/kernels/scala/launcher.jar --connection-file {connection_file}\"
  ]
}" > /usr/local/share/jupyter/kernels/scala/kernel.json

Lets check if the scala is working or not. if it does not work refresh the page.
println("Hello, world!")

Hello, world!

In this section, we give some overview of basics of Scala programming, data types, function and sclasses. This is quick hands on tutorial of going through few examples based on the the course Scala Essential Training for Data Science. I am not going to write the details descriptions of commands.
These the below sections covered.

Data Types
Scala Expressions
Scala Functions
Scala Classes/Objects
Scala Parrallel Processing

Data Types
Scala Expressions
Scala Functions
Scala Classes/Objects
Scala Parrallel Processing
I assume some basic knowledge of at least one programming languange. 
Since, I am coming from Python back ground, typecasting is usually ignored in python however Scala required you to specify the data type of variables hence making more sense of understanding the data types implemented in Scala. Below are the most basic data types covered in above course.

Scala Data Types: Int , Char, Float, Double, Long, Short, Boolean

Scala Collections : Arrays , Vectors

Scala Maps : Key value pairs


Scala Data Types: Int , Char, Float, Double, Long, Short, Boolean

Scala Data Types: Int , Char, Float, Double, Long, Short, Boolean
Scala Collections : Arrays , Vectors

Scala Collections : Arrays , Vectors
Scala Maps : Key value pairs

Scala Maps : Key value pairs
You can practice these types using below sample codes.
# This is formatted as code
// define int variable
var a_int:Int=3

a_int: Int = 3
var a_char: Char = 'd'

a_char: Char = 'd'
var a_long: Long = 832569

a_long: Long = 832569L
var b_char = 'c'

b_char: Char = 'c'
var b_long = 8345679489l

(console):1: integer number too large
var b_long = 8345679489
             ^
(console):1: integer number too large
var b_long = 8345679489
             ^
var b_long = 8345679489l

b_long: Long = 8345679489L
var c_float=1.2345

c_float: Double = 1.2345
var c_float = 1.21356f

c_float: Float = 1.21356F
val z=8321569l

z: Long = 8321569L
val temps= Array(50,51,56,53,40)

temps: Array[Int] = Array(50, 51, 56, 53, 40)
temps(1)

res13: Int = 51
temps(0)

res14: Int = 50
temps.length

res15: Int = 5
val temps2: Array[Int]=new Array[Int](10)

temps2: Array[Int] = Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
val temps3 = Array.ofDim[Int](10,10)

temps3: Array[Array[Int]] = Array(
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
  Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
)
import Array._

import Array._
concat(temps, temps2)

res19: Array[Int] = Array(50, 51, 56, 53, 40, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
temps. 

val vec1: Vector[Int] = Vector(1,2,3,4,5,6,7,8,9,10)

vec1: Vector[Int] = Vector(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
vec1(0)

res22: Int = 1
val myrange = 1 to 10

myrange: Range.Inclusive = Range.Inclusive(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
val myRange2 : Range  = new Range(1, 100, 2) 

myRange2: Range = Range(
  1,
  3,
  5,
  7,
  9,
  11,
  13,
  15,
  17,
  19,
  21,
  23,
  25,
  27,
  29,
  31,
  33,
  35,
  37,
  39,
  41,
  43,
  45,
  47,
  49,
  51,
  53,
  55,
  57,
  59,
  61,
  63,
  65,
  67,
  69,
  71,
  73,
  75,
...
val capitals= Map("Argentina" -> "Buenos Aires", "Canada" -> "Ottowa", "Egypt" -> "Cairo", "Liberia" -> "Monrovia", "Netherland" -> "Amstradam", "United Stats"->"Washinghton D.C")

capitals: Map[String, String] = Map(
  "Argentina" -> "Buenos Aires",
  "Egypt" -> "Cairo",
  "Canada" -> "Ottowa",
  "Liberia" -> "Monrovia",
  "Netherland" -> "Amstradam",
  "United Stats" -> "Washinghton D.C"
)
capitals.keys

res5: Iterable[String] = Set(
  "Argentina",
  "Egypt",
  "Liberia",
  "Cananda",
  "Netherland",
  "United Stats"
)
capitals.values

res7: Iterable[String] = MapLike.DefaultValuesIterable(
  "Buenos Aires",
  "Cairo",
  "Monrovia",
  "Ottowa",
  "Amstradam",
  "Washinghton D.C"
)
capitals get "United States"

res8: Option[String] = None
capitals("Canada")

res11: String = "Ottowa"
capitals contains "Egypt"

res12: Boolean = true
capitals getOrElse("China", "No capital found")

res13: String = "No capital found"
capitals+ ("Ireland"->"Dublin")

res14: Map[String, String] = Map(
  "Argentina" -> "Buenos Aires",
  "Egypt" -> "Cairo",
  "Canada" -> "Ottowa",
  "Liberia" -> "Monrovia",
  "Ireland" -> "Dublin",
  "Netherland" -> "Amstradam",
  "United Stats" -> "Washinghton D.C"
)
capitals - "Liberia"

res15: Map[String, String] = Map(
  "Argentina" -> "Buenos Aires",
  "Egypt" -> "Cairo",
  "Canada" -> "Ottowa",
  "Netherland" -> "Amstradam",
  "United Stats" -> "Washinghton D.C"
)
As with other languages, Scala expressions follow the similar approach with numerical, boolean and ligical expressions. Follow below lines of codes to practice some of the most commonly used expressions.
2+2

res16: Int = 4
5-6

res17: Int = -1
4*6

res18: Int = 24
33/4

res19: Int = 8
33%6

res20: Int = 3
3>4

res21: Boolean = false
6==9

res22: Boolean = false
(3>4) &(5<=10)

res23: Boolean = false
(3>4)  || (5<=10)

res24: Boolean = true
~(3>4) &(5<=10)

cmd27.sc:1: value unary_~ is not a member of Boolean
val res27 = ~(3>4) &(5<=10)
            ^Compilation Failed
Compilation Failed
var a : Int = 6
var b : Int = 10
var c : Int = 41

a: Int = 6
b: Int = 10
c: Int = 41
var d = a + c

d: Int = 47
a+=b

a

res30: Int = 16
c *= a

c

res32: Int = 656
println({
val a = 2 * 3
a + 4
})

10

println({
  val a = 2*3
  a+4
})

10

Scala is an objected oriented programming languange  and you can write functions pretty much following the same style. Below are some of the examples to practice writing functions in Scala.
def myFunction(a:Int, b:Int): Int = {
  val c = a*b
  return c
}

defined function myFunction
myFunction(2,4)

res36: Int = 8
def myProcedure(inStr: String) : Unit ={
  println(inStr)
}

defined function myProcedure
myProcedure("Log it as a test message")

Log it as a test message

Working with big data and complex code structures, classes make things simpler for programmers. Like other languages, you can write classes in Scala. Below are some examples.
val y = Array("England", "Liberia", "Haiti", "Austrailia", "Sweden" )

y: Array[String] = Array("England", "Liberia", "Haiti", "Austrailia", "Sweden")
y.sorted

res41: Array[String] = Array(
  "Austrailia",
  "England",
  "Haiti",
  "Liberia",
  "Sweden"
)
class location(var latitude:Int, var lat_direction:Char, var longitude:Int, var long_direction:Char, var altitude:Int)

defined class location
val loc1 = new location(45, 'N', 10, 'W', 300)

loc1: location = ammonite.$sess.cmd42$Helper$location@39023d48
loc1.altitude

res44: Int = 300
loc1.lat_direction

res45: Char = 'N'
class myPublicPrivate(val x:Int=0, val y:Int=0, private val z:Int=0)

defined class myPublicPrivate
val myPP = new myPublicPrivate

myPP: myPublicPrivate = ammonite.$sess.cmd47$Helper$myPublicPrivate@5afb1526
class Point2D(coord1:Int, coord2:Int){
  var a: Int = coord1
  var b: Int = coord2

  def move(delta_a: Int, delta_b: Int){
    a = a + delta_a
    b = b + delta_b
  }

}

defined class Point2D
val point1 = new Point2D(2,4)

point1: Point2D = ammonite.$sess.cmd49$Helper$Point2D@c13b69e
point1.a

res59: Int = 2
point1.b

res60: Int = 4
point1.move(1,4)

point1.a
point1.b

res63_0: Int = 3
res63_1: Int = 8
The key objective of Scala is to work with large data sets. To achieve that Scala has very clean solution of processing in parallel with minimal code change. If you are coming from Python, multiprocessing is hard to setup and often results with unintended results and failures. Scala has much better support of multi processing. Among other operations, filter and map are the most commonly used when processing in parallel. Follow some examples below.
val tng10 = 1 to 10

tng10: Range.Inclusive = Range.Inclusive(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
val prng10 = tng10.par

prng10: collection.parallel.immutable.ParRange = ParRange 1 to 10
import scala.collection.parallel.immutable.ParVector

import scala.collection.parallel.immutable.ParVector
val pvec200 = ParVector.range(0,200)

pvec200: ParVector[Int] = ParVector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199)
// mapping fucntion over Parallel Collection

val v = (1 to 100).toArray

v: Array[Int] = Array(
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24,
  25,
  26,
  27,
  28,
  29,
  30,
  31,
  32,
  33,
  34,
  35,
  36,
  37,
  38,
...
val pv = v.par

pv: collection.parallel.mutable.ParArray[Int] = ParArray(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100)
v.map(_*2)

res72: Array[Int] = Array(
  2,
  4,
  6,
  8,
  10,
  12,
  14,
  16,
  18,
  20,
  22,
  24,
  26,
  28,
  30,
  32,
  34,
  36,
  38,
  40,
  42,
  44,
  46,
  48,
  50,
  52,
  54,
  56,
  58,
  60,
  62,
  64,
  66,
  68,
  70,
  72,
  74,
  76,
...
pv.map(_ *2)

res73: collection.parallel.mutable.ParArray[Int] = ParArray(2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98, 100, 102, 104, 106, 108, 110, 112, 114, 116, 118, 120, 122, 124, 126, 128, 130, 132, 134, 136, 138, 140, 142, 144, 146, 148, 150, 152, 154, 156, 158, 160, 162, 164, 166, 168, 170, 172, 174, 176, 178, 180, 182, 184, 186, 188, 190, 192, 194, 196, 198, 200)
def square(x:Int) : Int = {return x*x}

defined function square
square(3)

res76: Int = 9
v.map(square(_))

res77: Array[Int] = Array(
  1,
  4,
  9,
  16,
  25,
  36,
  49,
  64,
  81,
  100,
  121,
  144,
  169,
  196,
  225,
  256,
  289,
  324,
  361,
  400,
  441,
  484,
  529,
  576,
  625,
  676,
  729,
  784,
  841,
  900,
  961,
  1024,
  1089,
  1156,
  1225,
  1296,
  1369,
  1444,
...
pv.map(square(_))

res78: collection.parallel.mutable.ParArray[Int] = ParArray(1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144, 169, 196, 225, 256, 289, 324, 361, 400, 441, 484, 529, 576, 625, 676, 729, 784, 841, 900, 961, 1024, 1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849, 1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916, 3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225, 4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776, 5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569, 7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604, 9801, 10000)
// Filtering parallel collections
val v = (1 to 10000).toArray
val pv = v.par

v: Array[Int] = Array(
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24,
  25,
  26,
  27,
  28,
  29,
  30,
  31,
  32,
  33,
  34,
  35,
  36,
  37,
  38,
...
pv: collection.parallel.mutable.ParArray[Int] = ParArray(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644,...
v.length

res80: Int = 10000
pv.length

res81: Int = 10000
val pvf = pv.filter(_ > 5000)

pvf: collection.parallel.mutable.ParArray[Int] = ParArray(5001, 5002, 5003, 5004, 5005, 5006, 5007, 5008, 5009, 5010, 5011, 5012, 5013, 5014, 5015, 5016, 5017, 5018, 5019, 5020, 5021, 5022, 5023, 5024, 5025, 5026, 5027, 5028, 5029, 5030, 5031, 5032, 5033, 5034, 5035, 5036, 5037, 5038, 5039, 5040, 5041, 5042, 5043, 5044, 5045, 5046, 5047, 5048, 5049, 5050, 5051, 5052, 5053, 5054, 5055, 5056, 5057, 5058, 5059, 5060, 5061, 5062, 5063, 5064, 5065, 5066, 5067, 5068, 5069, 5070, 5071, 5072, 5073, 5074, 5075, 5076, 5077, 5078, 5079, 5080, 5081, 5082, 5083, 5084, 5085, 5086, 5087, 5088, 5089, 5090, 5091, 5092, 5093, 5094, 5095, 5096, 5097, 5098, 5099, 5100, 5101, 5102, 5103, 5104, 5105, 5106, 5107, 5108, 5109, 5110, 5111, 5112, 5113, 5114, 5115, 5116, 5117, 5118, 5119, 5120, 5121, 5122, 5123, 5124, 5125, 5126, 5127, 5128, 5129, 5130, 5131, 5132, 5133, 5134, 5135, 5136, 5137, 5138, 5139, 5140, 5141, 5142, 5143, 5144, 5145, 5146, 5147, 5148, 5149, 5150, 5151, 5152, 5153, 5154, 5155, 5156, 5157, 5158, 5159, 5160, 5161, 5162, 5163, 5164, 5165, 5166, 5167, 5168, 5169, 5170, 5171, 5172, 5173, 5174, 5175, 5176, 5177, 5178, 5179, 5180, 5181, 5182, 5183, 5184, 5185, 5186, 5187, 5188, 5189, 5190, 5191, 5192, 5193, 5194, 5195, 5196, 5197, 5198, 5199, 5200, 5201, 5202, 5203, 5204, 5205, 5206, 5207, 5208, 5209, 5210, 5211, 5212, 5213, 5214, 5215, 5216, 5217, 5218, 5219, 5220, 5221, 5222, 5223, 5224, 5225, 5226, 5227, 5228, 5229, 5230, 5231, 5232, 5233, 5234, 5235, 5236, 5237, 5238, 5239, 5240, 5241, 5242, 5243, 5244, 5245, 5246, 5247, 5248, 5249, 5250, 5251, 5252, 5253, 5254, 5255, 5256, 5257, 5258, 5259, 5260, 5261, 5262, 5263, 5264, 5265, 5266, 5267, 5268, 5269, 5270, 5271, 5272, 5273, 5274, 5275, 5276, 5277, 5278, 5279, 5280, 5281, 5282, 5283, 5284, 5285, 5286, 5287, 5288, 5289, 5290, 5291, 5292, 5293, 5294, 5295, 5296, 5297, 5298, 5299, 5300, 5301, 5302, 5303, 5304, 5305, 5306, 5307, 5308, 5309, 5310, 5311, 5312, 5313, 5314, 5315, 5316, 5317, 5318, 5319, 5320, 5321, 5322, 5323, 5324, 5325, 5326, 5327, 5328, 5329, 5330, 5331, 5332, 5333, 5334, 5335, 5336, 5337, 5338, 5339, 5340, 5341, 5342, 5343, 5344, 5345, 5346, 5347, 5348, 5349, 5350, 5351, 5352, 5353, 5354, 5355, 5356, 5357, 5358, 5359, 5360, 5361, 5362, 5363, 5364, 5365, 5366, 5367, 5368, 5369, 5370, 5371, 5372, 5373, 5374, 5375, 5376, 5377, 5378, 5379, 5380, 5381, 5382, 5383, 5384, 5385, 5386, 5387, 5388, 5389, 5390, 5391, 5392, 5393, 5394, 5395, 5396, 5397, 5398, 5399, 5400, 5401, 5402, 5403, 5404, 5405, 5406, 5407, 5408, 5409, 5410, 5411, 5412, 5413, 5414, 5415, 5416, 5417, 5418, 5419, 5420, 5421, 5422, 5423, 5424, 5425, 5426, 5427, 5428, 5429, 5430, 5431, 5432, 5433, 5434, 5435, 5436, 5437, 5438, 5439, 5440, 5441, 5442, 5443, 5444, 5445, 5446, 5447, 5448, 5449, 5450, 5451, 5452, 5453, 5454, 5455, 5456, 5457, 5458, 5459, 5460, 5461, 5462, 5463, 5464, 5465, 5466, 5467, 5468, 5469, 5470, 5471, 5472, 5473, 5474, 5475, 5476, 5477, 5478, 5479, 5480, 5481, 5482, 5483, 5484, 5485, 5486, 5487, 5488, 5489, 5490, 5491, 5492, 5493, 5494, 5495, 5496, 5497, 5498, 5499, 5500, 5501, 5502, 5503, 5504, 5505, 5506, 5507, 5508, 5509, 5510, 5511, 5512, 5513, 5514, 5515, 5516, 5517, 5518, 551...
pvf.length

res83: Int = 5000
val pvf2 = pv.filterNot(_ > 5000)

pvf2: collection.parallel.mutable.ParArray[Int] = ParArray(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, 636, 637, 638, 639, 640, 641, 642, 643, 644,...
// function to return True or False whether value is divisible by 3
def div3(x:Int) : Boolean = { val y:Int = (x%3); return(y==0)}

defined function div3
div3(9)

res87: Boolean = true
pv.filter(div3(_))

res88: collection.parallel.mutable.ParArray[Int] = ParArray(3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, 63, 66, 69, 72, 75, 78, 81, 84, 87, 90, 93, 96, 99, 102, 105, 108, 111, 114, 117, 120, 123, 126, 129, 132, 135, 138, 141, 144, 147, 150, 153, 156, 159, 162, 165, 168, 171, 174, 177, 180, 183, 186, 189, 192, 195, 198, 201, 204, 207, 210, 213, 216, 219, 222, 225, 228, 231, 234, 237, 240, 243, 246, 249, 252, 255, 258, 261, 264, 267, 270, 273, 276, 279, 282, 285, 288, 291, 294, 297, 300, 303, 306, 309, 312, 315, 318, 321, 324, 327, 330, 333, 336, 339, 342, 345, 348, 351, 354, 357, 360, 363, 366, 369, 372, 375, 378, 381, 384, 387, 390, 393, 396, 399, 402, 405, 408, 411, 414, 417, 420, 423, 426, 429, 432, 435, 438, 441, 444, 447, 450, 453, 456, 459, 462, 465, 468, 471, 474, 477, 480, 483, 486, 489, 492, 495, 498, 501, 504, 507, 510, 513, 516, 519, 522, 525, 528, 531, 534, 537, 540, 543, 546, 549, 552, 555, 558, 561, 564, 567, 570, 573, 576, 579, 582, 585, 588, 591, 594, 597, 600, 603, 606, 609, 612, 615, 618, 621, 624, 627, 630, 633, 636, 639, 642, 645, 648, 651, 654, 657, 660, 663, 666, 669, 672, 675, 678, 681, 684, 687, 690, 693, 696, 699, 702, 705, 708, 711, 714, 717, 720, 723, 726, 729, 732, 735, 738, 741, 744, 747, 750, 753, 756, 759, 762, 765, 768, 771, 774, 777, 780, 783, 786, 789, 792, 795, 798, 801, 804, 807, 810, 813, 816, 819, 822, 825, 828, 831, 834, 837, 840, 843, 846, 849, 852, 855, 858, 861, 864, 867, 870, 873, 876, 879, 882, 885, 888, 891, 894, 897, 900, 903, 906, 909, 912, 915, 918, 921, 924, 927, 930, 933, 936, 939, 942, 945, 948, 951, 954, 957, 960, 963, 966, 969, 972, 975, 978, 981, 984, 987, 990, 993, 996, 999, 1002, 1005, 1008, 1011, 1014, 1017, 1020, 1023, 1026, 1029, 1032, 1035, 1038, 1041, 1044, 1047, 1050, 1053, 1056, 1059, 1062, 1065, 1068, 1071, 1074, 1077, 1080, 1083, 1086, 1089, 1092, 1095, 1098, 1101, 1104, 1107, 1110, 1113, 1116, 1119, 1122, 1125, 1128, 1131, 1134, 1137, 1140, 1143, 1146, 1149, 1152, 1155, 1158, 1161, 1164, 1167, 1170, 1173, 1176, 1179, 1182, 1185, 1188, 1191, 1194, 1197, 1200, 1203, 1206, 1209, 1212, 1215, 1218, 1221, 1224, 1227, 1230, 1233, 1236, 1239, 1242, 1245, 1248, 1251, 1254, 1257, 1260, 1263, 1266, 1269, 1272, 1275, 1278, 1281, 1284, 1287, 1290, 1293, 1296, 1299, 1302, 1305, 1308, 1311, 1314, 1317, 1320, 1323, 1326, 1329, 1332, 1335, 1338, 1341, 1344, 1347, 1350, 1353, 1356, 1359, 1362, 1365, 1368, 1371, 1374, 1377, 1380, 1383, 1386, 1389, 1392, 1395, 1398, 1401, 1404, 1407, 1410, 1413, 1416, 1419, 1422, 1425, 1428, 1431, 1434, 1437, 1440, 1443, 1446, 1449, 1452, 1455, 1458, 1461, 1464, 1467, 1470, 1473, 1476, 1479, 1482, 1485, 1488, 1491, 1494, 1497, 1500, 1503, 1506, 1509, 1512, 1515, 1518, 1521, 1524, 1527, 1530, 1533, 1536, 1539, 1542, 1545, 1548, 1551, 1554, 1557, 1560, 1563, 1566, 1569, 1572, 1575, 1578, 1581, 1584, 1587, 1590, 1593, 1596, 1599, 1602, 1605, 1608, 1611, 1614, 1617, 1620, 1623, 1626, 1629, 1632, 1635, 1638, 1641, 1644, 1647, 1650, 1653, 1656, 1659, 1662, 1665, 1668, 1671, 1674, 1677, 1680, 1683, 1686, 1689, 1692, 1695, 1698, 1701, 1704, 1707, 1710, 1713, 1716, 1719, 1722, 1725, 1728, 1731, 1734, 1737, 1740, ...
In this episode, we have learned the basics of Scala and covered below key concepts with excercises.
Running Scala in Google colab
Basic Scala Data Types
Scala Functions and Classes
Paralel Processing in Scala
In the next Episode, we will learn about spark with Scala using Google Colab
In this episode, we have learned the basics of Scala and covered below key concepts with exercises.
Running Scala in Google colab
Basic Scala Data Types
Scala Functions and Classes
Parallel Processing in Scala
In the next episode, we will learn about spark with Scala using Google colab.
Machine Learning with Scala in Google Colaboratory
Scala Docs -https://docs.scala-lang.org/tour/tour-of-scala.html