Programming with Functions #2: Functions as Data
By: Maciek Gorywoda
Tags: Programing, Functional Programming, Scala, Fp

Paraphrasing Martin Odersky, the creator of Scala, Functional Programming is on its most fundamental level simply programming with functions. A function is a piece of code that takes some data as arguments, performs some operations on it, and returns the result as another piece of data. Adding 2 and 2 means executing a function. Both numbers are arguments. The name of the function is “add” if we want to use the standard notation, or “+” if we want to use an operator — both are equivalent. The result is 4.

Of course, not only numbers can be data. Although, strictly speaking, yes, only numbers. We have booleans, false and true, but these are just 0s and 1s. We have strings, which are sequences of characters, but these characters are really just numbers which are interpreted as letters of the alphabet when we display them to a human being. We have complex data structures, but they consist of simpler data structures, which are eventually made of numbers. We have pointers, or references, depending on the programming language — they are addresses of where we can find other data, but these addresses are also basically just numbers.

In Imperative Programming, functions and data are separate concepts. Data is what a function works on. A function is what works on data. They belong to different worlds.

In Functional Programming, functions are also data. Data which tells us what to do with other data. We can use it as an argument for another function, so that higher-order function will be able to use it at some point of its own operation. If you wonder how it can work in practice, I’ll give you just the first of many examples: collections. In programming, collections are everywhere. If there’s only one piece of data, there’s nothing to do with it. What would it be, anyway? Your name? Your name is a sequence of characters. A location? A location makes sense only if there are other locations to compare it with. Aside from very trivial examples, no matter what your program does, at some level there is a collection of different elements that gives meaning to the program’s functionality. And when you have a collection of elements, you can perform operations on them. Operations mean functions. For example, you can have a bag — a collection — of cats, and you want to have a bag of black cats only. So you use the filter method of the collection (for the moment the difference between a method and a function is not that important). The filter method takes the collection as its first argument, a “subject”, so to say, called this, or self, and, as its second argument, a function which for every cat will return true or false. The result of the filter method is a new collection with only those cats in it which pass the test. In our case — those which are black.

bagOfCats.filter(cat => cat.colour == Black)

In Imperative Programming to do that we would use a loop. We would create a new, mutable, empty collection at first, and then run a for loop for all cats in the original collection, and if they’re black we would put them in the new collection.

var bagOfBlackCats = new Bag[Cat]()

for (cat in bagOfCats) {

if (cat.colour == Black) bagOfBlackCats.add(cat)

}

Two problems here: First, the mutable bag of black cats will stay mutable even after the loop finishes its operation which makes the code vulnerable to someone adding more cats (maybe even not black ones!) later. Second, we have to write our own loop implementation, even if it’s the same implementation we were writing again and again for years, since our first programming course. It’s boring, it takes time, and because of the routine it’s even possible that we will write it wrong and try to access an overflowing cat and get a cat out of range exception. Functional programming lets us separate the code that takes care of iterating over the collection, and that which actually cares about what this collection consists of and what to do with it. The first part doesn’t care if it’s a bag of cats or dogs, or maybe something completely different, as long as it is a collection of elements which can be filtered. This way we can put the implementation of the filter method in the collections library and never think of it again.

Similarly, we have other methods, like map, flatMap, foreach, fold... traverse ... I don’t think I need to discuss them here in detail, but I’d like to point out that each of them encapsulates a way of operating on elements in a collection that does not need to know what elements it operates on. All that's left to do for us is to give those methods more specific functions as arguments, so they will know what to do. That means less code for us to write, and thus fewer opportunities to make errors, and more time to argue on Twitter.

Total and partial functions

A function is a piece of code that takes arguments and produces a result. When we talk about total and partial functions, it means that we are interested in the type of the argument and whether we can produce a result from it. Or we can say that we are interested in the function’s domain, in a more computer-sciency language. I think the easiest way to explain what is, is with examples.

Let’s say we have a multiplication function,

def mul(x: Int, y: Int): Int = x * y

It takes two integers, x and y , and produces a third integer — the result of multiplying x and y . Our multiplication function is total: You can provide any two integers to it and you will obtain a result of the declared type — Int as well.

But how about division? Let’s say we have another function, div, which looks like this:

def div(x: Int, y: Int): Int = x / y

For a lot of values of x and y , the div function gives a proper result of the type Int , but… you already see where we’re going… If x can’t be divided by y without a remainder, in Scala the remainder will be ignored and we will get only the integer part. For example, div(5, 2) == 2 . It’s surely not an ideal solution. More importantly, for y == 0 another weird thing happens: we get java.lang.ArithmeticException . It means that a division of a pair of integers which produces an integer is a partial function — it does not work for all pairs of integers. Again, from a more mathematical point of view we can say that such a function is a mapping from one set of elements to another. The set of arguments, in our case, are all tuples, or pairs, of integers. The set of results are all integers. The function tries to map each tuple from the set of arguments to one of integers in the set of results. The multiplication function is able to do it, but the division function fails for some tuples. The division function is partial.

It doesn’t mean that partial functions are never useful. Sometimes we can use to our advantage the fact that they work only on a subset of possible input. One such popular case in Scala is the collect method in collections, also called filterMap in some other languages. It’s a shorthand for a common case where we first filter the original collection and then map the obtained subset to get a smaller collection of something else. Instead of having to first filter the whole collection and then map over the subset, we can use collect and provide it with a partial function which will work only for a subset of elements and it will map them. collect will ignore all elements which the partial function doesn’t work for.

val blackCatNames1 =

bagOfCats.filter(_.colour == Black).map(_.name)

val blackCatNames2 = bagOfCats.collect {

case cat if cat.colour == Black => cat.name

}

There are two ways to turn a partial function into a total one. One is to simply broaden the domain — in our case, to change the result type from Int to Double and make a small change in the implementation so that we will convert x and y to Double before we use it.

def div(x: Int, y: Int): Double = x.toDouble / y.toDouble

The type Double , named that way for historical reasons, long forgotten by almost everyone, represents rational numbers — not only integers, but also fractions, but only down to a certain number of zeros after the dot. And on top of that we have two constants — Infinity and -Infinity — and one magic not-a-number number: NaN .

If y == 0 , and x > 0 , we will get Infinity as the result. Similarly, if x < 0 we will get -Infinity , and if x == 0 , we will get NaN . That virtually solves all our problems, but it also forces us to deal with a result type that is more flexible that we might have wanted. What if we actually wanted to use the div function only for integers which can be divided without a remainder, and any other outcome should be treated as an error? That’s how the other, better way works: it turns a partial function into a total one using types to better describe what the function can accept as arguments, and what it returns. We will discuss that in one of the next chapters.

Here’s what’s important is that it gives us better control.

A correctly-defined type is a piece of information that can help us avoid many errors. We can use this information to narrow down our scope of thinking and focus on what’s important to us, our happy path, and at the same time clearly outline corner cases and errors. We can also pass it to other developers as a form of minimalistic, but always available and always up-to-date documentation. We say “this function here takes only arguments of such and such types, nothing else will be accepted, but also all arguments which conform to these rules are always valid, and it returns results exactly like that, nothing more”.

That’s it for today. Thank you for watching. In the next video we will talk about pattern matching — a feature which can let us write very powerful code if we defined our types well — and about smart constructors, and sealed traits. As always, if you have any questions or want to contact me for another reason, you can just leave a comment below. Or you can also find me on Twitter.

Scala collections library — Just browse through it. The most important classes are probably Seq, Set, and Map. Write some simple programs with their methods. Experiment. Have fun.

Scala with Cats by Noel Welsh and Dave Gurnell — Be aware the book quickly goes from simple to very advanced stuff.

Creative Scala by Noel Welsh — This book is more beginner-friendly, but it still focuses on FP from the very start.

Previous: Programming with Functions #1 — Introduction

Next: Programming with Functions #3 — Pattern Matching

If you want to help me translating the video’s subtitles, here you can download their English version, and then please find me on Twitter: @makingthematrix.