The Conundrum of Functional Paradigm
By: Alireza Goudarzi
Tags: Functional Programming, Python Programming, Software Engineering, Coding Style, Clean Code

At Datance, we allow ourselves time to get into heated debates on algorithms, efficiency, coding style, etc. It is a good way for reflection, and renewing oneself and to keep our senses of reality sharp. A usual topic of discussion among us is functional programming. One way or another, we have all embraced it, but as I came to realize how I understand it is somewhat different than my colleagues. For the old times sake, I decided to analyze the differences between a standard code and a fully functional code with respect to the design approach behind it and the final result in a piece of code I wrote recently. What I found was more than amusement and merit a public discussion.

Starting at the top. The goal is to write a function that starting from a root directory recursively return the list of all the filenames that exist in it or any of its subdirectories. The subtext here is that the filenames should match a criteria, but as you will see this does not match the general setup of the problem and the final solution.

Here is the basic code:

So on the surface, this is probably no the most simplistic way of writing this code, but still easy enough to understand. Here is the outline:

Line 32: Define function DigFiles that receives a string pointing to the root dir to scan and a list of allowed file extensions and returns a list of files. Consider these as conditions and as we write the rest of the algorithm we have to respect these conditions. Line 34: Create a list of all the files inside the root directory given by Base . Note that in the list comprehension we are concatenating the root dir to the name of all the files and that we are filtering the files we consider by the extensions that exist in the FileExtFilter list. Line 38: Create a list of all the directories inside the root given by Base .

Simple enough, right? Let’s look at the output. Here is how I am invoking the function and the given directory structure:

Invocation and the target directory structure.

The allowed file extension only contains axy and the Base is ./src . I call the function 3000 times and time the execution.

It returns the correct result and takes 1.24 seconds for 3000 calls. Not bad! The function is plug and play in any situation too, it does not depend on the environment except for the imported modules and if you wanted you could pass the functions it uses such as isdir, isfile, and listdir to be completely independent of the external. On the surface this looks like as functional as it gets.

However, to anyone graduating from a serious programming language class this doesn’t look functional at all. Right at the beginning of the function on line 34, we are committing a mortal sin by breaking the commandment “ye shall not manipulate thy environment.” The truth is in pure functional programming the entire mindset of how you go about decomposing and solving the problem is different. It is not just about the syntax and modularization. A function receives a series of inputs as a list, and outputs a series of results as a list. Internally you solve the problem by performing list comprehension: go through the result for the “first” element of the list (also called head), and merge the result to the result of computation on the “rest” of the list, or sometimes called tail. Let’s see how would this play out in our situation (here is the code).

If you know traditional functional languages this may look a bit odd, but here we are using python and don’t have to nice features such as pattern matching that is standard in a pure functional language. We will have to make do with what we have, but that shouldn’t stop us from adopting philosophy of functional programming. The first thing you notice is that Base is a list and the time of the first invocation of the function it is initialized to the a list with a single string pointing to the path to the root directory to start the search from. The first second line root out the obvious, if the input list is empty we return an empty list. This is the so called the bottom of recursion. That’s how the function detects it should not infinitely call itself. The second if says if the Base has more than one element run the DigFiles on the first element (the head), and the rest of the list (the tail) and return the result. the third and forth ifs are what will actually happens when the function receives the a list of only one element. These are run at the first invocation, and the invocation on the head. If this single element is a directory we list the content run DigFiles on the list and and return its result. if it is a file we check if it matches the allowed condition and return it. Note that the syntax and the structure of the code almost verbatim matches how you verbalize solving the problem. That is a signature characteristic of pure functional code, the code matches the solution and result in very clean code that you can grasp at one glance (if you are used to it). And the output:

The result is the same of course, and in addition to the nice code note a whopping 65% improvement in performance. Wouldn’t that worth getting familiar with this kind of problem solving? And here ends the lesson.

P.S. note that we are doing recursion here and depending on which programming language you are using your recursion may not be efficient, so you can’t use this for big data analytics for example.