To Discuss Algorithms We Must Agree on the Primitives
By: Estevão Uyrá Pardillos Vieira
Tags: Cognition, Neuroscience, Computational Thinking, Algorithms

Although computational thinking is growing, it is still tricky to propose or discuss theories in algorithmic terms, at least in the cognitive sciences. One reason for this difficulty may be that those standing for computational thinking do not invest enough in setting the groundwork. One such investment is, at each discussion, specifying the primitive operations, to set precisely the level of abstraction. In this post, I give an illustrative example of the problem and a practical example from the neuroscience of time perception.

The primitives of multiplication

Take the example of the multiplication algorithm we learned at school. We can still sketch and solve a huge 10-digits multiplication if we get pen and paper. If you want to teach the algorithm to a child, you will probably describe it as a series of 1-digit multiplications and 1-digit sums (also some positional translations and copying but they are not important for this discussion). These primitive operations, when taken in the right sequence, accomplish the n-digit multiplication.

N-digit multiplication is accomplished by a series of simpler transformations. Image from Roughgarden, Tim. Algorithms Illuminated. Soundlikeyourself Publishing LLC, 2017.

Nowadays 1-digit divisions and sums are easy enough for us adults, but is their position as primitive operations unassailable? For a very young child multiplication may be hard even for 1 digit — the child still has to memorize the times tables. Does this mean the algorithm is yet unlearnable?

Not if we change the primitive operations to other ones that make more sense to the child, such as searching. Give them a printed multiplication table and tell them to search for the corresponding number in the table, and we just changed the primitive operations of 1-digit multiplication to search.

Part of a multiplication table.

We could do the same for addition by giving a second table, and the n-digits multiplication algorithm could then be described as a series of search-and-copy steps, without any further reference to 1-digit multiplication or 1-digit addition.

But search is also not a fundamental operation, as becomes clear when we have to find a specific symbol in a table of chinese ideograms for example. In this case, we may partition the symbol and compare specific parts or characteristics, reducing iteratively the number of possibilities, until we have only a single match and have successfully searched. The process of searching in children that do not have yet much familiarity with numbers may be much similar to the ideogram case than to the familiar way we adults search for an entire word inside a block text.

Ideograms. For those without familiarity, it is sometimes hard to compare entire complex symbols when they are similar, and we might go back and forth. Photo by Cherry Lin on Unsplash

Is there a correct level?

I will compare two opposed answers to this question (yes, and no), which I will call ontological and operational.

In the ontological view, there is a single correct primitive, which can be an atomic operation — one that can not be split into a series of sub-operations. In the multiplication example from before, this atomic level depends on the experience of the agent performing multiplication, and since you probably know most of the times tables by heart, your primitive operation for the n-digit multiplication is the 1-digit multiplication.

Pseudo-python: Standard multiplication algorithm with one-digit-multiplication as primitive.

In the operational view, there is no correct primitive, but there is a single best primitive to support specific discussions and answer specific questions. For different questions, other primitives may be better suited. To illustrate this, let’s say we want to understand why it takes longer to multiply large numbers. It suffices to note that there are more 1-digit multiplications necessary, no matter if the child is multiplying with a physical times tables. On the other hand, if we want to understand why a kid takes longer to multiply 789x123 than 123x789, we would need to look at its search strategy, and maybe note that the kid searches the table in a specific direction (from top-left to bottom-right), and their eyes take longer to reach the 7x1 versus the 1x7.

I have stretched somewhat the multiplication example, but there is a very good example of the operational view’s power in the practice of computer programming. When writing long and complex programs, a common approach is to try to encapsulate things as much as possible, so that you can understand what each part of your code does without needing to peer into the depths of the code. For example, you can use good libraries without thinking about how exactly each function does its work. To use scikit-learn’s auto-regularized Logistic Regression you don’t even need to know what equations are minimized inside the function, you just need to input your data and get a classification algorithm on the other side. If you are not into machine learning or Python, just think about some library you have used recently, and how much time you have saved by the well-encapsulated abstractions (If you are not into programming, [sorry for the code] maybe you should start).

Pseudo-python: Standard multiplication algorithm, where the one-digit multiplication is composed of other functions. Note that the n-digit-multiplication algorithm does not change. All changes are encapsulated in the inner functions so that we could understand much about the n-digit without knowing this details.

Anyway, all this to say that when programming, writing human-made algorithms, we are accustomed to using abstractions as building blocks, and the abstractions we use depend on the problem we want to solve (and the libraries available). Going to higher abstractions (higher-level code) you usually get much easier coding but less flexibility. When we talk about nature-made algorithms, the operational view is just an extension of the view we use to code — think of the building blocks that will best support the question we wish to answer. No profound reason to choose one over another (no epistemic primacy in either).

Primitives on the wild

In the wild of scientific research, the choice of primitives is often left implicit, as are the reasons for choosing them. I brought a very nice example from a paper I read during my Master’s, when I was mainly studying the acquisition of interval-timing behavior, which is for example learning to hold down a button for precise 3 seconds, or to distinguish between a 2s flash of light from a 1.5s flash.

Performing this kind of behavior requires the brain to perform specific computations, and we want to understand better which and how. To conceive which algorithms are in use, underlying these computations, we need to establish a level of abstraction to support the conversation and build upon it.

Four-level of abstractions model for timing. Each level takes the abstraction farther from neurons and closer to the behavior being modeled. Image from Simen, Patrick, et al. “A model of interval timing by neural integration.” Journal of Neuroscience 31.25 (2011): 9238–9253.

The image I put here is not meant to explain everything in detail, but to illustrate the interplay between levels of abstraction. They break down from complicated functions to simple ones, the same way that we can break down from n-digit multiplication to search-and-comparison.

The timing computation can be learned and performed with a “diffusion model”, shown in Level 4, and they show how you can bridge between this model and the single-neuron model, which does not have further partitioning, and is thus is the primitive operation for them (it could be partitioned, right? With ion channels, the dynamics of membrane potential and so on). Think of the dashed lines from each line to the line above as complicated functions calling simpler functions. I made a very-sketchy pseudo-python just to give the idea of what I mean.

Sketch of a pseudo-python for the 4-levels timing model. It is not proper pseudo-code, but just to give an overall idea of the abstractions. It could be a good practice to write these down to better understand which inputs are defined in a specific level and which are parameters from the level above, as well as to have a more precise delineation of when one level of abstraction ends and the other starts.

Refraining from going deeper into the specifics of timing in the brain, I can say that many neuroscientists have a tendency to set the neuronal level as the primitive level, and have some resistance to accept models for which the abstraction does not go down to the neuron. Nevertheless, there is a lot of work on algorithmic models with primitive operations such as peacemaker and switch. These models are generally called cognitive models since they do not refer to the neural substrate where the computations do occur.

The Scalar Expectancy Theory (SET) model of timing. This model captured some statistics of error distribution in timing tasks for human subjects and sufficed to explain some pharmacological effects. It is used as a reference model in timing studies since 40 years ago. Image from Gibbon, John. “Scalar expectancy theory and Weber’s law in animal timing.” Psychological review 84.3 (1977): 279.

Whether one set of primitive operations is better than the other I leave open for the reader, but I disclosure that I am biased to the operational view. During the time I spent studying the neuroscience of time, I noticed a strong preference to models that could give a specific role to the single neuron or to a circumscribed region in the brain (as the atomic element performing the primitive operation) but it was not clear to me that this was an explicitly decided approach so much as a historically developed one, especially in literatures (such as the timing literature) that have a mix of psychologists, neuroscientists and cognitive scientists discussing each with their own frame.

Probably each approach has its advantages, and they could probably be combined or integrated, however many times the discussion takes the form “well your model contains a pacemaker, but which region in the brain is the pacemaker?”, when the pacemaker could very well be a function of oscillatory activity in multiple regions, or of attractor states in a functional network, or other elaborate function not necessarily easily mapped into neurons. It is asking about the n-digit multiplication “but which kind of comparison is your 1-digit-multiplication?” — we could answer to this since we have mapped the case of ideogram multiplication, but we have already agreed this should not be necessary to answer much of the questions.

Conclusion

While an ontological stance is hardly defensible in software engineering (you have to explain your algorithms in binary!!1!01!) this discussion is much more intricate in the sciences since we are not ourselves creating algorithms to solve a problem, but devising algorithms that approximate the functioning of evolved complex systems, which could then be used to further our understanding about these systems.

In other posts, I will write more about the level-of-abstraction discussion, of which we just scratched on the surface. I may also write more about the timing literature specifically, which has a lot of interesting discussion around modelling. I like to think (and I’m not alone) that we should allow ourselves to devise algorithms with abstract primitive operations, based on what the system is supposed to be doing, and then later on if it seems to make sense, break down the primitives into simpler ones — actually I’m just rephrasing Chomsky here, but this is the theme for another post.

Probably setting the groundwork would be helpful for collaboration in these research areas where the historical assumptions taken by distinct lines of research differ so much. Hope to have tickled your brains with this meta-science discussion which I find so interesting!