C++ lambdas, for beginners
Tags: beginners, cpp, tutorial
Published: 2020-09-14 00:09:54

* [What's this all about](#about)
* [A Lambda's full name](#lambdas_full_name)
* [Analysis of the lambda expression](#lambda_expression)
  * [The return type of the lambda](#return_type)
  * [The capture list](#capture_list)
  * [Mutable lambdas](#mutable_lambdas)
  * [Generalized captures (#C++14 and beyond)](#generalized_captures)
* [What can and cannot be captured](#can_and_cannot)
* [Captures and the `this` pointer](#lambdas_and_this)
* [An intuition of the closure class](#intuition)
* [Conclusion](#the_end)

## What this all about  <a name="about"></a>

When C++11 came out it came with a lot of goodies, and one of the best additions to the language repertoire were lambda functions.

Most people's first exposure to lambda functions in C++ is in the way of "throwaway functions" scattered around the code, because that's what lambdas are used for most of the time. Take for instance:
```
auto addition = [](int x, int y) { return x + y; };
std::cout << addition(4, 3) << std::endl;
std::cout << addition(99, 1) << std::endl;
```
Here a lambda function is created to perform a simple addition between two numbers, and then the lambda gets called a couple of times with different input arguments each time. When execution exits the scope where the lambda object was created, the lambda will be destroyed just like any other local variable.

Lambdas can do more than that, though. Lambdas are executable objects, and just like other objects, they can be moved around, copied, stored in containers, passed as arguments, etc. Lambdas can carry a context with them, so that they can be created in one place and transferred to another for execution. Lambdas can be stateful, and can also be linked to the state of an external object.

It must be said that lambdas don't really bring anything radically new to the C++ table. Everything that can be done with lambdas today could already be done with pre-C++11 versions of C++ using good-old functor objects (objects that can be called like functions), if you bothered to do the effort to write them.

The real advantage of the lambda notation, though, is that it provides a compact way to describe an executable object in such a way that **the compiler can do the implementation for us**. The programmer can then focus on the _what_ instead of on the _how_, and explore design possibilities that might have been avoided otherwise

During the rest of the article I'll try to introduce you to how lambda functions get declared, and what the ins and outs of variable capture are. 

Please notice that the article is aimed at beginners trying to "get it" and/or intermediate users looking for a refresh. If you already "get" lambdas, you can look try the references in cite at the very end for better material.

## A Lambda's full name <a name="lambdas_full_name"></a>

Lets start by getting semantics out of the way. 

In reality, there's no such thing as a lambda. "Lambda" is just a colloquial term to that, depending on the context, will usually refer to one of three things.

- A **lambda expression**.
- A **closure object**.
- A **closure class**.

All three of them are present in the following sentence:
```
auto addition = [](int x, int y) { return x + y; };
```
Here, 
- `[](int x, int y) { return x + y; }` is the lambda expression.
- `addition` is the closure object.
- You can't see the closure class, but it's there in the form of the type of the closure object.

Whenever you hear someone talking about a lambda, they are most likely talking about one of the three above, and in most cases it'll be one of the first two (the third one gets far less press time).

I'll keep it light and talk about `lambdas` when it suits me better, but I'll use the terms above when the distinction is important.

## Analysis of the lambda expression <a name="lambda_expression"></a>

The lambda expression is the most visible expression of the lambda in the code.

The structure of the lambda expression is
```
[...capture list...](...parameters...) -> <return type> { ... body ... }
```
but in practice, it will be a lot simpler than this. In fact, the humblest of all lambda expressions is
```
[]{}
```
which defines a lambda that defines captures nothing, takes in nothing as parameters, does nothing, and returns no data; that lambda is, in other words, equivalent to this function:
```
void f() {}
```

Just like for function declarations, the code of the lambda is enclosed within the `{}` brackets, and the input parameters within the `()` pair. There's not a lot to say about either of these components of the lambda, since they follow both in form and in function from their equivalents on the function declaration.

The `()` of the parameters list can be completely omitted when the lambda is not meant to receive any parameters. That's why the humblest lambda is `[]{}` and not `[](){}`. If a return type is specified, however, the `()` needs to be written even if the parameter list is empty.

I want to talk at length about captures, which is what separates lambdas from just being throwaway functions, but allow me to get something out of the way first and discuss the return type of the lambda first.

### The return type of the lambda <a name="return_type"></a>

The return value type of the lambda can be omitted from the expression, and most people will not write it. When the type is not present the compiler will infer it from the rest of the expression.

I won't be writing the return type in the rest of the examples in this articles unless I really need it. That's why I wanted to task about this early: it's less typing for me.

Only in a minority of cases you'll see lambda expression that fully state the return types of the lambda. In those cases it will usually be because of one of two reasons:
- The lambda body is such that the return type is ambiguous to the compiler.
- The coder wants to coerce the return type into something different from what the compiler would have inferred.

Whenever the return type is not written in the lambda expression the compiler uses a simple but effective set of rules to determine what type _you meant_ to return:
- If there are no `return` statements, or if the ones present are bare `returns` with no return value (such as `return;`), then the lambda is assumed to return `void`.
- If there's a single `return` statement and it has a return expression, the return type will be the type that results from evaluating the expression.
- If there are multiple `return` statements and all of them have a return expression that evaluates to the very same type, that will be the return type of the lambda.
- If none of the above is the case, the compiler will give up and then you'll have to explicitly state the return type.

A few examples:

This declares a lambda that just returns `void`:
```
auto f1 = [](){};  // could be just []{}
```

Here the return expression evaluates to `int`, and therefore the return type of the lambda will be the same type:
```
auto f3 = [](int n) { return n; };
```

The return type in the expression here evaluates to `bool`, so that'll be the type of the return value:
```
auto f4 = [](int n) { return n > 5; };
```

There are two return expressions in the following example, but both are `bool`. Since there's no ambiguity the return type will be `bool`:
```
auto f5 = [](int n) {
  if (n == 5) {
      return true;
  } else {
      return false;
  }
};
```

In this case there are two return statements with expressions that evaluate to different types (`bool` and `int`). In this case the programmer needs to state state the return type explicitly, because it's not possible for the compiler to decide what the return type should be:
```
auto f6 = [](int n) -> bool {                
  if (n < 0) {
      return false;
  } else {
      return n; /* returns true if n > 0 */
  }
};
```

In this case we state the return value to coerce the return type to be a `bool` instead of an `int`. 
```
auto f7 = [](const std::vector& n) -> bool { return v.size(); };
```

### The capture list <a name="capture_list"></a>

The capture list of a lambda is a list of variable names *in the enclosing scope of the lambda* that have to be _captured_ to become accessible within the lambda. 

In a lambda expression the capture list is written between the square brackets `[]`. It is not optional, and the brackets must be present even if the capture list is empty.

A capture creates a variable within the scope of the lambda body that has the same name as the variable being captured, and which can be (depending on the mode of capture) either **a copy** of the external variable, or **a reference** to it.

Captures that generate a copy of the external variable will be called _by-copy_ captures, while the ones that generate a reference to an external variable will be called _by-reference_ captures.
- A _by-copy_ capture creates a variable within the scope of the lambda that is a copy of the variable being captured, with the same value the later had at the time the lambda was created. By-copy captures are read-only, unless the lambda is _mutable_ (more on that later).
- A _by-reference_ capture stores a reference to the variable being captured, which can be used to read or update the value of the external variable at any time during the lifetime of the lambda.

To ground those definitions with an example, take a look at the code fragment below. In it two local variables get defined, followed by the definition of a lambda function that captures them both.
```
int foo = 33;
int bar = 22;
auto within_range = [foo, &bar](int n) {
  return foo + bar;
}
```
The capture list in the lambda expression captures both variables: `foo` is captured by-copy, while `bar` is captured by-reference.

It's important to realize that that while the names of the captured variables are the same as the names of the external variables that they mirror, they are different variables. i.e. **the `foo` variable within the body of the lambda is not the same `foo` outside.**

As you probably guessed from the example, the capture mode is declared by prefixing by-reference captures with `&`. By-copy captures, on the other hand, have no prefix at all.

Additionally, there are two  default capture modes that allow us to capture every variable that is used in the body and not explicitly mentioned in the capture list:
- A bare `&` will capture by-reference anything used but not explicitly captured by-copy.
- A bare `=` will capture by-copy anything used but not explicitly captured by-reference.

In a typical capture list you'll find a mixture of default capture modes with named captures. There are a few rules to this mix, however. 
* The default modes `=` and `&` cannot be both present.
* If a default mode is present, it must lead the list.
* If a `=` default capture is present, any named capture that follows must be by-reference.
* If a `&` default capture is present, any named capture that follows must be by-copy.
The rules make sense if you think about it, so it's not really necessary to worry about them too much.

These are some examples of typical capture lists:
* **[]** Empty capture list, nothing will be captured.
* **[foo]** Capture `foo` by copy.
* **[&bar]** Capture `bar` by reference.
* **[foo, &bar]** Capture `foo` by-copy and `bar` by-reference.
* **[=]** Capture anything named from the enclosing scope by-copy.
* **[&]** Capture anything named from the enclosing scope by-reference.
* **[&, foo, bar]** Capture anything named from the enclosing scope by reference, except `foo` and `bar` which must be captured by-copy.
* **[=, &foo]** Capture anything named from the enclosing scope by copy, except `foo` which must be captured by-reference.

#### Mutable lambdas <a name="mutable_lambdas"></a>

By default _by-copy_ captures are not writable, and therefore the following fragment is an error:
```
int value;
auto bad_lambda = [value]() { value += 10; };
```
By-copy captures can be made writable if the lambda is declared as `mutable`. This makes the lambda stateful: any change you do to a by-copy capture will be carried over to the next execution of the same lambda.

For example, in this example the lambda will _remember_ any update to the value of the captured `initial_value` variable. The value of the external variable, however, will remain unchanged because the lambda updates a copy.
```
int initial_value{5};
    
auto counter_lambda = [initial_value]() mutable {
    std::cout << initial_value++ << std::endl;
};

// each call will increment the internal copy
// stored within the lambda, and change carry over to 
// the next call.
counter_lambda(); // will print 5
counter_lambda(); // will print 6
counter_lambda(); // will print 7

// the original variable outside of the lambda is unchanged
std::cout << initial_value << std::endl;
```

_by-reference_ captures, on the other hand, can be both read and written regardless of whether the lambda is `mutable` or not.
```
int total{0};

auto accumulate = [&total](int n) { total += n; };

// each call updates the value of the references variable
accumulate(1);
accumulate(2);
accumulate(3);

// print the accumulated value, 6
std::cout << total << std::endl;
```

#### Generalized captures (C++14 and beyond) <a name="generalized_captures"></a>

All the talk so far has been about captures as they were introduced when C++11 came out.

These captures work great, and there's a lot you can do with them, but after a while you'll notice that there are a couple of ways in which they fall short:
* A capture always has the same name as the variable that was captured. This is not a big deal, of course, but sometimes you'd like to be able to name them something else.
* To capture a value it needs to be previously stored in a variable; it's not possible to capture the result of an expression.
* You can't use move semantics with captures. Captured objects need to be copyable; if they are not then you'll need to capture them by-reference, which may create a problem of ownership, or you'll need to do some other trick. This may be particularly annoying if you use `unique_ptr` a lot.

To mend this, C++14 upgraded lambdas with `generalized lambda captures` which 
* allow you to name the internal name of the capture anything you like.
* allow you to capture not only variables, but also the result of expressions (only by-copy).
* more importantly, allow you to capture move-only variables like `unique_ptr` instances.

The price to pay for these welcome improvements is that generalized captures are bit more verbose than regular captures because you need to state both the name for the variable being captured, and the name of the capture variable created within the lambda. The syntax is:
* `internal_name=expression` for by-copy captures.
* `&internal_name=external_name` for by-reference captures.

For example, this example uses generalized captures to capture `counter` by reference (naming it `cnt` within the lambda), and also captures the result of `3 * mean_level` by copy (naming the result `limit` within the lambda).
```
int mean_level{5};
int counter{0};

auto f = [&cnt = counter, limit = 3 * mean_level]() {
  if (cnt < limit) cnt++;
};
```

To capture a move-only object, you just need to make sure the right side of the by-copy assignment is an rvalue, which can be done by providing a temporary value, or by using `std::move`:
```
auto adapter = std::make_unique<Adapter>();

auto runner = [adapter = std::move(adapter)]() { adapter->run(); }
```

The extra verbosity of generalized lambda captures is a very small price to pay given that you're not even required to pay for it: you can still use regular C++11 captures when that suits you better, and mix generalized and regular captures to get the best of each:
```
auto f = [&counter, limit = 3 * mean_level]() {
  if (counter < limit) counter++;
};
```

## What can and cannot be captured <a name="can_and_cannot"></a>

Earlier I said that only variables in the immediate local scope of the lambda can be captured. I mentioned it only in passing, and it probably flew under the radar when I said it.

However, this not a minor detail or a technicality, and to see why lets see what **cannot** be captured.
* Global scope variables and static data members can't be captured.
* Non-static class members can't be captured directly.

The first one may strike you as obvious if you think of it, since globals are accessible from within any function, and lambdas are function-like ("callable") objects, there's no reason for them to be an exception to this.

Still, you should keep in mind that globals must be regarded within a lambda just like by-reference captures are. This may have important implications in multi-threaded programs.

Static class members are just globals in disguise, so it's no surprise that as far as lambdas are concerned, they have the exact same restrictions.

Non-static class members cannot be captured either, but here the truth is more nuanced: actually they can, kind-of, but they cannot be captured in the same sense in which captures work for regular local variables.

Before we dig deeper into this, however, we need to take a short detour to talk about the pointer `this`. 

## Captures and the `this` pointer <a name="lambdas_and_this"></a>

During execution each non-static class method has access to an implicitly created `this` pointer that references the instance on which the method was called. This is what gives methods access to non-static data members of the class.

Typically you don't need to de-reference that pointer explicitly since the compiler will do it for you, but you can if want to be more explicit. For instance, in this fragment
```
class Value {
 public:
  void set(const int x) { x_ = x; }
  int get() const { return x_; }
 private:
  int x_;
};
```
we can make the dependency on `this` more visible by rewriting both methods as
```
void set(const int x) { this->x_ = x; }
int get() const { return this->x_; }
```
It's important to notice that this does not change the way the code is compiled, it only makes explicit what the compiler is doing behind your back. 

This detour to talk about `this` (pun intended) is because now that we have unmasked how access to data members works it's easier to understand the _nuanced_ version of how non-static data member capture works.

Non-static class members cannot be captured because they are not in the local scope around the lambda, they are in the class scope.

However, the `this` pointer can be captured, because _it is_ a variable within the immediate scope of lambdas that get created in non-static methods of a class! By capturing `this` the lambda gets access to all non-static data members and also to instance methods.

In order to capture this, you just add it to the capture list:
```
auto is_empty = [this]() { return queue_.empty(); } 
```
It's important to not that `this` is not a regular variable, and it's nature imposes a limitation to the capture process: `this` can only be captured _by-copy_; trying to capture the pointer like in `[&this]` is syntactic error. Default capture modes will also capture `this`, but notice that even if `this` is captured by the `&` default capture mode, the pointer will still be copied.

Now, here comes the catch: remember that `this` is not the instance, `this` is a pointer to the instance. You're not capturing the instance by copy, your only copying a pointer to it. 

This is really important, because it means that any access to the instance members are still _reference-like_: by de-referencing `this` the lambda is accessing the original external variables, not copies of them!

This is the reason [some sources](https://en.cppreference.com/w/cpp/language/lambda#Lambda_capture) explain the capture `[this]` as capturing _the object by reference_. It's not exactly true, but it's close enough.

Now, this detail can bite you hard, especially if you fall for thinking that `[=]` means that "everything gets captured by-copy", as in the following example:
```
#include <iostream>
#include <functional>
#include <string>

using namespace std;
using Filter = std::function<bool(const std::string &)>;

class FilterFactory {
 public:
  Filter buildFilter(const string &name) {
    name_ = name;
    return [=](const std::string &name) { 
        return (name == name_);
    };
  }
 private:
  std::string name_;
};

int main() {
    FilterFactory factory;
    auto filter_adam = factory.buildFilter("adam");
    auto filter_eva  = factory.buildFilter("eva");
    cout << filter_adam("adam") << endl; // should have returned true, but returns false
    cout << filter_adam("eva") << endl;  // should have returned false, but returns true
    cout << filter_eva("adam") << endl;
    cout << filter_eva("eva") << endl;
}
```
Here the unsuspecting programmer may have expected `[=]` to capture everything by-copy, including class members. That would have made each lambda completely self-contained and independent of the original factory instance.

In reality, however, the default capture mode is not capturing `name_` at all; it is `this` that's getting captured, and the access to `name_` is actually being done through `this->name_`.

The code builds, but behavior is not what the coder expected. For all practical purposes `name_` is being accessed by reference, allowing lambdas to "see" changes to the variable  value that are done after the creation of the lambda.

And it gets worse: if the `factory` variable is destroyed before the lambdas, the `this` pointer stored in the lambdas becomes invalid and any access to the data members of the instance it used to point to becomes undefined behavior.

## An intuition of the closure class <a name="intuition"></a>

Before closing the article I'll talk a bit about the closure class. The idea is not to be rigorous, but just to provide the reader with an intuition of how lambdas get implemented by the compiler under the hood.

The first thing to state is that there's no single closure class. A custom closure class gets created automatically by the compiler for each lambda expression.

These compiler-generated classes cannot be seen or changed, because only the compiler knows what they look like. That's why it is frequently said that the closure objects have an anonymous type.

To take a peek under the hood we can implement our own closure class from the description in the lambda expression. Without loss of generality, lets say we were asked to compile a fragment like the one in the following fragment:
```
int foo;
bool bar;

auto lf = [foo, &bar](int factor) { return foo * bar * factor; };
```
From the expression we can deduce that upon construction the lambda needs to capture two variables, one by copy (an integer) and another by reference (a boolean). The lambda objects that get instantiated from the expression need to be callable with  a single input parameter (an integer) and must return a value after execution (another integer).

A possible implementation of the closure type for the lambda in the example above would be the following one:
```
class ClosureType {
 public:
    ClosureType(int foo, double &bar) : foo_{foo}, bar_{bar} {}

    double operator()(int factor) const {
        return foo_ * bar_ * factor;
    }
  private:
   int foo_;
   double &bar_;
};
```
Where you can see that:
* Captures become closure class constructor parameters. 
* By-copy captures like `foo` are stored within member variables in each instance.
* By-reference captures like `&bar`, are not stored themselves, but a reference to them is stored in the class.
* By overloading `operator()` the closure objects produced by the class become callable objects.
* The code body, return type and parameter list of the lambda expression become the body, return type and parameter list of the `operator()` overload.
* In  this case the `operator()` overload is a `const` method because the lambda is not `mutable`.

In the closure example above it is readily visible that captured variables get read on construction, while lambda parameters get passed to the lambda on execution.

While not very rigorous, our example above is good enough to get an idea of what a closure class may look like and how each part of the expression affects the implementation of the lambda.

The closure class example shows how the number and mode of captures impacts on the size of the closure objects that get instantiated from it: lambdas are more than just code (like a function would be), they carry a context with them, and the size of the context depends on the size and type of the captures.

By-reference captures are light, and only add a pointer to the size of the lambda object, but they don't guarantee that the captured object will exist at least as long as the lambda exists. By-copy captures, on the other hand, do guarantee the lifetime, but they can be heavy to move around because of the copy operation. Moving around lambdas will be at least as expensive as the most expensive by-copy capture they own.

In reality compilers can use a variety of implementations for lambdas depending on what's better suited for each case in particular. In particular, if a lambda does not capture any variable then it's usually cheaper to implement it with a simple anonymous function instead of an anonymous class; closure objects in this case become just pointers to that anonymous function.

That's why no-capture lambdas can be assigned to variables of correct pointer-to-function type variable, but lambdas that capture variables cannot.
```
int factor = 2;
auto no_capture_lambda = [](int n) { return 2 * n; }; 
auto with_capture_lambda = [factor](int n) { return factor * n; }; 

int (*f_ptr_1)(int) = no_capture_lambda;   // this is ok
int (*f_ptr_2)(int) = with_capture_lambda; // this fails to build
```

## Conclusion <a name="the_end"></a>

There's more to say about lambdas, of course, but this is probably enough for a good first bite.

For a deeper coverage you can visit the somewhat terse but extremely complete [cppreference.com page on the topic](https://en.cppreference.com/w/cpp/language/lambda). There you will find also lots of information on new developments in lambda functions in C++17 and beyond, which is something I intentionally left out.

If you have access to a copy of [Effective Modern C++](https://www.oreilly.com/library/view/effective-modern-c/9781491908419/), go read it. If you don't, go get it. The chapter on lambdas is extremely well written and informative, just like the rest of the book. You'll find there a neat trick to use move semantics with lambdas when generalized captures are not available in your compiler. 

I did not get a lot into passing lambdas around, but I did try to put some emphasis on the problems caused by capturing stuff by reference, either intentionally or by mistake. Understanding that is extremely important to safely execute lambdas in a different context from the one they were created in, and to determine how they will interact with other objects in multi-threaded code.

I hope you enjoyed this, until next time!