Creating Lambdas in a Loop

There is one pitfall in Python with lambda functions defined in a loop regarding capture of the loop variable. I'll show a few examples.

You might write something that boils down to something like the following. Using a list comprehension or a loop one creates lambda functions. Here I just have them print different strings from a list. Then I call all functions.

functions = [lambda: print(text) for text in ["One", "Two", "Three"]]
for function in functions:
    function()

I would expect that this prints “One, Two Three”, but it actually prints “Three, Three, Three”. Huh?

The problem here is that the lambda doesn't resolve the reference text when it is defined. Rather it just keeps a latent reference to the variable within the function that it is defined with.

We can boil this down to the following code, which might be easier to understand:

text = "One"
function = lambda: print(text)
text = "Two"
function()

This will print “Two” because that is what the reference text points to at the point of calling of the function.

In order to capture the value of the variable that it has at the time of definition, one can use a function argument with a default argument.

text = "One"
function = lambda text=text: print(text)
text = "Two"
function()

This will create a local parameter reference text within the lambda. Therefore it prints “One”. We can also make it more explicit by changing the name of the local variable:

text = "One"
function = lambda captured_text=text: print(captured_text)
text = "Two"
function()

This shows the subtle way that references work in Python. This continues the thoughts that I have outlined in my previous post Python has no Variables.

Comparison to C++

We can also have a look a C++ for comparison. Writing something like this in C++ would look like this:

#include <iostream>
#include <string>

int main() {
  std::string text{"One"};
  auto function = []() { std::cout << text << std::endl; };
  text = "Two";
  function();
}

This doesn't work, we haven't captured text in any way. GCC will complain here:

lambdas.cpp:6:39: error: 'text' is not captured
    6 |   auto function = []() { std::cout << text << std::endl; };
      |                                       ^~~~
lambdas.cpp:6:20: note: the lambda has no capture-default
    6 |   auto function = []() { std::cout << text << std::endl; };
      |                    ^
lambdas.cpp:5:15: note: 'std::string text' declared here
    5 |   std::string text{"One"};
      |               ^~~~

So we have to capture it either way. There are two ways to capture: by value or by reference. And that already forces us to make a concious decision. If we capture it by value, the lambda definition will look like this:

auto function = [text]() { std::cout << text << std::endl; };

We could alternatively also capture everything by value:

auto function = [=]() { std::cout << text << std::endl; };

Both versions will print “One” to the screen. If we capture by reference, it will look like this:

auto function = [&text]() { std::cout << text << std::endl; };

Alternatively one can just capture everything by reference:

auto function = [&]() { std::cout << text << std::endl; };

Both of these versions will print “Two”.

The behavior here is not surprising at all because we have to explicitly say what we want. When we capture by reference, we accept that there are changes to the outside object.

Python only has references, but some objects are immutable. Therefore there is a bit of confusion. And often these sort of problems surface rather late and are pretty hard to debug. This one has bitten me once a while back. Now it came back and it took me a while to realize what I have done. Hopefully I won't do it a third time.