In this post, we will go over how to create thread-safe generators in Python. This post is heavily based on this excellent article.
Generator makes life so much easier when coding in Python, but there is a catch; raw generators are not thread-safe. Consider the example below:
We see that the generator does not produce correct output when multiple threads are accessing this at the same time.
One easy way to make it thread-safe is by creating a wrapper class that simply lets only one thread to execute the generator's next method at any given time with threading lock. This is shown below:
Note that the generator now is thread-safe but doesn't execute its next method in parallel. You can also use Python's decorator to make it look even easier, although it basically does the same thing.