Concurrency in Python: Threads, processes, and asyncio

Length: 1 day

Description

Want to do more than one thing at a time? The two traditional ways to handle this in a programming language are threads and processes. Python has long supported both types of concurrency. But Python has also suffered from a number of limitations in its threading, ranging from a simple API to the GIL (global interpreter lock), which has been removed (if only experimentally) in recent versions of Python.

In this course, we’ll explore threading via the traditional API, including such topics as locks, mutexes, and queues. We’ll discuss the recent moves to free threading, and what that means for our programs now and in the future. We’ll also explore the “futures” API, and how we can use it for more natural invocations of threaded programs. Following that, we’ll talk about the “multiprocessing” module, and how it’s both similar to and different from threading

Then we’ll pivot to asyncio, the newest form of concurrency in Python. We’ll talk about the thinking behind asyncio, how it works, and how to use it — both for simple tasks, and for more complex ones. We’ll look at a number of common appliations for asyncio, including network communication, and discuss best practices for woriking with it. We’ll also discuss where it’s inappropriate, and shouldn’t be used. We’ll also look at the evolution of asyncio’s API, and some of the changes that have taken place in the last few years.

You’ll come out of this class more aware of how concurrency works in Python, and how to use it in day-to-day work. A large part of the course will be hands-on exercises, which will help participants to learn and understand the regular expression syntax.

Like all of my courses, this is taught without slides. Instead, I live-code into a Jupyter notebook and files that are available in real time, and which I distribute to participants at the end of the course.

Let’s talk about how to customize this course for your team! Set a meeting at https://savvycal.com/reuven/corp-training.

Audience

Experience writing Python functions and classes, including passing functions as arguments.

Syllabus

• What are threads?

• The threading API

• Passing arguments

• Thread-safe data structures (i.e., queues)

• Sharing data across threads

• Locks

• Exceptions and threads

• ThreadPoolExecutor

• executor and map

• The GIL and free-threading Python

• Writing threaded applications

• Are threads better?

• multiprocessing

• Sharing data across processes

• Writing applications with processes

• ProcessPoolExecutor

• Comparing threads and processes

• Subinterpreters

• How asyncio works

• The evolution of asyncio’s API

• Threads vs. processes vs. asyncio

• Coroutines and tasks

• async and await

• Attaching to the event loop

• Capturing return values

• Looping with asyncio

• Context managers with asyncio

• Web scraping

• Network servers

• Handling exceptions with asyncio

•