Asynchronous programming is becoming more and more important in today’s fast-paced world, where people want apps that work well and respond quickly. With the asyncio library in Python 3.4, it’s now easy for Python writers to use asynchronous programming to make high-performance apps that don’t block. In this blog post, we’ll look at the basics of asynchronous programming in Python with asyncio and give you a practical guide to help you learn this powerful technique.
Understanding Asynchronous Programming
Asynchronous programming is a way of writing code that lets multiple jobs run at the same time, without having to wait for one to finish before starting the next. This is especially helpful for jobs that have to wait for external resources, like web requests, database operations, or reading and writing files. By running these tasks in the background, apps can keep responding and keep working on other tasks while I/O-bound tasks finish.
The Evolution of Asynchronous Programming in Python
Before asyncio, Python writers had to use different libraries and methods to write code that ran in parallel. Here are some of these methods:
- Callbacks: A callback is a function that is passed to another function as an argument and is run when an asynchronous action is done. Callbacks can be used to make delayed code, but they can also make code hard to read and hard to understand. This is often called “callback hell.”
- Threads: The threading module in Python makes it possible for multiple threads of processing to run at the same time. But the CPython interpreter’s Global Interpreter Lock (GIL) means that only one thread at a time can run Python bytecode. This limits the speed benefits of multithreading.
- Greenlets: Greenlets are lightweight threads that work together. They are part of the greenlet library. The program, not the operating system, can schedule and run Greenlets. Greenlets offer more concurrency than Python threads, but they still need to be managed by hand.
- Tornado and Twisted: Tornado and Twisted are popular event-driven networking libraries for Python. They allow asynchronous I/O operations by using callbacks, generators, and other methods. But they each have their own ways of writing and can be hard to work with.
With the addition of asyncio to Python 3.4, writers now have a standard library for writing asynchronous code that is faster, easier to read, and more flexible than what they had before.
The Basics of Asyncio
Asyncio is a Python library that helps you write code that runs in parallel. It has an event loop, coroutines, and other tools to help you do this. Asyncio is made up of these main parts:
- Event Loop: The event loop is the most important part of asyncio. It plans and controls how coroutines and other asynchronous tasks are run.
- Coroutines: Coroutines are special functions that can be stopped and started again, letting other jobs run at the same time.
- Futures and Tasks: A future is the output of a computation that may not be done yet, and a task is how asyncio implements coroutines.Future instances.
- Asynchronous Context Managers and Iterators: Asyncio gives you asynchronous forms of context managers and iterators, which make it easier to work with asynchronous resources.
Writing Coroutines with Async/Await
In Python 3.5 and later, you can define coroutines using the async def syntax and the await keyword. The async def syntax creates a coroutine function, and the await keyword is used within a coroutine to pause its execution until the awaited coroutine completes. Here’s a simple example of a coroutine using async/await:
import asyncio
async def my_coroutine():
print("Start coroutine")
await asyncio.sleep(1)
print("End coroutine")
asyncio.run(my_coroutine())Running Coroutines with Asyncio
To run a coroutine, you need to use the asyncio event loop. There are several ways to do this:
asyncio.run(): The simplest way to run a coroutine is by using theasyncio.run()function, which takes a coroutine as an argument and runs it until completion. This function is available in Python 3.7 and later.loop.run_until_complete(): For Python 3.6 and earlier, you can use therun_until_complete()method of the event loop, which runs the coroutine until it finishes.asyncio.gather(): If you need to run multiple coroutines concurrently, you can use theasyncio.gather()function, which returns a single coroutine that completes when all the input coroutines have finished.
Creating Asynchronous Generators
Python 3.6 introduced asynchronous generators, which are similar to regular generators but work with asynchronous code. Asynchronous generators use the async def syntax and the yield keyword to produce values asynchronously. Here’s an example of an asynchronous generator that yields numbers with a delay:
import asyncio
async def async_generator():
for i in range(3):
await asyncio.sleep(1)
yield i
async def main():
async for number in async_generator():
print(number)
asyncio.run(main())Working with Asynchronous I/O
Asyncio provides several high-level APIs for working with asynchronous I/O, such as asyncio.open_connection() for establishing TCP connections, asyncio.start_server() for creating a TCP server, and asyncio.StreamReader and asyncio.StreamWriter for reading and writing data asynchronously. Here’s an example of an asynchronous TCP echo server:
import asyncio
async def echo_handler(reader, writer):
data = await reader.read(100)
message = data.decode()
addr = writer.get_extra_info("peername")
print(f"Received {message} from {addr}")
# Echo the message back to the client
writer.write(data)
await writer.drain()
print(f"Sent: {message}")
writer.close()
async def main():
server = await asyncio.start_server(
echo_handler, '127.0.0.1', 8888)
addr = server.sockets[0].getsockname()
print(f'Serving on {addr}')
async with server:
await server.serve_forever()
asyncio.run(main())In this example, the echo_handler coroutine reads data from the client, echoes it back, and closes the connection. The main() coroutine creates the server and starts serving on the specified address and port.
Asynchronous Context Managers and Iterators
Asyncio also supports asynchronous context managers and iterators, which make it easier to work with asynchronous resources. To define an asynchronous context manager, you need to implement the __aenter__() and __aexit__() methods in your class. For asynchronous iterators, you need to implement the __aiter__() and __anext__() methods. Here’s an example of using an asynchronous context manager to work with an asynchronous file:
import aiofiles
async def read_file(file_path):
async with aiofiles.open(file_path, mode='r') as f:
content = await f.read()
return content
async def main():
content = await read_file('example.txt')
print(content)
asyncio.run(main())Error Handling and Task Cancellation
Error handling in asyncio is similar to traditional synchronous programming. You can use try-except blocks within your coroutines to catch exceptions. To cancel a running task, you can call the cancel() method on the task object, which raises an asyncio.CancelledError in the coroutine. You can catch this exception to perform cleanup before the coroutine exits.
import asyncio
async def my_coroutine():
try:
await asyncio.sleep(5)
except asyncio.CancelledError:
print("Coroutine cancelled")
# Perform any necessary cleanup here
async def main():
task = asyncio.create_task(my_coroutine())
await asyncio.sleep(2)
task.cancel()
await asyncio.gather(task, return_exceptions=True)
asyncio.run(main())In this example, the my_coroutine() coroutine is cancelled after 2 seconds, and the CancelledError is caught to display a message.
Using Asyncio with Third-Party Libraries
Many popular third-party libraries now provide asyncio-compatible versions or extensions to support asynchronous programming. Some notable examples include:
- aiohttp: A fully asynchronous HTTP client and server library for Python built on top of asyncio.
- aiomysql and aiopg: Asynchronous libraries for working with MySQL and PostgreSQL databases, respectively, using asyncio.
- aioredis: An asyncio-based Redis client library for Python.
- Channels: An extension to the Django web framework that enables support for asynchronous views, WebSockets, and other asynchronous features using asyncio.
By using these libraries, you can further enhance the performance of your asynchronous Python applications and integrate with other services more efficiently.
Conclusion
Mastering asynchronous programming in Python with asyncio allows you to build efficient and responsive applications. By understanding the fundamentals of asyncio, writing and running coroutines, working with asynchronous I/O, using asynchronous context managers and iterators, handling errors and task cancellation, and leveraging third-party libraries, you’ll be well on your way to leveraging the power of asynchronous programming in your Python projects. Remember to keep experimenting, learning, and exploring new ideas as you continue to develop your asynchronous programming skills. Happy Pythoning!
