20. Multithreading and Multiprocessing

Multithreading and multiprocessing help Python programs perform multiple tasks simultaneously.

Used for:

  • Faster execution
  • Background tasks
  • Parallel processing
  • Network applications

Concurrency vs Parallelism

ConceptMeaning
ConcurrencyMultiple tasks progress together
ParallelismMultiple tasks run at same time

What is a Thread?

A thread is a lightweight unit of execution inside a process.

Threads share:

  • Memory
  • Resources

Useful for:

  • I/O operations
  • File handling
  • Network requests

Multithreading in Python

Python provides:

threading

module.


Creating a Thread

import threading

def task():
print("Thread running")

thread = threading.Thread(target=task)

thread.start()

thread.join()

print("Main program finished")

Multiple Threads Example

import threading

def print_numbers():

for i in range(5):
print("Number:", i)

def print_letters():

for letter in "ABCDE":
print("Letter:", letter)

t1 = threading.Thread(
target=print_numbers
)

t2 = threading.Thread(
target=print_letters
)

t1.start()
t2.start()

t1.join()
t2.join()

Thread with Arguments

import threading

def greet(name):
print("Hello", name)

thread = threading.Thread(
target=greet,
args=("Aditya",)
)

thread.start()

Thread Benefits

✅ Faster I/O operations
✅ Background execution
✅ Improved responsiveness


What is a Process?

A process is an independent program execution unit.

Each process has:

  • Separate memory
  • Separate resources

Useful for:

  • CPU-intensive tasks
  • Heavy computations

Multiprocessing in Python

Python provides:

multiprocessing

module.


Creating a Process

from multiprocessing import Process

def task():
print("Process running")

process = Process(target=task)

process.start()

process.join()

Multiple Processes Example

from multiprocessing import Process

def square():
for i in range(5):
print(i * i)

def cube():
for i in range(5):
print(i * i * i)

p1 = Process(target=square)
p2 = Process(target=cube)

p1.start()
p2.start()

p1.join()
p2.join()

Thread vs Process

FeatureThreadProcess
MemorySharedSeparate
SpeedFasterSlower
Best ForI/O tasksCPU tasks
CommunicationEasierMore complex

Synchronization

Synchronization prevents conflicts when multiple threads access shared data.


Race Condition

Occurs when threads modify shared resource simultaneously.


Example Problem

counter = 0

Multiple threads updating this variable can create incorrect results.


Lock in Python

Use:

Lock

to synchronize threads.


Example Using Lock

import threading

counter = 0

lock = threading.Lock()

def increment():

global counter

for _ in range(1000):

lock.acquire()

counter += 1

lock.release()

threads = []

for i in range(5):

thread = threading.Thread(
target=increment
)

threads.append(thread)

thread.start()

for thread in threads:
thread.join()

print(counter)

Using with Lock

Cleaner approach.

import threading

lock = threading.Lock()

with lock:
print("Critical section")

Semaphore

Controls access to limited resources.

import threading

semaphore = threading.Semaphore(2)

Async Programming (asyncio)

Async programming allows non-blocking execution.

Useful for:

  • APIs
  • Web servers
  • Network applications

Python provides:

asyncio

module.


Basic Async Function

import asyncio

async def hello():

print("Hello")

asyncio.run(hello())

Async with await

import asyncio

async def task():

print("Task started")

await asyncio.sleep(2)

print("Task completed")

asyncio.run(task())

Running Multiple Async Tasks

import asyncio

async def task1():

await asyncio.sleep(1)

print("Task 1 done")

async def task2():

await asyncio.sleep(2)

print("Task 2 done")

async def main():

await asyncio.gather(
task1(),
task2()
)

asyncio.run(main())

Async vs Threading

FeatureAsyncioThreading
Best ForAsync I/OConcurrent tasks
Uses ThreadsNoYes
Memory UsageLowerHigher

Thread Pool

Python provides thread pools for managing threads efficiently.


Example: ThreadPoolExecutor

from concurrent.futures import ThreadPoolExecutor

def square(n):
return n * n

with ThreadPoolExecutor() as executor:

results = executor.map(
square,
[1, 2, 3, 4]
)

print(list(results))

Process Pool

from concurrent.futures import ProcessPoolExecutor

def cube(n):
return n * n * n

with ProcessPoolExecutor() as executor:

results = executor.map(
cube,
[1, 2, 3]
)

print(list(results))

Practical Example

Download Multiple URLs Concurrently

import threading
import requests

urls = [
"https://example.com",
"https://example.org"
]

def download(url):

response = requests.get(url)

print(
url,
response.status_code
)

threads = []

for url in urls:

thread = threading.Thread(
target=download,
args=(url,)
)

threads.append(thread)

thread.start()

for thread in threads:
thread.join()

Advantages of Multithreading and Multiprocessing

✅ Faster execution
✅ Better CPU utilization
✅ Improved responsiveness
✅ Handles multiple tasks efficiently


Best Practices

✅ Use threads for I/O tasks
✅ Use processes for CPU tasks
✅ Synchronize shared resources
✅ Avoid unnecessary threads
✅ Use async for network applications


Summary

In this chapter you learned:

✅ Threads
✅ Processes
✅ Synchronization
✅ Locks and semaphores
✅ Async programming with asyncio
✅ Thread pools and process pools