Multiprocessing in Python

Hi Pythonistas!

In the last post we learned about multithreading in Python. Today we will learn about multiprocessing libraries. Unlike the multithreading, multiprocessing can be used for I/O-bound tasks and CPU-bound tasks and also it is not limited by GIL as it is using different processes.

Let us learn with a simple example

Create new process

import multiprocessing

process = multiprocessing.Process(target=<callable>, args=<argument to function>)

Start a process

process.start()

Join a process

process.join()

Example for single process

import time

def print_numbers():
    for i in range(5):
        time.sleep(1)
        print(f"print_numbers : {i}")

def print_letters():
    for letter in 'ABCDE':
        time.sleep(1)
        print(f"print_letters: {letter}")

print('Single process')
start_time = time.time()
print_numbers()
print_letters()
print("Time taken single process: ", time.time()-start_time)

Output

Single process

print_numbers : 0

print_numbers : 1

print_numbers : 2

print_numbers : 3

print_numbers : 4

print_letters: A

print_letters: B

print_letters: C

print_letters: D

print_letters: E

Time taken single process:  10.01513123512268

Using multiprocessing

import multiprocessing

process1 = multiprocessing.Process(target=print_numbers)
process2 = multiprocessing.Process(target=print_letters)

start_time =  time.time()
process1.start()
process2.start()

process1.join()
process2.join()

print("Time taken with multiprocessing: ", time.time()-start_time)

Output

print_numbers : 0

print_letters: A

print_numbers : 1

print_letters: B

print_numbers : 2

print_letters: C

print_letters: D

print_numbers : 3

print_numbers : 4

print_letters: E

Time taken with multiprocessing:  5.016874551773071

This is only the basics of the multiprocessing library. Using queues and pipes we can communicate between processes also. We can make posts on this in future.

I hope you have learned something from this post.Please share your valuable suggestions with afsal@parseltongue.co.in