Concurrency in Python

Damini Aiyeer

8 months ago

Concurrency in Python | InsideAIML
Table of Contents
  • Introduction
  • What is concurrency?
  • Explanation
  • When to Use Concurrency

Introduction

          The concurrency in a fashionable English dictionary is described as simultaneous occurrence. The matters that are going on concurrently in Python are known as with the aid of distinctive names (thread, task, process) however at an excessive level, they all refer to a sequence of directions that run in order.
Each one can be stopped at certain points, and the CPU or brain that is processing them can switch to a different one. The state of each one is saved so it can be restarted right where it was interrupted.

What is concurrency?

          Concurrency is working on multiple things at the same time. In Python, this can be done in several ways:
  • With threading, by letting multiple threads take turns.
  • By firing off a task and continuing to do other stuff, instead of waiting for an answer from the network or disk. This is how asynchronous IO operates with the asyncio library.
  • With multiprocessing we’re using multiple processes. This way we can truly do more than one thing at a time using multiple processor cores. This is called parallelism.
This chapter focuses on the execution of concurrency for an operating system using Python.
The following program helps in the execution of concurrency for an operating system −
import os
import time
import threading
import multiprocessing

NUM_WORKERS = 4

def only_sleep():
   print("PID: %s, Process Name: %s, Thread Name: %s" % (
      os.getpid(),
      multiprocessing.current_process().name,
      threading.current_thread().name)
   )
   time.sleep(1)

def crunch_numbers():
   print("PID: %s, Process Name: %s, Thread Name: %s" % (
      os.getpid(),
      multiprocessing.current_process().name,
      threading.current_thread().name)
   )
   x = 0
   while x < 10000000:
      x += 1
for _ in range(NUM_WORKERS):
   only_sleep()
end_time = time.time()
print("Serial time=", end_time - start_time)

# Run tasks using threads
start_time = time.time()
threads = [threading.Thread(target=only_sleep) for _ in range(NUM_WORKERS)]
[thread.start() for thread in threads]
[thread.join() for thread in threads]
end_time = time.time()

print("Threads time=", end_time - start_time)

# Run tasks using processes
start_time = time.time()
processes = [multiprocessing.Process(target=only_sleep()) for _ in range(NUM_WORKERS)]
[process.start() for process in processes]
[process.join() for process in processes]
end_time = time.time()

print("Parallel time=", end_time - start_time)

Output

The above program generates the following output −
Python Concurrency Program Output | Insideaiml

Explanation

          “multiprocessing” is a package similar to the threading module. This package supports local and remote concurrency. Due to this module, programmers get the advantage to use multiple processes on the given system.

When to Use Concurrency

          You’ve covered a lot of ground here, so let’s review some of the key ideas and then discuss some decision points that will help you determine which, if any, concurrency module you want to use in your project.
The first step of this process is deciding if you should use a concurrency module. concurrency always comes with extra complexity and can often result in bugs that are difficult to find.
Hold out on adding concurrency until you have a known performance issue and then determine which type of concurrency you need. As Donald Knuth has said, “Premature optimization is the root of all evil (or at least most of it) in programming.”
Once you’ve decided that you should optimize your program, figuring out if your program is CPU-bound or I/O-bound is a great next step. Remember that I/O-bound programs are those that spend most of their time waiting for something to happen while CPU-bound programs spend their time processing data or crunching numbers as fast as they can.
As you saw, CPU-bound problems only really gain from using multiprocessing. threading and asyncio did not help this type of problem at all.
For I/O-bound problems, there’s a general rule of thumb in the Python community: “Use asyncio when you can, threading when you must.” asyncio can provide the best speed up for this type of program, but sometimes you will require critical libraries that have not been ported to take advantage of asyncio. Remember that any task that doesn’t give up control to the event loop will block all of the other tasks.
  
Liked what you read? Then don’t break the spree. Visit our insideAIML blog page to read more awesome articles. 
Or if you are into videos, then we have an amazing Youtube channel as well. Visit our InsideAIML Youtube Page to learn all about Artificial Intelligence, Deep Learning, Data Science and Machine Learning. 
Keep Learning. Keep Growing. 

Submit Review