Concurrency in Python

Damini Aiyeer

a year ago

Concurrency in Python | Insideaiml
Concurrency in Python | Insideaiml
    The concurrency in a fashionable English dictionary is described as simultaneous occurrence. The matters that are going on concurrently in Python are known as with the aid of distinctive names (thread, task, process) however at an excessive level, they all refer to a sequence of directions that run in order.
     Each one can be stopped at certain points, and the CPU or brain that is processing them can switch to a different one. The state of each one is saved so it can be restarted right where it was interrupted.

What is concurrency?

Concurrency is working on multiple things at the same time. In Python, this can be done in several ways:
  • With threading, by letting multiple threads take turns.
  • By firing off a task and continuing to do other stuff, instead of waiting for an answer from the network or disk. This is how asynchronous IO operates with the asyncio library.
  • With multiprocessing we’re using multiple processes. This way we can truly do more than one thing at a time using multiple processor cores. This is called parallelism.
 This chapter focuses on the execution of concurrency for an operating system using Python.
The following program helps in the execution of concurrency for an operating system −
import os
import time
import threading
import multiprocessing

NUM_WORKERS = 4

def only_sleep():
   print("PID: %s, Process Name: %s, Thread Name: %s" % (
      os.getpid(),
      multiprocessing.current_process().name,
      threading.current_thread().name)
   )
   time.sleep(1)

def crunch_numbers():
   print("PID: %s, Process Name: %s, Thread Name: %s" % (
      os.getpid(),
      multiprocessing.current_process().name,
      threading.current_thread().name)
   )
   x = 0
   while x < 10000000:
      x += 1
for _ in range(NUM_WORKERS):
   only_sleep()
end_time = time.time()
print("Serial time=", end_time - start_time)

# Run tasks using threads
start_time = time.time()
threads = [threading.Thread(target=only_sleep) for _ in range(NUM_WORKERS)]
[thread.start() for thread in threads]
[thread.join() for thread in threads]
end_time = time.time()

print("Threads time=", end_time - start_time)

# Run tasks using processes
start_time = time.time()
processes = [multiprocessing.Process(target=only_sleep()) for _ in range(NUM_WORKERS)]
[process.start() for process in processes]
[process.join() for process in processes]
end_time = time.time()

print("Parallel time=", end_time - start_time)

Output

The above program generates the following output −
Python Concurrency Program Output | Insideaiml
Python Concurrency Program Output | Insideaiml

Explanation

      “multiprocessing” is a package similar to the threading module. This package supports local and remote concurrency. Due to this module, programmers get the advantage to use multiple processes on the given system.

When to Use Concurrency

       You’ve covered a lot of ground here, so let’s review some of the key ideas and then discuss some decision points that will help you determine which, if any, concurrency module you want to use in your project.
    The first step of this process is deciding if you should use a concurrency module. concurrency always comes with extra complexity and can often result in bugs that are difficult to find.
     Hold out on adding concurrency until you have a known performance issue and then determine which type of concurrency you need. As Donald Knuth has said, “Premature optimization is the root of all evil (or at least most of it) in programming.”
      Once you’ve decided that you should optimize your program, figuring out if your program is CPU-bound or I/O-bound is a great next step. Remember that I/O-bound programs are those that spend most of their time waiting for something to happen while CPU-bound programs spend their time processing data or crunching numbers as fast as they can.
          As you saw, CPU-bound problems only really gain from using multiprocessing. threading and asyncio did not help this type of problem at all.
For I/O-bound problems, there’s a general rule of thumb in the Python community: “Use asyncio when you can, threading when you must.” asyncio can provide the best speed up for this type of program, but sometimes you will require critical libraries that have not been ported to take advantage of asyncio. Remember that any task that doesn’t give up control to the event loop will block all of the other tasks.
For more related articles and courses visit InsideAIML.

Submit Review