CALLBACK REQUEST   |    CALL (+91) 97633 96156

Download our e-book of Introduction To Python

Related Blog

Matplotlib - Transforms Microsoft Cognitive Toolkit CNTK Introduction The Word2Vec Model Matplotlib - Scatter Plot Image Recognition using TensorFlow Violin plots Meta Pseudo Label H2O - AutoML Understanding LSTM Networks The Skip-gram Model View More

Top Discussion

about installing the softwere Exception Type: JSONDecodeError at /update/ Exception Value: Expecting value: line 1 column 1 (char 0) How to leave/exit/deactivate a Python virtualenvironment What is the difference between covariance and correlation? TypeError: 'int' object is not subscriptable Which are different modes to open a file ? How to handle imbalanced data and achieve good performance? What does it mean to cross-validate a machine learning model? Join Discussion

Top Courses

Machine Learning with Python & Statistics

4 (4,001 Ratings)

218 Learners

Webinars

More webinars

Python Memory Management

Balasaheb Garule

6 months ago

Python Memory Management | insideAIML
Table of Content
Introduction
Memory Management in Python
Why Do We Need to Know Memory Management?
Python Memory allocation
  • Static Memory Allocation:
  • Dynamic Memory Allocation:
Python Program execution
  • Compilation
  • Interpreter
Reference Counting
  • How to view reference counts in Python?
Python Garbage Collection
Python Objects in Memory
Some practical Python code tips:
Summary

Introduction

          Python is an easy to understand, dynamic, interpreted language, and high-level language. There is no need to care about memory management in python. But it is required to know about memory management in python. So we know how memory is managed in python?
It is very important to manage memory when available memory is very little in size. Programs may crash if memory is not properly managed.

Memory Management in Python

          Memory management in Python is the process of sharing, redistributing, and linking memory so that all the various processes can run smoothly and have access to a variety of system resources. Memory management includes memory clearing of obsolete objects.
In Python, a memory manager manages these types of tasks by working from time to time to clean, share, and manage memory. In python, objects are managed by using reference counts. This means that the memory manager keeps track of the number of clues in each item in the system. When referring to an item's reference drops to zero, which means that the item is no longer in use, the garbage collector (part of the memory manager) automatically releases the memory from that item.
The user does not have to worry about memory management in python as the sharing and memory allocation process is completely automatic. Recovered memory can be used for other purposes.

Why Do We Need to Know Memory Management?

          Python memory manager manages memory automatically but for efficient program execution and to improve the speed of execution software developers must know the memory management in python. Improper memory management leads to downgrade performance, memory leaks, and makes the execution process time consuming.
Although most memory management in python is done by Python Memory Manager, understanding the best coding methods and how Python Memory Manager works can lead to efficient and secure code.

Python Memory allocation  

          There are two types of python memory allocation
  • Stack (Static) Memory Allocation
  • Heap (Dynamic) Memory  Allocation

Static Memory Allocation

          Memory is allocated at the compile time, for this Operating system using stack data structure is called static memory allocation. C, C++ use this type of techniques. Memory is allocated at the beginning of the execution, and it is fixed when the program is created. Memory is allocated for global variables, static variables and functions.
e.g. 
static int n = 30; #static memory management in python
Memory is deallocated automatically based on the scope of variables. If the scope is ended then memory is deallocated. If the memory is not free then a memory leakage problem may occur. Stack overflow may occur in static allocation if the memory gets full.

Dynamic Memory Allocation

          Memory allocated at time of execution is called dynamic memory allocation. 
Pointer in C is an example of dynamic memory allocation.
#dynamic memory management in python

int * d;

d = new int;
The good thing about Python is that everything in Python is objective. This means that Powerful Memory Sharing is under Memory Management in python. When items are no longer needed, the Python Memory manager will automatically restore the memory to them.

Python Program Execution

          A Python program is a block of code. This block of code is executed in the execution frame.
Python programs use the .py extension for saving programs. 
There are two steps in python program execution.
  • Compilation
  • Interpreter 
Python Program Execution | insideAIML

Compilation

          Python .py file is converted into byte code which uses the .pyc extension. All this process is done internally. In this process, syntax checking is done.

Interpreter

          Interpreters convert the .pyc byte code into machine code (binary code).Python Virtual Machine (PVM) is responsible for this conversion.

Reference Counting

          Reference means names or objects are pointed to another object. We can count references by how many times objects are referenced by other objects.  Reference count increases when other objects refer to the object.
e.g.
a = 100 then the reference count is 1.
The reference count is increased by 1      
Ref count decreases when reference is changed. the reference count is also decreased by using the del keyword. del keyword delete the reference not the object, so reference count decreases.
When an object is out of scope then reference count decreases.
e.g.
def display():
	str1 = “insideaiml”      # ref count + 1
	print(str1)
display()
# ref count -1 
# insideaiml is out of scope
When the reference count equals zero, the object is deallocated or removed from memory.
Global namespace object never goes out of scope.

How to view reference counts in Python?

          Using the sys module we can see the reference count.
e.g.
import sys
str1 = 'hello'
print(sys.getrefcount(str1))
Output
4
The Python memory manager did not remove the instance from memory. The instance does not have a reference count of zero because it has a reference to itself. This is called a reference cycle problem. To solve this problem we use python garbage collection.

Python Garbage Collection

          If an object is no longer available then python free up the memory space.it is called garbage collection. The garbage collector must keep track of all objects in memory. A new object that starts its life in the first generation of garbage collectors. When Python performs a garbage collection process in one generation, and it's an object that survives, goes up, and after a while, the older generation.. The Python garbage collector must have a total of three generations, and the object will move in a younger generation when it survives the garbage collection process in the current generation to the next.
A threshold for the number of objects is set for each generation of the garbage collector module. If the number of objects exceeds the threshold value, the garbage collector will eventually trigger the collection process. For all objects that survive this process, they will be transferred from the previous generation to the next.
Unlike the reference counting mechanism, you can control the behaviour of the generational garbage collector in Python. This includes changing the threshold value to start the garbage collection process in the code. Alternatively, you can manually start the garbage collection process to enable or disable it altogether.

Python Objects in Memory

         In python, there is no variable it has names. The object has multiple names like ‘a’, ’b’. Each name is just a label of a python object. Numbers, strings are simple objects or containers are dictionaries, lists, or user-defined classes. Python is a dynamically typed language i.e. it does not need to declare the variable types.
e.g.
text = "Welcome to insideAIML"
print(text)
Output:
Welcome to insideAIML
e.g.
del(text)
print(text)
Output: 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'text' is not defined
In the first program, the object knows the reference but after del operation the reference is removed and it raises the error.
In memory management in python garbage collection is a completely automated process.

Some Practical Python Code Tips:

  • Use join to add items to the list
  • Avoid using operator +. Better use% or format
  • Use Generators
  • Place the test outside the loop
  •  Assign function to local variables
  • Use built-in functions and libraries instead of row codes

Summary

          In this blog, we learn about Memory Management in Python. We cover the topic like memory allocation in python, how memory is managed in python? , reference count, python garbage collection, python objects in memory.
We also learn about practical Python code tips for memory management.
If you have any doubt about insideAIML’s python numbers blog, drop a comment below and we will get back to you.
           
Liked what you read? Then don’t break the spree. Visit our blog page to read more awesome articles. 
Or if you are into videos, then we have an amazing Youtube channel as well. Visit our InsideAIML Youtube Channel to learn all about Artificial Intelligence, Deep Learning, Data Science and Machine Learning. 
Keep Learning. Keep Growing. 

Submit Review

Request a callback
Call us 97633 96156