Profiling Memory Usage in Python with memory_profiler

Hi Pythonistas!

Today we will learn about a package called memory_profiler. memory_profiler is a Python package that allows you to profile memory usage in your code. It provides a detailed analysis of memory consumption by tracking memory allocations and deallocations. By profiling your code with memory_profiler, you can pinpoint specific functions or lines of code that contribute to memory growth, enabling you to optimize memory usage and improve the performance of your application.

Installation

pip install memory-profiler

To profile a function we just need a decorator called profile. And decorate the required function with this profile

Code

from memory_profiler import profile

@profile
def function():
    a = list(range(1000))
    b = list(range(100000000))
    del a
    del b

function()

Here the function is decorated with profile. Let analys the memory usage by running the script

python -m memory_profiler <filname>

For my case

python -m memory_profiler profile_example.py

Output

Filename: profile_example.py

Line # Mem usage Increment  Occurrences   Line Contents

=============================================================

  4 41.9 MiB 41.9 MiB       1   @profile

  5                                     def function():

  6 41.9 MiB  0.0 MiB       1   a = list(range(1000))

  7   3788.2 MiB   3746.3 MiB       1   b = list(range(100000000))

  8   3788.7 MiB  0.5 MiB       1   del a

  9  8.3 MiB  -3780.3 MiB       1   del b

We can see total memory usage and increment of memory in each line. When b is created a large memory is used and when we delete it that memory is released

Using this package we can identify the memory usage of functions very easily and you can optimize the code if required

Hope you have learned something from this post. Please share your valuable suggestion and topics to learn with afsal@parseltongue.co.in