Understanding the Magic of Integer and String Interning in Python

Hi Pythonistas,

See the example below

>>> a = 1
>>> b = 1
>>> a == b
True
>>> a is b
True
>>> c = 1000
>>> d = 1000
>>> c == d
True
>>> c is d
False
>>>

>>> a = "hello"
>>> b = "hello"
>>> c = "world"
>>> d = "world"
>>> e = a + c
>>> f = a + c
>>> a == b
True
>>> a is b
True
>>> c == d
True
>>> c is d
True
>>> e == f
True
>>> e is f
False

This code makes us confusion why some cases work and some don’t. Today we will learn the reason behind this confusion, which is called interning. Behind the scenes, Python employs a clever optimization technique called "interning" for both strings and small integers. Interning is like hidden magic that can significantly impact your code's performance and memory usage.

At its core, interning is the process of storing only one copy of each unique object in memory and reusing it whenever needed. This optimization is particularly useful for immutable objects like strings and small integers, which are frequently used in Python programs.

String Interning

When you define a string in your code, Python checks if an identical string already exists in the "string intern pool." This pool acts like a repository for unique string values. If a match is found, the new string reference points to the existing object in the intern pool, saving memory by avoiding duplicate copies. If no match is found, the new string is added to the pool.

It's important to note that string interning happens automatically for string literals and strings created using the str() constructor. However, dynamically generated strings during runtime may not be interned.

Integer Interning

Similarly, Python interns small integers to reduce memory overhead. Integers within the range of -5 to 256 are pre-created and stored in memory at startup. When you create an integer within this range, Python ensures that all references to the same value point to the same object in memory.

Hope you have understood what is cause of the above confusion. Please share your valuable suggestions and topic you want to learn with afsal@parseltongue.co.in