Python Data Model

All about Pythonic Class: The Life Cycle®

Python Data Model — Part 2[b]

Image: source

This is the second part of “All about Pythonic Class”. Now, our “Python Data Model” series —

1. Objects, Types, and Values; Python Data Model — Part 1
2. All about Pythonic Class: The Birth and Style; Python Data Model — Part 2[a]
3. All about Pythonic Class: The Life Cycle; Python Data Model — Part 2[b]

In the previous chapter, we have shown How to write a basic class, What is the relation between class and object, and the differences between new-style and old-style class. We will start this chapter from where we left the previous one-

4. What actually happens when we define a class

So what happens when we declare a class? How does the Python interpreter interpret it?

Two special methods are called under the hood when we call a class to form an object.

• First, the interpreter calls __new__ method which is the ‘true’ constructor of a class. This special method then creates an object, in an unnamed location into the memory, with the correct type.
• Then __init__ is called to initialize the class object with the previous object created by __new__. Most of the times we don’t need to declare a __new__ special method without an obvious reason. It mainly takes place in the background. Instantiating an object with __init__ is the most common and standard practice that is sufficient to serve most of our purposes.

It is an intricate process that creates a class behind the scene. It will also need in-depth knowledge of the design of the Python interpreter and the implementation of the language at its core. For the sake of simplicity, we will describe the process from a high-level point of view and write hypothetical or pseudo-codes of the backend process.

When we call a class to form an object, the interpreter calls type. It takes three parameters--
type(classname, superclass, attribs) or
type("", (), {})

where the class name is the string representation of the declared class, the superclass is the tuple representation of class(es) from which our class will inherit properties and attributes is the dictionary representation of the class.__dict__.
Let's declare a class normally

class Footballer():
    game = "Football"
 
    def __init__(self, name, club):
        self.name = name
        self.club = club
 
    def a_method(self):
        return None
 
print(Footballer.__dict__)    
 
# Output
'''
{'__module__': '__main__', 'game': 'Football', '__init__': <function Footballer.__init__ at 0x7f10c4563f28>, 'a_method': <function Footballer.a_method at 0x7f10c45777b8>, '__dict__': <attribute '__dict__' of 'Footballer' objects>, '__weakref__': <attribute '__weakref__' of 'Footballer' objects>, '__doc__': None}
'''

Now we will write the same type of class using type metaclass

def outer_init(self, name, club):
    self.name = name
    self.club = club
 
Footballer1 = type("Footballer1", (), {"game":"Soccer",  "__init__": outer_init, "b_method": lambda self: None})
print(Footballer1.__dict__)  
print(Footballer1.game)
 
# Output
'''
{'game': 'Soccer', '__init__': <function outer_init at 0x7f10c4510488>, 'b_method': <function <lambda> at 0x7f10c4510f28>, '__module__': '__main__', '__dict__': <attribute '__dict__' of 'Footballer1' objects>, '__weakref__': <attribute '__weakref__' of 'Footballer1' objects>, '__doc__': None}
Soccer
'''

Aha! we have just made a regular user-defined class using type metaclass!! What actually is going on here?
When type is called, its __call__ method is executed that runs two more special methods--

1. type.__new__(typeclass, classname, superclasses, attributedict)the constructor
2. type.__init__(cls, classname, superclasses, attributedict) the initiator/initializer

__new__(cls, [...) Special method __new__ is the very first method that is called when an object is created. Its first argument is the class itself following other arguments needed to form it. This function is rarely used and by default called by the interpreter during object creation.
__init__(self, [...) The initializer special method initializes a class and gives the default attributes of an object. This method takes an object as the first parameter following other arguments for it. This is the most frequently used special method in the Python world.

The following pseudo-code shows how the Python interpreter creates an object from a class. Remember we have omitted the low-level things done by the interpreter for the sake of a basic understanding. Have a look at the self-explanatory code instructions

class Bar():
    def __init__(self, obVar):
        self.obVar = obVar
 
>>> obj = Bar("a string") ----------------[1]
# putting  object parameter into a temporary variable
>>> tmpVar = Bar.__new__(Bar, "another string") 
>>> type(tmpVar)
>>> __main__.Bar
>>> tmpVar.obVar -------------------------[2]
# the class is not initialised yet 
# It will throw errors
>>> AttributeError Traceback (most recent call last)
....
AttributeError: 'Bar' object has no attribute 'obVar'
------------------------------------------------------
# initialise a temporary variable 
>>> tmpVar.__init__("a string") ---------------[3]
>>> tmpVar.obVar
>>> 'another string'
>>> obVar = tmpVar 
>>> obVar.obVar
>>> 'another string' 

The Obj is behaving as we expected [1]. But, when we createdtmpVar by calling Bar.__new__ and trying to access obVar, it throws an error [2]. Thus, we need to initialize with the __init__ and after that, it will work nicely as the previous variable [3] .

P.S.- The method with double underscores on both sides of his name is known as special methods or magic methods. The special method is an “implementation detail” or low-level detail of CPython. Why do we call them magic methods? Because these methods add “magic” to our class. We will have two separate chapters for magic methods.

5. Life Cycle of a Class

So far we have discussed almost every basic concept of a Python class. What we are going to discuss in this segment may be counted as a repetition of the previous segment, we are going to discuss more subtle details of a class.

When we run a Python programme, it spawns objects either from the built-in or user-defined classes in runtime. Each object needs memory once it is created. Python does not need the object when it accomplishes its assigned task. It becomes a piece of unwanted information or garbage.

On the other hand, the Python interpreter needs to free up memory periodically for further computation, space for new objects, programme efficiency, and memory security. When a piece of ‘garbage object’ is disposed it ceases to exist in the memory. The question is how long does a Python class exist and when does it cease to exist? The simplest answer is an object or class exists as long it

1. Holds a reference to its attributes or derivatives and
2. It is referenced by another object.

When neither of the two criteria is met, an object or class ceases to exist. The answer is the simplest one but a lot of things go on behind the scene. We are going to put light on a few of them.

The following class is designed to connect to a web server and print the status code, server info and then we will close the connection. We will ‘pack’ attributes into Response class from another namespace which will be a metaclass of type; as expected.

import requests
 
# Another name space
var = "a class variable."
 
def __init__(self, url):
    self.__response = requests.get(url)
    self.code = self.__response.status_code
 
def server_info(self):
    for key, val in self.__response.headers.items():
        print(key, val)
 
def __del__(self): ---------------------- [1]
    self.__response.close()
    print("[!] closing connection done...")
# Injecting meta from another namespace
Response = type('Response', (), {"classVar":var,"__init__": __init__, "server":server_info, "__del__":__del__})  
    
# Let's connect to our home server on my father's desk 
resp = Response("http://192.168.0.19")
print(type(Response))
print(resp.code)
resp.server()
del resp  # It will call the DE-CONSTRUCTOR
# Output
'''
<class 'type'>
401
Date Fri, 25 Sep 2020 06:13:50 GMT
Server Apache/2.4.38 (Raspbian)
WWW-Authenticate Basic realm="Restricted Content"
Content-Length 461Keep-Alive timeout=5, max=100
Connection Keep-Alive
Content-Type text/html; charset=iso-8859-1
[!] closing connection done... # resp object destroyed
'''

__del__(self) Special method __del__ can be termed the terminator or destructor responsible for the termination of an object or class.__del__ is a finalizer that is called during the garbage collection process of an object. This process takes place at some point when all the references to an object have been deleted.

In the above piece of code, we have omitted the low-level things done by the interpreter for the sake of a basic understanding. When we deleted the response object del resp the __del__ special method or the de-constructor is executed to wipe out the object from RAM.

The method is useful in special cases that require extra cleanup upon deletion, e.g. sockets and file objects. However, keep in mind the fact that __del__ is not guaranteed whether it will be executed even if the target object is still alive when the interpreter exits. So, it is a good practice to close any file or connection manually or by using a wrapper.

6. Garbage Collection

A class gives birth to an object. When an object ceases to exist in a namespace, hypothetically a class also needs not to exist in the space. A class can also be deleted like an ordinary object because a class itself is an object. The garbage collection process disposes of unnecessary objects for the sake of memory safety and efficient pieces of code.

Python has adopted two garbage collection mechanisms in CPython*

1. Reference Counting
2. Generational Garbage Collection

Between the two, the first one is used most often because the freedom of destroying an object in userspace might bring a disaster if implemented carelessly, e.g. any strong references to a destroyed object may become orphan/dangling ones and leak the memory.
Reference counting is easy to implement but it cannot handle reference cycles which makes it more prone to a memory leak.

A class gets a type and reference count by default when it is created. Each time when we create an object by calling it, the interpreter increases its reference count. The reference count of an object or class increases when it is

1. Passed as a function argument
2. Assigned to another object or variable
3. Appended to a list
4. Added as a property of a class etc.

When this count reaches zero, the class ceases to exist in the memory which can be treated as the ‘death of a class’. In other words, the reference count of an object increases when it is referenced anywhere and decreases whenever it is dereferenced. The high-level codes we write cannot affect this low-level implementation although it is possible to handle garbage collection by hand using generational garbage collection or the gc module. Observe the following codes

import sys
 
class Bar():
    pass
 
print(sys.getrefcount(Bar)) # default reference count is 5
 
a = Bar()
b = Bar()
c = Bar()
 
print(sys.getrefcount(Bar)) # reference count increased by 3 
 
 
del a , b , c  
 
print(sys.getrefcount(Bar)) # reference count decreased by 3 
 
# Output
'''
5
8
5
 
'''

A reference cycle occurs when one or more Python objects refer to each other. So, in these cases deleting an object merely using del only decreases its reference counts and makes it inaccessible to a programme. It actually remains in the memory.
For example

class Foo():
    pass
 
obj.var = obj

In the above example, we have created a reference cycle by assigning the object obj itself to its property obj.var. Only decreasing obj's reference count will not destroy it from the memory rather it needs generational garbage collection to be cleaned permanently. This implementation can be handled by the gc module in the high-level coding space.
This is another way to 'kill a class'.

Final Thoughts:

From these two parts, we tried to show 6 important facts about the Pythonic class. It is expected that now we have at least some basic ideas about how a Python class works and gets cleaned up from the memory behind the scene.

To cover up the Pythonic Data Model, we will have separate and more detailed articles on Magic Methods, Decorators, Metaprogramming, Pythonic inheritance, etc. A good understanding of the process enables a developer to utilize cent percent of the hidden power of Python and debugging complex issues when things get more Pythonic.

References:

1. https://docs.python.org/3/reference/datamodel.html
2. Fluent Python by Luciano Ramalho. This is the book every Pythonista should read to sharpen their python skills.
3. https://stackify.com/python-garbage-collection/