Metaprogramming with Metaclasses in Python - GeeksforGeeks

11 Oct, 2021
At first, the word Metaprogramming seems like a very funky and alien thing but if you have ever worked with decorators or metaclasses, you were doing metaprogramming there all along. In a nutshell, we can say metaprogramming is the code that manipulates code.In this article, we are going to discuss Metaclasses, why and when we should use them, and what are the alternatives. This is a fairly advance Python topic and the following prerequisite is expected – 
OOP concept in Python
Decorators in Python
Note: This article considers Python 3.3 and above 
Metaclasses
In Python, everything has some type associated with it. For example, if we have a variable having an integer value then its type is int. You can get the type of anything using the type() function. 
Python3
num = 23print("Type of num is:", type(num)) lst = [1, 2, 4]print("Type of lst is:", type(lst)) name = "Atul"print("Type of name is:", type(name))
Output: 
Type of num is: <class 'int'>
Type of lst is: <class 'list'>
Type of name is: <class 'str'>
Every type in Python is defined by Class. So in the above example, unlike C++ or Java where int, char, float are primary data types, in Python they are objects of int class or str class. So we can make a new type by creating a class of that type. For example, we can create a new type of Student by creating a Student class. 
Python3
class Student:    passstu_obj = Student() # Print type of object of Student classprint("Type of stu_obj is:", type(stu_obj))
Output: 
Type of stu_obj is: <class '__main__.Student'>
A Class is also an object, and just like any other object, it’s an instance of something called Metaclass. A special class type creates these Class objects. The type class is default metaclass which is responsible for making classes. In the above example, if we try to find out the type of Student class, it comes out to be a type. 
Python3
class Student:    pass # Print type of Student classprint("Type of Student class is:", type(Student))
Output: 
Type of Student class is: <class 'type'>
Because Classes are also an object, they can be modified in the same way. We can add or subtract fields or methods in class in the same way we did with other objects. For example – 
Python3
# Defined class without any# class methods and variablesclass test:pass # Defining method variablestest.x = 45 # Defining class methodstest.foo = lambda self: print('Hello') # creating objectmyobj = test() print(myobj.x)myobj.foo()
Output: 
45
Hello
This whole meta thing can be summarized as – Metaclass create Classes and Classes creates objects  
The metaclass is responsible for the generation of classes, so we can write our custom metaclasses to modify the way classes are generated by performing extra actions or injecting code. Usually, we do not need custom metaclasses but sometimes it’s necessary. There are problems for which metaclass and non-metaclass-based solutions are available (which are often simpler) but in some cases, only metaclass can solve the problem. We will discuss such a problem in this article.
Creating custom Metaclass
To create our custom metaclass, our custom metaclass has to inherit type metaclass and usually override – 
__new__(): It’s a method which is called before __init__(). It creates the object and returns it. We can override this method to control how the objects are created.
__init__(): This method just initialize the created object passed as a parameter
We can create classes using the type() function directly. It can be called in following ways – 
When called with only one argument, it returns the type. We have seen it before in the above examples.When called with three parameters, it creates a class. Following arguments are passed to it – Class nameTuple having base classes inherited by classClass Dictionary: It serves as a local namespace for the class, populated with class methods and variables
When called with only one argument, it returns the type. We have seen it before in the above examples.
When called with three parameters, it creates a class. Following arguments are passed to it – Class nameTuple having base classes inherited by classClass Dictionary: It serves as a local namespace for the class, populated with class methods and variables
Class nameTuple having base classes inherited by classClass Dictionary: It serves as a local namespace for the class, populated with class methods and variables
Class name
Tuple having base classes inherited by class
Class Dictionary: It serves as a local namespace for the class, populated with class methods and variables
Consider this example –  
Python3
def test_method(self):    print("This is Test class method!") # creating a base classclass Base:    def myfun(self):        print("This is inherited method!") # Creating Test class dynamically using# type() method directlyTest = type('Test', (Base, ), dict(x="atul", my_method=test_method)) # Print type of Testprint("Type of Test class: ", type(Test)) # Creating instance of Test classtest_obj = Test()print("Type of test_obj: ", type(test_obj)) # calling inherited methodtest_obj.myfun() # calling Test class methodtest_obj.my_method() # printing variableprint(test_obj.x)
Output: 
Type of Test class:  <class 'type'>
Type of test_obj:  <class '__main__.Test'>
This is inherited method!
This is Test class method!
atul
Now let’s create a metaclass without using type() directly. In the following example, we will be creating a metaclass MultiBases which will check if the class being created has inherited from more than one base class. If so, it will raise an error. 
Python3
# our metaclassclass MultiBases(type):    # overriding __new__ method    def __new__(cls, clsname, bases, clsdict):        # if no of base classes is greater than 1        # raise error        if len(bases)>1:            raise TypeError("Inherited multiple base classes!!!")                 # else execute __new__ method of super class, ie.        # call __init__ of type class        return super().__new__(cls, clsname, bases, clsdict) # metaclass can be specified by 'metaclass' keyword argument# now MultiBase class is used for creating classes# this will be propagated to all subclasses of Baseclass Base(metaclass=MultiBases):    pass # no error is raisedclass A(Base):    pass # no error is raisedclass B(Base):    pass # This will raise an error!class C(A, B):    pass
Output: 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 8, in __new__
TypeError: Inherited multiple base classes!!!
Solving problems with metaclass
There are some problems which can be solved by decorators (easily) as well as by metaclasses. But there are a few problems whose results can only be achieved by metaclasses. For example, consider a very simple problem of code repetition. We want to debug class methods, what we want is that whenever the class method executes, it should print its fully qualified name before executing its body.
The very first solution that comes to our mind is using method decorators, following is the sample code – 
Python3
from functools import wraps def debug(func):    '''decorator for debugging passed function'''         @wraps(func)    def wrapper(*args, **kwargs):        print("Full name of this method:", func.__qualname__)        return func(*args, **kwargs)    return wrapper def debugmethods(cls):    '''class decorator make use of debug decorator       to debug class methods '''         # check in class dictionary for any callable(method)    # if exist, replace it with debugged version    for key, val in vars(cls).items():        if callable(val):            setattr(cls, key, debug(val))    return cls # sample class@debugmethodsclass Calc:    def add(self, x, y):        return x+y    def mul(self, x, y):        return x*y    def div(self, x, y):        return x/y     mycal = Calc()print(mycal.add(2, 3))print(mycal.mul(5, 2))
Output: 
Full name of this method: Calc.add
5
Full name of this method: Calc.mul
10
This solution works fine but there is one problem, what if we want to apply this method decorator to all subclasses which inherit this Calc class. In that case, we have to separately apply the method decorator to every subclass just like we did with the Calc class.The problem is if we have many such subclasses, then in that case we won’t like adding a decorator to each one separately. If we know beforehand that every subclass must have this debug property, then we should look up to the metaclass-based solution.Have a look at this metaclass based solution, the idea is that classes will be created normally and then immediately wrapped up by debug method decorator – 
Python3
from functools import wraps def debug(func):    '''decorator for debugging passed function'''         @wraps(func)    def wrapper(*args, **kwargs):        print("Full name of this method:", func.__qualname__)        return func(*args, **kwargs)    return wrapper def debugmethods(cls):    '''class decorator make use of debug decorator       to debug class methods '''         for key, val in vars(cls).items():        if callable(val):            setattr(cls, key, debug(val))    return cls class debugMeta(type):    '''meta class which feed created class object       to debugmethod to get debug functionality       enabled objects'''         def __new__(cls, clsname, bases, clsdict):        obj = super().__new__(cls, clsname, bases, clsdict)        obj = debugmethods(obj)        return obj     # base class with metaclass 'debugMeta'# now all the subclass of this# will have debugging appliedclass Base(metaclass=debugMeta):pass # inheriting Baseclass Calc(Base):    def add(self, x, y):        return x+y     # inheriting Calcclass Calc_adv(Calc):    def mul(self, x, y):        return x*y # Now Calc_adv object showing# debugging behaviourmycal = Calc_adv()print(mycal.mul(2, 3))
Output: 
Full name of this method: Calc_adv.mul
6
When to use Metaclasses
Most of the time we do not use metaclasses, it’s usually used for something complicated, but a few cases where we use metaclasses are – 
As we have seen in the above example, metaclasses propagate down the inheritance hierarchies. It will affect all the subclasses as well. If we have such a situation, then we should use metaclasses.
If we want to change class automatically, when it is created, we use metaclasses
For API development, we might use metaclasses
As quoted by Tim Peters 
Metaclasses are deeper magic that 99% of users should never worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why). 
References
http://www.dabeaz.com/py3meta/Py3Meta.pdf
https://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python
This article is contributed by Atul Kumar. If you like GeeksforGeeks and would like to contribute, you can also write an article using write.geeksforgeeks.org or mail your article to review-team@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above. 
edgriebel
Akanksha_Rai
surinderdawra388
clintra
anikaseth98
cannycushalappa
abhishek0719kadiyan
puja84375
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
Read JSON file using Python
Adding new column to existing DataFrame in Pandas
Python map() function
How to get column names in Pandas dataframe
Read a file line by line in Python
Enumerate() in Python
How to Install PIP on Windows ?
Iterate over a list in Python
Different ways to create Pandas Dataframe
Python String | replace()