A few lesser-known but pretty useful Python concepts
Tags: python, useful, unknown
Published: 2023-06-27 11:34:45

---
title: A few lesser-known but pretty useful Python concepts
published: true
description: 
tags: python,useful,unknown
# cover_image: https://direct_url_to_image.jpg
# Use a ratio of 100:42 for best results.
# published_at: 2023-06-27 11:34 +0000
---

1. [Context Managers](#a-context-managers)
2. [Generators and Generator Expressions](#b-generators-and-generator-expressions)
3. [Function Decorators](#c-function-decorators)
4. [Namedtuples](#d-namedtuples)
5. [Coroutines](#e-coroutines)
6. [Operator Overloading](#f-operator-overloading)
7. [Monkey Patching](#g-monkey-patching)
8. [Type Hints and Annotations](#h-type-hints-and-annotations)
9. [Metaclasses](#i-metaclasses)
10. [Data Classes](#j-data-classes)
11. [Context Variables](#k-context-variables)
12. [Callable Objects](#l-callable-objects)
13. [Extended Iterable Unpacking](#m-extended-iterable-unpacking)
14. [Multiple Inheritance and Method Resolution Order (MRO)](#n-multiple-inheritance-and-method-resolution-order-mro)

---

### A. Context Managers: 

Context managers allow you to allocate and release resources when needed, such as opening and closing files, acquiring and releasing locks, etc. They can be implemented using the `with` statement and the `contextlib` module.

#### Examples

- File Handling
```python
with open('file.txt', 'r') as file:
    data = file.read()
    # Perform operations on the file
# File automatically closed outside the 'with' block
```

- Lock Acquisition and Release
```python
import threading

lock = threading.Lock()
with lock:
    # Perform thread-safe operations
# Lock automatically released outside the 'with' block
```

- Database Connection
```python
import sqlite3

with sqlite3.connect('database.db') as connection:
    cursor = connection.cursor()
    # Perform database operations
# Connection automatically closed outside the 'with' block
```

- Timing Execution
```python
import time

class Timer:
    def __enter__(self):
        self.start_time = time.time()
        return self
   
    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed_time = time.time() - self.start_time
        print(f"Execution time: {elapsed_time} seconds")

with Timer():
    # Code to measure execution time
# Timer context manager prints the execution time
```

#### Creating a custom context manager
```python
class CustomContextManager:
    def __enter__(self):
        # Code to run when entering the 'with' block
        print("Entering the 'with' block")
        # You can optionally return an object or set up resources
        
    def __exit__(self, exc_type, exc_val, exc_tb):
        # Code to run when exiting the 'with' block
        print("Exiting the 'with' block")
        # Perform any cleanup actions or handle exceptions if necessary

# Usage example:
with CustomContextManager():
    # Code inside the 'with' block
    print("Inside the 'with' block")

# Output:
# Entering the 'with' block
# Inside the 'with' block
# Exiting the 'with' block
```


### B. Generators and Generator Expressions: 

Generators are functions that generate values on the fly, allowing you to iterate over a sequence of values without creating the entire sequence in memory. Generator expressions are similar to list comprehensions but return a generator object instead of a list.

- Generator Function
```python
def countdown(n):
    while n > 0:
        yield n
        n -= 1

# Using the generator function
for num in countdown(5):
    print(num)

# Output:
# 5
# 4
# 3
# 2
# 1
```

- Generator Expression
```python
# Using a generator expression to calculate the squares of numbers
squares = (x ** 2 for x in range(1, 6))

# Accessing the values from the generator expression
for square in squares:
    print(square)

# Output:
# 1
# 4
# 9
# 16
# 25
```

- Infinite Generator
```python
def infinite_sequence():
    num = 0
    while True:
        yield num
        num += 1

# Using the infinite generator
for i in infinite_sequence():
    if i > 10:
        break
    print(i)

# Output:
# 0
# 1
# 2
# 3
# 4
# 5
# 6
# 7
# 8
# 9
# 10
```


### C. Function Decorators: 

Decorators are a way to modify the behavior of functions or classes by wrapping them with another function. They are denoted by the `@` symbol and can be used for tasks like logging, timing, and caching.

- Logging Decorator: This decorator logs the name of the decorated function before and after its execution
```python
def log_decorator(func):
    def wrapper(*args, **kwargs):
        print(f'Calling {func.__name__}...')
        result = func(*args, **kwargs)
        print(f'{func.__name__} called.')
        return result
    return wrapper

@log_decorator
def add_numbers(a, b):
    return a + b

result = add_numbers(3, 5)
print(result)

# Output:
# Calling add_numbers...
# add_numbers called.
# 8
```

- Timing Decorator: This decorator measures the execution time of the decorated function
```python
import time

def time_decorator(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        execution_time = end_time - start_time
        print(f'{func.__name__} executed in {execution_time} seconds.')
        return result
    return wrapper

@time_decorator
def factorial(n):
    if n == 0 or n == 1:
        return 1
    else:
        return n * factorial(n - 1)

result = factorial(5)
print(result)  # Output: 120

# Output:
# factorial executed in 9.5367431640625e-07 seconds.
# factorial executed in 7.605552673339844e-05 seconds.
# factorial executed in 9.322166442871094e-05 seconds.
# factorial executed in 0.00010275840759277344 seconds.
# factorial executed in 0.00011801719665527344 seconds.
# 120
```

- Authorization Decorator: This decorator checks if the user is authorized to access the decorated function
```python
def is_user_authorized():
	return False

def authorize_decorator(func):
    def wrapper(*args, **kwargs):
        if is_user_authorized():
            return func(*args, **kwargs)
        else:
            raise PermissionError('User is not authorized to access this function.')
    return wrapper

@authorize_decorator
def sensitive_operation():
    return 'Sensitive data'

result = sensitive_operation()
print(result)

# Output:
# Traceback (most recent call last):
#   File "/Users/dev1/python/app.py", line 16, in <module>
#     result = sensitive_operation()
#   File "/Users/dev1/mygitlab/python/app.py", line 9, in wrapper
#     raise PermissionError('User is not authorized to access this function.')
# PermissionError: User is not authorized to access this function.
```


### D. Namedtuples: 

Namedtuples are lightweight data structures that are similar to tuples but have named fields. They provide a convenient way to define simple classes without writing a custom class definition. They are commonly used in scenarios where lightweight data containers are required.

- Creating a Namedtuple
```python
from collections import namedtuple

# Define a namedtuple called 'Point' with fields 'x' and 'y'
Point = namedtuple('Point', ['x', 'y'])

# Create an instance of Point
p = Point(2, 3)

# Access the fields using dot notation
print(p.x)  # Output: 2
print(p.y)  # Output: 3
```

- Namedtuple as a Return Type
```python
from collections import namedtuple

# Define a namedtuple called 'Person' with fields 'name', 'age', and 'city'
Person = namedtuple('Person', ['name', 'age', 'city'])

# Function that returns a Person namedtuple
def get_person():
    return Person('John', 25, 'New York')

# Call the function and access the fields of the returned namedtuple
person = get_person()
print(person.name)  # Output: John
print(person.age)   # Output: 25
print(person.city)  # Output: New York
```

- Unpacking a Namedtuple
```python
from collections import namedtuple

# Define a namedtuple called 'Color' with fields 'red', 'green', and 'blue'
Color = namedtuple('Color', ['red', 'green', 'blue'])

# Create an instance of Color
color = Color(255, 128, 0)

# Unpack the values into separate variables
red, green, blue = color

print(red)    # Output: 255
print(green)  # Output: 128
print(blue)   # Output: 0
```


### E. Coroutines: 

Coroutines are functions that can pause their execution and yield control back to the caller while maintaining their state. They are used for asynchronous programming and are a powerful tool for managing concurrency and asynchronous programming, allowing for more efficient and flexible code execution.

- Simple Coroutine
```python
def coroutine_example():
    while True:
        x = yield
        print('Received:', x)

coroutine = coroutine_example()
next(coroutine)  # Initialize the coroutine
coroutine.send(10)  # Send a value to the coroutine
coroutine.send('Hello')  # Send another value to the coroutine
coroutine.close()   # close it

# Output:
# Received: 10
# Received: Hello
```

- Coroutine with Producer and Consumer
```python
def producer(coroutine):
    for i in range(5):
        print('Producing:', i)
        coroutine.send(i)
    coroutine.close()

def consumer():
    while True:
        x = yield
        print('Consumed:', x)

coroutine = consumer()
next(coroutine)  # Initialize the coroutine
producer(coroutine)

# Output:
# Producing: 0
# Consumed: 0
# Producing: 1
# Consumed: 1
# Producing: 2
# Consumed: 2
# Producing: 3
# Consumed: 3
# Producing: 4
# Consumed: 4
```

- Coroutine Chaining
```python
def coroutine1():
    while True:
        x = yield
        print('Coroutine 1:', x)

def coroutine2():
    while True:
        x = yield
        print('Coroutine 2:', x)

coroutine = coroutine1()
next(coroutine)  # Initialize the first coroutine
coroutine2()  # Initialize the second coroutine
coroutine.send(10)  # Send a value to the first coroutine
coroutine.send('Hello')  # Send another value to the first coroutine

# Output:
# Coroutine 1: 10
# Coroutine 1: Hello
```


### F. Operator Overloading: 

Python allows you to redefine the behavior of operators for your custom classes by implementing special methods such as `__add__`, `__sub__`, `__mul__`, etc. This concept is known as operator overloading. You can overload various operators such as arithmetic operators, comparison operators, and more. Operator overloading allows you to customize the behavior of objects to make your code more expressive and intuitive

- Arithmetic Operators
```python
class Vector:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __add__(self, other):
        return Vector(self.x + other.x, self.y + other.y)

    def __sub__(self, other):
        return Vector(self.x - other.x, self.y - other.y)

    def __mul__(self, scalar):
        return Vector(self.x * scalar, self.y * scalar)

    def __str__(self):
        return f"x: {self.x}, y: {self.y}"

# Usage:
v1 = Vector(2, 3)
v2 = Vector(4, 5)

v3 = v1 + v2  # Addition
v4 = v1 - v2  # Subtraction
v5 = v1 * 2   # Scalar multiplication

print(v3)
print(v4)
print(v5)

# Output:
# x: 6, y: 8
# x: -2, y: -2
# x: 4, y: 6
```

- Comparison Operators
```python
class Point:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return self.x == other.x and self.y == other.y

    def __lt__(self, other):
        return self.x < other.x and self.y < other.y

# Usage:
p1 = Point(2, 3)
p2 = Point(4, 5)

print(p1 == p2)  # Equality
print(p1 < p2)   # Less than

# Output:
# False
# True
```

- String Representation
```python
class Book:
    def __init__(self, title, author):
        self.title = title
        self.author = author

    def __str__(self):
        return f"{self.title} by {self.author}"

# Usage:
book = Book("Python Programming", "John Smith")
print(book)  # String representation

# Output:
# Python Programming by John Smith
```


### G. Monkey Patching: 

Monkey patching refers to the ability to modify or extend the behavior of existing code at runtime. In Python, you can dynamically modify classes, objects, or modules by adding, replacing, or deleting attributes or methods.

- Adding a Method to a Class
```python
class MyClass:
    def __init__(self, name):
        self.name = name

def say_hello(self):
    print(f"Hello, {self.name}!")

# Monkey patching
MyClass.say_hello = say_hello

# Creating an instance and calling the patched method
obj = MyClass("Alice")
obj.say_hello()

# Output:
# Hello, Alice!
```

- Modifying an Existing Method
```python
class MyClass:
    def greeting(self):
        return "Hello!"

# Monkey patching
def modified_greeting(self):
    return "Hola!"

MyClass.greeting = modified_greeting

# Creating an instance and calling the modified method
obj = MyClass()
print(obj.greeting())

# Output:
# Hola!
```

- Adding a Function to a Module
```python
def multiply(a, b):
    return a * b

# Monkey patching
import math
math.multiply = multiply

# Calling the patched function
result = math.multiply(5, 6)
print(result)

# Output:
# 30
```


### H. Type Hints and Annotations: 

Type hints are a way to statically declare the expected types of variables, arguments, and return values in Python code. They are not enforced at runtime but can be used by static analysis tools to catch potential type-related errors.

- Variable type hints:
```python
def add(a: int, b: int) -> int:
    return a + b
```

- Class annotations:
```python
class Person:
    def __init__(self, name: str, age: int):
        self.name = name
        self.age = age
```

- Type hints for collections
```python
from typing import List, Tuple

def process(data: List[Tuple[str, str]]) -> List[str]:
    result: List[str] = []

    for item in data:
        result.append(item[0] * item[1])

    return result
```

### I. Metaclasses

Metaclasses in Python provide a way to define the behavior and structure of classes themselves. They allow you to customize the creation and initialization of classes.

Metaclasses provide a powerful mechanism for customizing class creation and behavior, but they should be used sparingly and only when necessary, as they can make the code more complex and harder to understand.

- Creating a Singleton Metaclass
```python
class SingletonMeta(type):
    _instances = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super().__call__(*args, **kwargs)
        return cls._instances[cls]


class SingletonClass(metaclass=SingletonMeta):
    def __init__(self):
        print("Initializing SingletonClass")


# Usage
instance1 = SingletonClass()
instance2 = SingletonClass()
print(instance1 is instance2)

# Output:
# Initializing SingletonClass
# True
```

- Creating an Attribute Validation Metaclass
```python
class ValidationMeta(type):
    def __new__(cls, name, bases, attrs):
        for name, value in attrs.items():
            if name.startswith("_"):
                continue

            if not isinstance(value, (int, float)):
                raise TypeError(f"Attribute '{name}' must be numeric.")

        return super().__new__(cls, name, bases, attrs)


class MyClass(metaclass=ValidationMeta):
    x = 10
    y = "test"

# Output:
# Traceback (most recent call last):
#   File "/Users/dev1/python/learnings/app.py", line 13, in <module>
#     class MyClass(metaclass=ValidationMeta):
#   File "/Users/dev1/python/learnings/app.py", line 8, in __new__
#     raise TypeError(f"Attribute '{name}' must be numeric.")
# TypeError: Attribute 'y' must be numeric
```

- Registering Subclasses with a Metaclass
```python
class PluginMeta(type):
    def __init__(cls, name, bases, attrs):
        if not hasattr(cls, "plugins"):
            cls.plugins = []
        else:
            cls.plugins.append(cls)


class PluginBase(metaclass=PluginMeta):
    pass


class Plugin1(PluginBase):
    pass


class Plugin2(PluginBase):
    pass


# Usage
print(PluginBase.plugins)

# Output
# [<class '__main__.Plugin1'>, <class '__main__.Plugin2'>]
```

### J. Data Classes

Python's data classes are a convenient way to define classes that primarily hold data, similar to structs in other programming languages. They provide several benefits, such as automatic generation of common methods like `__init__`, `__repr__`, and `__eq__`. Data classes provide additional features like type hints, default values, ordering, and more. 

- Basic Data Class
```python
from dataclasses import dataclass

@dataclass
class Point:
    x: float
    y: float
    z: float

p = Point(1.0, 2.0, 3.0)
q = Point(1.0, 2.0, 3.0)

print(p.x)
print(p.y)
print(p.z)

print(p)
print(p == q)

# Output
# 1.0
# 2.0
# 3.0
# Point(x=1.0, y=2.0, z=3.0)
# True
```

- Data Class with Default Values
```python
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int = 0
    profession: str = "Unknown"

# Create an instance of the Person class
person1 = Person("Alice", 25)
person2 = Person("Bob", profession="Engineer")

# Access the fields of the instances
print(person1.name, person1.age, person1.profession)
print(person2.name, person2.age, person2.profession)

# Output:
# Alice 25 Unknown
# Bob 0 Engineer
```

- Inheritance with Data Classes
```python
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int

@dataclass
class Employee(Person):
    company: str

# Create an instance of the Employee class
employee = Employee("Alice", 30, "Acme Corporation")

# Access the fields of the instance
print(employee.name)
print(employee.age)
print(employee.company)

# Output:
# Alice
# 30
# Acme Corporation
```

### K. Context Variables

Introduced in Python 3.7, context variables allow you to create variables that retain their values within a context, even if the context is asynchronous or multi-threaded. They are useful for propagating values across functions without passing them explicitly as arguments.

- Using `contextvars` module
```python
import contextvars

user_id = contextvars.ContextVar("user_id", default=None)

def process_request(request):
    user_id.set(request.user_id)
    process_data()


def process_data():
    uid = user_id.get()
    print(f"processing for user: {uid}")
```

- Using `threading.local` for thread-local context
```python
import threading

context = threading.local()

def process_request(request):
    context.user_id = request.user_id
    process_data()

def process_data():
    uid = context.user_id
    print(f"processing for user: {uid}")
```

- Using `contextlib.ContextDecorator` for context-based behavior
```python
from contextlib import ContextDecorator
import time

class TimingContext(ContextDecorator):
    def __enter__(self):
        self.start_time = time.time()

    def __exit__(self, exc_type, exc_val, exc_tb):
        elapsed_time = time.time() - self.start_time
        print("Elapsed time:", elapsed_time)

@TimingContext()
def process_data():
    # Perform some time-consuming operation
    time.sleep(3)
    print("Data processing complete.")

process_data()

# Output:
# Data processing complete.
# Elapsed time: 3.0054383277893066
```

### L. Callable Objects

In Python, any object that can be called as a function is considered callable. This includes functions, methods, classes (which create instances when called), and objects implementing the `__call__` method. You can use callable objects to create more flexible and dynamic code.

- Classes with `__call__` method: Classes that define the `__call__` method can be called like functions
```python
class Multiply:
    def __call__(self, a, b):
        return a * b

multiply = Multiply()
result = multiply(2, 3)
print(result)

# Output:
# 6
```

-     Built-in callable objects: Some built-in objects in Python are callable, such as `int`, `str`, `list`, `dict`, etc.
```python
result = str(42)
print(result)
# Output
# 42
```

### M. Extended Iterable Unpacking

Extended iterable unpacking, introduced in Python 3, allows you to unpack an iterable into multiple variables, including capturing remaining items into another variable. It simplifies tasks such as splitting lists, assigning values from tuples, and processing variable-length iterables.

- Unpacking a list into variables
```python
my_list = [1, 2, 3, 4, 5]
first, *middle, last = my_list

print(first)
print(middle)
print(last)

# Output:
# 1
# [2, 3, 4]
# 5
```

- Unpacking a string into variables
```python
my_string = "Hello"
first, *rest = my_string

print(first)
print(rest)

# Output:
# H
# ['e', 'l', 'l', 'o']
```

- Unpacking a tuple of unknown length
```python
my_tuple = (1, 2, 3, 4, 5)
first, *middle, last = my_tuple

print(first)
print(middle)
print(last)

# Output:
# 1
# [2, 3, 4]
# 5
```

### N. Multiple Inheritance and Method Resolution Order (MRO)

Python supports multiple inheritance, allowing a class to inherit from multiple base classes. Method Resolution Order (MRO) determines the order in which base classes are searched for a method. Understanding MRO helps you resolve method name conflicts and grasp the inheritance hierarchy.

```python
class A:
    def say_hello(self):
        print("Hello from A")

class B(A):
    def say_hello(self):
        print("Hello from B")

class C(A):
    def say_hello(self):
        print("Hello from C")

class D(B, C):
    pass

d = D()
d.say_hello()

# Output:
# Hello from B
```

When we create an instance of `D` and call the `say_hello()` method, Python follows the Method Resolution Order (MRO) to determine which implementation of the method should be executed.

The MRO is determined by the C3 linearization algorithm, which is a consistent and predictable method resolution order. In this case, the MRO for class `D` would be `[D, B, C, A, object]`. Thus, `say_hello()` of class `B` gets invoked.