Context Manager and the "with" Statement in Python
Tags: python, programming, beginners
Published: 2023-12-14 04:48:02

###Introduction

According to the Python official document, the context manager is defined as below. 

>A context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the with statement (described in section The with statement), but can also be used by directly invoking their methods.

>Typical uses of context managers include saving and restoring various kinds of global state, locking and unlocking resources, closing opened files, etc.

Source: [Python Official Documentation](https://docs.python.org/3/reference/datamodel.html?highlight=context%20manager#with-statement-context-managers)

In this blog post, I will confirm how the context manager in Python works with code examples. There are two ways to implement a context manager: class-based and function-based.

###How Class-Based Context Manager Works

To define a context manager, we need to implement the .\_\_enter\_\_ and .\_\_exit\_\_ special methods in our classes.

| Method | Description |
| :--- | :--- |
| .\_\_enter\_\_(self) | Enter the runtime context and return either this object or another object related to the runtime context. The value returned by this method is bound to the identifier in the as clause of with statements using this context manager. |
| .\_\_exit\_\_(self, exc_type, exc_val, exc_tb) | Exit the runtime context and return a Boolean flag indicating if any exception that occurred should be suppressed. If an exception occurred while executing the body of the with statement, the arguments contain the exception type, value and traceback information. Otherwise, all three arguments are None. |

Source: [Python Official Documentation](https://docs.python.org/3/library/stdtypes.html#typecontextmanager)

###Code Example: Class-Based Context Manager

Example 1: Normal end
```python
class ContextManager(object):

    def __init__(self):
        print("call __init__")

    def __enter__(self):
        print("call __enter__")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("call __exit__")
        if exc_type is None:
            print('exit normally')
        else:
            print(f'exc_type={exc_type}')
            return True  # Surpress the exception

    def work(self):
        print("start work")
        # raise Exception() # Exception occurred
        print("end work")


if __name__ == '__main__':
    try:
        with ContextManager() as cm:
            cm.work()
    except Exception as e:
        print("Exception is propagated to the caller")
```

Execution result:

```
call __init__
call __enter__
start work
end work
call __exit__
exit normally
yosuke@Yosuke-Hanaoka python-sandbox % 
```

Example 2: 
An exception occurs in the "with" block and .\_\_exit\_\_() return True.

```python
    def work(self):
        print("start work")
        raise Exception() # Exception occurred
        print("end work")
```

Execution result:
The exception is suppressed in .\_\_exit\_\_() and NOT propagated to the caller.

```
call __init__
call __enter__
start work
call __exit__
exc_type=<class 'Exception'>
yosuke@Yosuke-Hanaoka python-sandbox % 
```

Example 3:
An exception occurs in the "with" block and .\_\_exit\_\_() return False

```python
    def __exit__(self, exc_type, exc_value, traceback):
        print("call __exit__")
        if exc_type is None:
            print('exit normally')
        else:
            print(f'exc_type={exc_type}')
            return False  # Propagate the exception
```

Execution result:
The exception is NOT suppressed in .\_\_exit\_\_() and propagated to the caller.

```
call __init__
call __enter__
start work
call __exit__
exc_type=<class 'Exception'>
Exception is propagated to the caller
yosuke@Yosuke-Hanaoka python-sandbox % 
```

###How Function-Based Context Manager Works

Decorating an appropriately coded generator function with @contextmanager, we can automatically get a function-based context manager that provides .\_\_enter\_\_() and . \_\_exit\_\_().

The processing before and after yield will be .\_\_enter\_\_() and .\_\_exit\_\_(). If there is an object to return by \_\_enter\_\_, we can return with yield. The object returned by yield is bound to the identifier in the as clause of with statements using this context manager.

If any exception occurs in the "with" statement, it can be caught in the except clause of the context manager function, as we can see below.

###Code Example: Function-Based Context Manager

Example 1: Normal end
```python
from contextlib import contextmanager

@contextmanager
def context_manager_func():
    print('__enter__')
    try:
        yield "test"
    except Exception as e:
        print("Exception is caught")
    finally:
        print('__exit__')


if __name__ == '__main__':
    try:
        with context_manager_func() as cm:
            print("start work")
            # raise Exception() # Exception occurred
            print("end work")
    except Exception as e:
        print("Exception is propagated to the caller")
```

Execution result
```
__enter__
start work
end work
__exit__
yosuke@Yosuke-Hanaoka python-sandbox %
```

Example 2:
An exception occurs in the "with" block. 

```python
if __name__ == '__main__':
    try:
        with context_manager_func() as cm:
            print("start work")
            raise Exception("")
            print("end work")
    except Exception as e:
        print("Exception is propagated to caller")
```

Execution result:
The exception is caught in the except clause of the context manager function.

```
__enter__
start work
Exception is caught
__exit__
yosuke@Yosuke-Hanaoka python-sandbox % 
```

Example 3:
An exception occurs in the "with" block, and the context manager function doesn't catch it.

```python
@contextmanager
def context_manager_func():
    print('__enter__')
    try:
        yield "test"
    # except Exception as e:
    #     print("Exception is caught")
    finally:
        # Always done last, whether an exception occurs or not.
        print('__exit__')
```

Execution result:
The exception is NOT suppressed in the context manager function and propagated to the caller.

```
__enter__
start work
__exit__
Exception is propagated to caller
yosuke@Yosuke-Hanaoka python-sandbox % 
```

###Reference:
[Python 3.12.0 documentation: Statement Context Managers](https://docs.python.org/3/reference/datamodel.html#context-managers)
[Python 3.12.0 documentation: Context Manager Types](https://docs.python.org/3/library/stdtypes.html#context-manager-types)
[PEP 343 – The “with” Statement](https://peps.python.org/pep-0343/)
[Context Managers and Python's with Statement](https://realpython.com/python-with-statement/#coding-class-based-context-managers)