# The Linux Kernel/System

* system
* User space interfaces, System calls
* Driver Model
* modules
* buses, PCI
* hardware interfaces, [re]booting
This article describes infrastructures used to support and manage other kernel functionalities. This functionality is named after system calls and sysfs.

## User space communication

User space communication refers to the exchange of data and messages between user space applications and the kernel. User space applications are programs that run in the user space of the operating system, which is a protected area of memory that provides a safe and isolated environment for applications to run in.
There are several mechanisms available in Linux for user space communication with the kernel. One of the most common mechanisms is through system calls, which are functions that allow user space applications to request services from the kernel, such as opening files, creating processes, and accessing system resources.
Another mechanism for user space communication is through service files, which are special files that represent physical or virtual devices, such as storage devices, network interfaces, and various peripheral devices. User space applications can communicate with these devices by reading from and writing to their corresponding device files.
In summary, Linux kernel provides several mechanisms for user space communication, including system calls, device files, procfs, sysfs, and devtmpfs. These mechanisms enable user space applications to communicate with the kernel and access system resources in a safe and controlled manner.
⚲ APIs:
    kernel space API for user space
       uapi$_{inc}$
       arch/x86/include/uapi$_{src}$
       man 2 ioctl
       System calls
       Device files
    user space API for kernel space
       linux/uaccess.h$_{inc}$:
       copy_to_user$_{id}$
       copy_from_user$_{id}$
📖 References
    User-space API guides$_{doc}$
    User space
    Linux kernel interfaces
    ULK3 Chapter 11. Signals

### System calls

System calls are the fundamental interface between user space applications and the Linux kernel. They provide a way for programs to request services from the operating system, such as opening a file, allocating memory, or creating a new process. In the Linux kernel, system calls are implemented as functions that can be invoked by user space programs using a software interrupt mechanism.
The Linux kernel provides hundreds of system calls, each with its own unique functionality. These system calls are organized into categories such as process management, file management, network communication, and memory management. User space applications can use these system calls to interact with the kernel and access the underlying system resources.
⚲ API
    Table of syscalls
    man 2 syscalls
⚙️ Internals
    linux/syscalls.h$_{inc}$
    syscall_init$_{id}$ installs entry_SYSCALL_64$_{id}$
    man 2 syscall ↪
       entry_SYSCALL_64$_{id}$ ↯ call hierarchy:
          do_syscall_64$_{id}$
             sys_call_table$_{id}$
📖 References
    System call
    Directory of system calls, man section 2
    Anatomy of a system call, part 1 and part 2
    syscalls$_{ltp}$
💾 Historical
    ULK3 Chapter 10. System Calls

### Device files

Classic UNIX devices are Char devices used as byte streams with man 2 ioctl.
⚲ API
Examples: misc_fops$_{id}$ usb_fops$_{id}$ memory_fops$_{id}$
    Allocated devices$_{doc}$
    drivers/char$_{src}$ - actually byte stream devices
    Chapter 13. I/O Architecture and Device Drivers

#### hiddev

⚠️ Warning: confusion. hiddev isn't real human interface device! It reuses USBHID infrastructure. hiddev is used for example for monitor controls and Uninterruptible Power Supplies. This module supports these devices separately using a separate event interface on /dev/usb/hiddevX (char 180:96 to 180:111) (⚙️ HIDDEV_MINOR_BASE$_{id}$)
⚲ API
    uapi/linux/hiddev.h$_{inc}$
    HID_CONNECT_HIDDEV$_{id}$
⚙️ Internals
    CONFIG_USB_HIDDEV
    linux/hiddev.h$_{inc}$
    hiddev_event$_{id}$
    drivers/hid/usbhid/hiddev.c$_{src}$, hiddev_fops$_{id}$
📖 References
    HIDDEV - Care and feeding of your Human Interface Devices$_{doc}$
📖 References
    Device file

### Administration

🔧 TODO
📖 References
    man 7 netlink
    The Linux kernel user’s and administrator’s guide$_{doc}$

#### procfs

The proc filesystem (procfs) is a special filesystem that presents information about processes and other system information in a hierarchical file-like structure, providing a more convenient and standardized method for dynamically accessing process data held in the kernel than traditional tracing methods or direct access to kernel memory. Typically, it is mapped to a mount point named /proc at boot time. The proc file system acts as an interface to internal data structures in the kernel. It can be used to obtain information about the system and to change certain kernel parameters at runtime.
/proc includes a directory for each running process —including kernel threads— in directories named /proc/PID, where PID is the process number. Each directory contains information about one process, including the command that originally started the process (/proc/PID/cmdline), the names and values of its environment variables (/proc/PID/environ), a symlink to its working directory (/proc/PID/cwd), another symlink to the original executable file —if it still exists— (/proc/PID/exe), a couple of directories with symlinks to each open file descriptor (/proc/PID/fd) and the status —position, flags, ...— of each of them (/proc/PID/fdinfo), information about mapped files and blocks like heap and stack (/proc/PID/maps), a binary image representing the process's virtual memory (/proc/PID/mem), a symlink to the root path as seen by the process (/proc/PID/root), a directory containing hard links to any child process or thread (/proc/PID/task), basic information about a process including its run state and memory usage (/proc/PID/status) and much more.
📖 References
    procfs
    man 5 procfs
    man 7 namespaces
    man 7 capabilities
    linux/proc_fs.h$_{inc}$
    fs/proc$_{src}$

#### sysfs

sysfs is a pseudo-file system that exports information about various kernel subsystems, hardware devices, and associated device drivers from the kernel's device model to user space through virtual files. In addition to providing information about various devices and kernel subsystems, exported virtual files are also used for their configuring. Sysfs is designed to export the information present in the device tree, which would then no longer clutter up procfs.
Sysfs is mounted under the /sys mount point.
⚲ API
    linux/sysfs.h$_{inc}$
📖 References
    sysfs
    man 5 sysfs
    sysfs - filesystem for exporting kernel objects$_{doc}$
    fs/sysfs$_{src}$

#### devtmpfs

devtmpfs is a hybrid kernel/userspace approach of a device filesystem to provide nodes before udev runs for the first time.
📖 References
    Device file
    drivers/base/devtmpfs.c$_{src}$

## Containerization

Containerization is a powerful technology that has revolutionized the way software applications are developed, deployed, and run. At its core, containerization provides an isolated environment for running applications, where the application has all the necessary dependencies and can be easily moved from one environment to another without worrying about any compatibility issues.
Containerization technology has its roots in the chroot command, which was introduced in the Unix operating system in the 1979. Chroot provided a way to change the root directory of a process, effectively creating a new isolated environment with its own file system hierarchy. However, this early implementation of containerization had limited functionality, and it was difficult to manage and control the various processes running within the container.
In the early 2000s, the Linux kernel introduced namespaces and control groups to provide a more robust and scalable containerization solution. Namespaces allow processes to have their own isolated view of the system, including the file system, network, and process ID space, while control groups provide fine-grained control over the resources allocated to each container, such as CPU, memory, and I/O.
Using these kernel features, containerization platforms such as Docker and Kubernetes have emerged as popular solutions for building and deploying containerized applications at scale. Containerization has become an essential tool for modern software development, allowing developers to easily package applications and deploy them in a consistent and predictable manner across different environments.

### Resources usage and limits

⚲ API
    man 2 chroot – change root directory
    man 2 sysinfo – return system information
    man 2 getrusage – get resource usage
    get/set resource limits:
       man 2 getrlimit
       man 2 setrlimit
       man 2 prlimit64
📖 References
    chroot

### Namespaces

Linux namespaces provide the way to to isolate and virtualize different aspects of the operating system. Namespaces allow multiple instances of an application to run in isolation from each other, without interfering with the host system or other instances.
🔧 TODO
⚲ API
    /proc/self/ns
    man 8 lsns, man 2 ioctl_ns ↪ ns_ioctl$_{id}$
    man 1 unshare, man 2 unshare
    man 1 nsenter, man 2 setns
    man 2 clone3 ↪ clone_args$_{id}$
    linux/ns_common.h$_{inc}$
    linux/proc_ns.h$_{inc}$
    namespaces definition
       uts_namespace$_{id}$
       ipc_namespace$_{id}$
       mnt_namespace$_{id}$
       pid_namespace$_{id}$
       net/net_namespace.h$_{inc}$ - struct net
       user_namespace$_{id}$
       time_namespace$_{id}$
       cgroup_namespace$_{id}$
⚙️ Internals
    init_nsproxy$_{id}$ - struct of namespaces
    kernel/nsproxy.c$_{src}$
    fs/namespace.c$_{src}$
    fs/proc/namespaces.c$_{src}$
    net/core/net_namespace.c$_{src}$
    kernel/time/namespace.c$_{src}$
    kernel/user_namespace.c$_{src}$
    kernel/pid_namespace.c$_{src}$
    kernel/utsname.c$_{src}$
    kernel/cgroup/namespace.c$_{src}$
    ipc/namespace.c$_{src}$
📖 References
    man 7 namespaces
    man 7 uts_namespaces
    man 7 ipc_namespaces
    man 7 mount_namespace
    man 7 pid_namespaces
    man 7 network_namespaces
    man 7 user_namespaces
    man 7 time_namespaces
    man 7 cgroup_namespaces

### Control groups

cgroups are used to limit and control the resource usage of groups of processes. They allow administrators to set limits on CPU usage, memory usage, disk I/O, network bandwidth, and other resources, which can be useful for managing system performance and preventing resource contention.
There are two versions of cgroups. Unlike v1, cgroup v2 has only a single process hierarchy and discriminates between processes, not threads.
Here are some of the key differences between cgroups v1 and v2:
| | cgroups v1 | cgroups v2 |
| Hierarchy | each subsystem had its own hierarchy, which could lead to complexity and confusion | unified hierarchy, which simplifies management and enables better resource allocation |
| Controllers | has several subsystems that are controlled by separate controllers, each with its own set of configuration files and parameters | controllers are consolidated into a single "cgroup2" controller, which provides a unified interface for managing resources |
| Resource distribution | distributes resources among groups of processes based on proportional sharing, which can lead to unpredictable results | resources are distributed based on a "weighted fair queuing" algorithm, which provides better predictability and fairness |
Cgroups v2 is not backward compatible with cgroups v1, which means that migrating from v1 to v2 can be challenging and requires careful planning.
🔧 TODO
⚲ API
    linux/cgroup.h$_{inc}$
    linux/cgroup-defs.h$_{inc}$
       css_set$_{id}$ – holds set of reference-counted pointers to cgroup_subsys_state$_{id}$ objects
       cgroup_subsys$_{id}$
    linux/cgroup_subsys.h$_{inc}$ – list of cgroup subsystems
⚙️ Internals
    cg_list$_{id}$ – list of css_set$_{id}$ in task_struct
    kernel/cgroup$_{src}$
    cgroup_init$_{id}$
    cgroup2_fs_type$_{id}$
    tools/testing/selftests/cgroup$_{src}$
📖 References
    Control Groups v1$_{doc}$
    man 1 systemd-cgtop
    man 5 systemd.slice – slice unit configuration
    man 7 cgroups
    man 7 cgroup_namespaces
    CFS Bandwidth Control for cgroups$_{doc}$
    Real-Time group scheduling$_{doc}$
📚 Further reading
    https://github.com/containers
    cgrc tool
💾 Historical
    https://github.com/mk-fg/cgroup-tools for cgrpup v1

## Driver Model

The Linux driver model (or Device Model, or just DM) is a framework that provides a consistent and standardized way for device drivers to interface with the kernel. It defines a set of rules, interfaces, and data structures that enable device drivers to communicate with the kernel and perform various operations, such as managing resources, livecycle and more.
DM core structure consists of DM classes, DM buses, DM drivers and DM devices.

### kobject

In the Linux kernel, a kobject$_{id}$ is a fundamental data structure used to represent kernel objects and provide a standardized interface for interacting with them. A kobject is a generic object that can represent any type of kernel object, including devices, files, modules, and more.
The kobject data structure contains several fields that describe the object, such as its name, type, parent, and operations. Each kobject has a unique name within its parent object, and the parent-child relationships form a hierarchy of kobjects.
Kobjects are managed by the kernel's sysfs file system, which provides a virtual file system that exposes kernel objects as files and directories in the user space. Each kobject is associated with a sysfs directory, which contains files and attributes that can be read or written to interact with the kernel object.
⚲ Infrastructure API
    linux/kobject.h$_{inc}$
    kobject$_{id}$
    Kernel objects manipulation$_{doc}$
    🔧 TODO

### Classes

A class is a higher-level view of a device that abstracts out low-level implementation details. Drivers may see a NVME storage or a SATA storage, but, at the class level, they are all simply block_class$_{id}$ devices. Classes allow user space to work with devices based on what they do, rather than how they are connected or how they work. General DM classes structure match composite pattern.
⚲ API
    ls /sys/class/
    class_register$_{id}$ registers class$_{id}$
    linux/device/class.h$_{inc}$
👁 Examples: input_class$_{id}$, block_class$_{id}$ net_class$_{id}$

### Buses

A peripheral bus is a channel between the processor and one or more peripheral devices. A DM bus is proxy for a peripheral bus.
General DM buses structure match composite pattern. For the purposes of the device model, all devices are connected via a bus, even if it is an internal, virtual, platform_bus_type$_{id}$. Buses can plug into each other. A USB controller is usually a PCI device, for example. The device model represents the actual connections between buses and the devices they control. A bus is represented by the bus_type$_{id}$ structure. It contains the name, the default attributes, the bus' methods, PM operations, and the driver core's private data.
⚲ API
    ls /sys/bus/
    bus_register$_{id}$ registers bus_type$_{id}$
    linux/device/bus.h$_{inc}$
👁 Examples: usb_bus_type$_{id}$, hid_bus_type$_{id}$, pci_bus_type$_{id}$, scsi_bus_type$_{id}$, platform_bus_type$_{id}$
    Peripheral buses

### Drivers

⚲ API
    ls /sys/bus/:/drivers/
    module_driver$_{id}$ - simple common driver initializer, 👁 for example used in module_pci_driver$_{id}$
    driver_register$_{id}$ registers device_driver$_{id}$ - basic device driver structure, one per all device instances.
    linux/device/driver.h$_{inc}$
👁 Examples: hid_generic$_{id}$ usb_register_device_driver$_{id}$
Platform drivers
    module_platform_driver$_{id}$ registers platform_driver$_{id}$ (platform wrapper of device_driver$_{id}$) with platform_bus_type$_{id}$
    linux/platform_device.h$_{inc}$
👁 Examples: gpio_mouse_device_driver$_{id}$

### Devices

⚲ API
    ls /sys/devices/
    device_register$_{id}$ registers device$_{id}$ - the basic device structure, per each device instance
    linux/device.h$_{inc}$ – Device drivers infrastructure$_{doc}$
    linux/dev_printk.h$_{inc}$
    Device Resource Management$_{doc}$, devres, devm ...
👁 Examples: platform_bus$_{id}$ mousedev_create
Platform devices
    platform_device$_{id}$ - platform wrapper of struct device - the basic device structure$_{doc}$, contains resources associated with the devie
    it is can be created dynamically automatically by platform_device_register_simple$_{id}$ or platform_device_alloc$_{id}$. Or registered with platform_device_register$_{id}$.
    platform_device_unregister$_{id}$ - releases device and associated resources
👁 Examples: add_pcspkr$_{id}$
⚲ API 🔧 TODO
    platform_device_info platform_device_id platform_device_register_full platform_device_add
    platform_device_add_data platform_device_register_data platform_device_add_resources
    attribute_group dev_pm_ops
⚙️ Internals
    linux/dev_printk.h$_{inc}$
    lib/kobject.c$_{src}$
    drivers/base/platform.c$_{src}$
    drivers/base/core.c$_{src}$
📖 References
    Device drivers infrastructure$_{doc}$
    Everything you never wanted to know about kobjects, ksets, and ktypes$_{doc}$
    Driver Model$_{doc}$
       The Linux Kernel Device Model$_{doc}$
       Platform Devices and Drivers$_{doc}$
    Linux Device Model, by linux-kernel-labs

## Modules

    Article about modules
⚲ API
    lsmod
    cat /proc/modules
⚙️ Internals
    kernel/kmod.c$_{src}$
📖 References
    LDD3: Building and Running Modules
    http://www.xml.com/ldd/chapter/book/ch02.html
    http://www.tldp.org/LDP/tlk/modules/modules.html
    http://www.tldp.org/LDP/lkmpg/2.6/html/ The Linux Kernel Module Programming Guide

## Peripheral buses

Peripheral buses are the communication channels used to connect various peripheral devices to a computer system. These buses are used to transfer data between the peripheral devices and the system's processor or memory. In the Linux kernel, peripheral buses are implemented as drivers that enable communication between the operating system and the hardware.
Peripheral buses in the Linux kernel include USB, PCI, SPI, I2C, and more. Each of these buses has its own unique characteristics, and the Linux kernel provides support for a wide range of peripheral devices.
The PCI (Peripheral Component Interconnect) bus is used to connect internal hardware devices in a computer system. It is commonly used to connect graphics cards, network cards, and other expansion cards. The Linux kernel provides a PCI bus driver that enables communication between the operating system and the devices connected to the bus.
The USB (Universal Serial Bus) is one of the most commonly used peripheral buses in modern computer systems. It allows devices to be hot-swapped and supports high-speed data transfer rates.
🔧 TODO: device enumeration
⚲ API
    Shell interface: ls /proc/bus/ /sys/bus/
See also Buses of Driver Model
See Input: keyboard, mouse etc
PCI
⚲ Shell API
    lspci -vv
    column -t /proc/bus/pci/devices
Main article: PCI
USB
⚲ Shell API
    lsusb -v
    ls /sys/bus/usb/
    cat /proc/bus/usb/devices
⚙️ Internals
    drivers/usb$_{src}$
📖 References
    USB$_{doc}$
    LDD3:USB Drivers
Other buses
    drivers/bus$_{src}$
Buses for 🤖 embedded devices:
    linux/gpio/driver.h$_{inc}$ linux/gpio.h$_{inc}$ drivers/gpio$_{src}$ tools/gpio$_{src}$
    drivers/i2c$_{src}$ https://i2c.wiki.kernel.org
SPI
⚲ API
    linux/spi/spi.h$_{inc}$
    tools/spi$_{src}$
⚙️ Internals
    drivers/spi$_{src}$
    spi_register_controller$_{id}$
    spi_controller_list$_{id}$🚧
📖 References
    SPI$_{doc}$

## Hardware interfaces

Hardware interfaces are basic part of any operating, enabling communication between the processor and other HW components of a computer system: memory, peripheral devices and buses, various controllers.
Interrupts

### I/O ports and registers

I/O ports and registers are electronic components in computer systems that enable communication between CPU and other electronic controllers and devices.
⚲ API
linux/regmap.h$_{inc}$ — register map access API
asm-generic/io.h$_{inc}$ — generic I/O port emulation.
    ioport_map$_{id}$
    ioread32$_{id}$ / iowrite32$_{id}$ ...
    readl$_{id}$/ writel$_{id}$ ...
    The {in,out}[bwl] macros are for emulating x86-style PCI/ISA IO space:
    inl$_{id}$/ outl$_{id}$ ...
linux/ioport.h$_{inc}$ — definitions of routines for detecting, reserving and allocating system resources.
    request_mem_region$_{id}$
arch/x86/include/asm/io.h$_{src}$
Functions for memory mapped registers:
ioremap$_{id}$ ...

### Hardware Device Drivers

Keywords: firmware, hotplug, clock, mux, pin
⚙️ Internals
    drivers/acpi$_{src}$
    drivers/base$_{src}$
    drivers/sdio$_{src}$ - Secure Digital Input Output
    drivers/virtio$_{src}$
    drivers/hwmon$_{src}$
    drivers/thermal$_{src}$
    drivers/pinctrl$_{src}$
    drivers/clk$_{src}$
📖 References
    Pin control subsystem$_{doc}$
    Linux Hardware Monitoring$_{doc}$
    Firmware guide$_{doc}$
    Devicetree$_{doc}$
    https://hwmon.wiki.kernel.org/
    LDD3:The Linux Device Model
    http://www.tldp.org/LDP/tlk/dd/drivers.html
    http://www.xml.com/ldd/chapter/book/
    http://examples.oreilly.com/linuxdrive2/

### Booting and halting
#### Kernel booting

This is loaded in two stages - in the first stage the kernel (as a compressed image file) is loaded into memory and decompressed, and a few fundamental functions such as essential hardware and basic memory management (memory paging) are set up. Control is then switched one final time to the main kernel start process calling start_kernel$_{id}$, which then performs the majority of system setup (interrupts, the rest of memory management, device and driver initialization, etc.) before spawning separately, the idle process and scheduler, and the init process (which is executed in user space).
Kernel loading stage
The kernel as loaded is typically an image file, compressed into either zImage or bzImage formats with zlib. A routine at the head of it does a minimal amount of hardware setup, decompresses the image fully into high memory, and takes note of any RAM disk if configured. It then executes kernel startup via startup_64 (for x86_64 architecture).
    arch/x86/boot/compressed/vmlinux.lds.S$_{src}$ - linker script defines entry startup_64$_{id}$ in
    arch/x86/boot/compressed/head_64.S$_{src}$ - assembly of extractor
    extract_kernel$_{id}$ - extractor in language C
       prints
Kernel startup stage
The startup function for the kernel (also called the swapper or process 0) establishes memory management (paging tables and memory paging), detects the type of CPU and any additional functionality such as floating point capabilities, and then switches to non-architecture specific Linux kernel functionality via a call to start_kernel$_{id}$.
↯ Startup call hierarchy:
    arch/x86/kernel/vmlinux.lds.S$_{src}$ – linker script
    arch/x86/kernel/head_64.S$_{src}$ – assembly of uncompressed startup code
    arch/x86/kernel/head64.c$_{src}$ – platform depended startup:
       x86_64_start_kernel$_{id}$
       x86_64_start_reservations$_{id}$
    init/main.c$_{src}$ – main initialization code
       start_kernel$_{id}$ 200 SLOC
          mm_init$_{id}$
             mem_init$_{id}$
             vmalloc_init$_{id}$
          sched_init$_{id}$
          rcu_init$_{id}$ – Read-copy-update
          rest_init$_{id}$
             kernel_init$_{id}$ - deferred kernel thread #1
                kernel_init_freeable$_{id}$ This and following functions are defied with attribute __init$_{id}$
                   prepare_namespace$_{id}$
                      initrd_load$_{id}$
                      mount_root$_{id}$
                run_init_process$_{id}$ obviously runs the first process man 1 init
             kthreadd$_{id}$ – deferred kernel thread #2
             cpu_startup_entry$_{id}$
                do_idle$_{id}$
start_kernel$_{id}$ executes a wide range of initialization functions. It sets up interrupt handling (IRQs), further configures memory, starts the man 1 init process (the first user-space process), and then starts the idle task via cpu_startup_entry$_{id}$. Notably, the kernel startup process also mounts the initial ramdisk (initrd) that was loaded previously as the temporary root file system during the boot phase. The initrd allows driver modules to be loaded directly from memory, without reliance upon other devices (e.g. a hard disk) and the drivers that are needed to access them (e.g. a SATA driver). This split of some drivers statically compiled into the kernel and other drivers loaded from initrd allows for a smaller kernel. The root file system is later switched via a call to man 8 pivot_root / man 2 pivot_root which unmounts the temporary root file system and replaces it with the use of the real one, once the latter is accessible. The memory used by the temporary root file system is then reclaimed.
⚙️ Internals
    arch/x86/Kconfig.debug$_{src}$
    linux/smpboot.h$_{inc}$
    kernel/smpboot.c$_{src}$
    arch/x86/kernel/smpboot.c$_{src}$
📖 References
    Article about booting of the kernel
    Initial RAM disk$_{doc}$
    initcall_debug, boot argument
    Linux startup process
    init
    Linux (U)EFI boot process
    The kernel’s command-line parameters$_{doc}$
    The EFI Boot Stub$_{doc}$
    Boot Configuration$_{doc}$
    Boot time memory management$_{doc}$
    Kernel booting process
    Kernel initialization process
📚 Further reading
    Boot-time tracing$_{doc}$
    E820
💾 Historical
    http://tldp.org/HOWTO/Linux-i386-Boot-Code-HOWTO/
    http://www.tldp.org/LDP/lki/lki-1.html
    http://www.tldp.org/HOWTO/KernelAnalysis-HOWTO-4.html

#### Halting or rebooting

🔧 TODO
⚲ API
    linux/reboot.h$_{inc}$
    linux/stop_machine.h$_{inc}$
       print_stop_info$_{id}$
       stop_machine$_{id}$
          stop_core_cpuslocked$_{id}$
    reboot_mode$_{id}$
    sys_reboot$_{id}$ calls
       machine_restart$_{id}$ or
       machine_halt$_{id}$ or
       machine_power_off$_{id}$
    linux/reboot-mode.h$_{inc}$
       reboot_mode_driver$_{id}$
          devm_reboot_mode_register$_{id}$
⚙️ Internals
    kernel/reboot.c$_{src}$
    kernel/stop_machine.c$_{src}$
       cpu_stopper$_{id}$
       cpu_stop_init$_{id}$
          cpu_stopper_thread$_{id}$ – "migration" tasks
    arch/x86/kernel/reboot.c$_{src}$
    Softdog Driver

#### Power management

Keyword: suspend, alarm, hibernation.
⚲ API
    /sys/power/
    /sys/kernel/debug/wakeup_sources
       ⌨️ hands-on:
       sudo awk '{gsub("^ ","?")} NR>1 {if ($6) {print $1}}' /sys/kernel/debug/wakeup_sources
    linux/pm.h$_{inc}$
       linux$_{inc}$
    linux/pm_qos.h$_{inc}$
    linux/pm_clock.h$_{inc}$
    linux/pm_domain.h$_{inc}$
    linux/pm_wakeirq.h$_{inc}$
    linux/pm_wakeup.h$_{inc}$
       wakeup_source$_{id}$
       wakeup_source_register$_{id}$
    linux/suspend.h$_{inc}$
       pm_suspend$_{id}$ suspends the system
    Suspend and wakeup depend on
       man 2 timer_create and man 2 timerfd_create with clock ids CLOCK_REALTIME_ALARM$_{id}$ or CLOCK_BOOTTIME_ALARM$_{id}$ will wake the system if it is suspended.
       man 2 epoll_ctl with flag EPOLLWAKEUP$_{id}$ blocks suspend
       See also man 7 capabilities CAP_WAKE_ALARM$_{id}$, CAP_BLOCK_SUSPEND$_{id}$
⚙️ Internals
    CONFIG_PM$_{id}$
    CONFIG_SUSPEND$_{id}$
    kernel/power$_{src}$
    alarm_init$_{id}$
    kernel/time/alarmtimer.c$_{src}$
    drivers/base/power$_{src}$: wakeup_sources$_{id}$
📖 References
    PM administration$_{doc}$
    CPU and Device PM$_{doc}$
    Power Management$_{doc}$
    sysfs power testing ABI$_{doc}$
    https://lwn.net/Kernel/Index/#Power_management
    PowerTOP
    cpupower
    tlp – apply laptop power management settings
    ACPI – Advanced Configuration and Power Interface

##### Runtime PM

Keywords: runtime power management, devices power management opportunistic suspend, autosuspend, autosleep.
⚲ API
    /sys/devices/.../power/:
       async autosuspend_delay_ms control runtime_active_kids runtime_active_time runtime_enabled runtime_status runtime_suspended_time runtime_usage
    linux/pm_runtime.h$_{inc}$
       pm_runtime_mark_last_busy$_{id}$
       pm_runtime_enable$_{id}$
       pm_runtime_disable$_{id}$
       pm_runtime_get$_{id}$ – asynchronous get
       pm_runtime_get_sync$_{id}$
       pm_runtime_resume_and_get$_{id}$ – preferable synchronous get
       pm_runtime_put$_{id}$
       pm_runtime_put_noidle$_{id}$ – just decrement usage counter
       pm_runtime_put_sync$_{id}$
       pm_runtime_put_autosuspend$_{id}$
    SET_RUNTIME_PM_OPS$_{id}$
👁 Example: ac97_pm$_{id}$
⚙️ Internals
    CONFIG_PM_AUTOSLEEP$_{id}$
📖 References
    Runtime Power Management Framework for I/O Devices$_{doc}$
    CPU idle power saving methods for real-time workloads
    Sysfs devices PM API
    Power Management for USB
    Opportunistic suspend

## Building and Updating

    Updating