# The Linux Kernel/Networking

* networking
* sockets access
* address families: inet, unix, ...
* network storage
* protocols
* network interfaces
* network drivers
Linux kernel network functionality spans from sockets interface through protocols to network cards.
⚲ Shell interfaces:
    man 8 netstat prints network connections, routing tables, interface statistics and other details
    man 8 ip shows and configures routing, network devices, interfaces and tunnels
    man 8 ss - socket statistics utility

## Sockets

⚲ API:
sys/socket.h – main user mode sockets header
Basic common and client side interface:
    man 2 socket ↪ __sys_socket$_{id}$ creates an endpoint for communication
    struct sockaddr$_{id}$ - abstract socket address
    man 2 connect ↪ __sys_connect$_{id}$;
    man 2 shutdown shuts down part of a full-duplex connection
    man 2 send ↪ __sys_sendto$_{id}$ sends a message on a socket
    man 2 recv ↪ __sys_recvfrom$_{id}$, __sys_recvmsg$_{id}$ receives a message from a socket
Additional server side interface:
    man 2 bind ↪ __sys_bind$_{id}$ - binds a sockaddr to a socket
    man 2 listen ↪ __sys_listen$_{id}$ - listens for connections on a socket
    man 2 accept ↪ __sys_accept4$_{id}$ - accepts a connection on a socket
⚙️ Internals
    struct socket$_{id}$ @ linux/net.h$_{inc}$ contains
       struct proto_ops$_{id}$ - abstract protocols interface
       struct sock$_{id}$ - network layer representation of sockets net/sock.h$_{inc}$
    __sys_socket$_{id}$ ↯ call hierarchy:
       sock_create$_{id}$
          __sock_create$_{id}$
             security_socket_create$_{id}$
             sock_alloc$_{id}$
                net_proto_family$_{id}$->create.
                   for example inet_create$_{id}$. See Address families for another options.
    __sys_connect$_{id}$ ↯ call hierarchy:
       move_addr_to_kernel$_{id}$
          audit_sockaddr$_{id}$
          __sys_connect_file$_{id}$
             sock_from_file$_{id}$
             security_socket_connect$_{id}$
             proto_ops$_{id}$->connect.
                for example inet_stream_connect$_{id}$. See Protocols for another options.
    net/socket.c$_{src}$
📚 References
    man 7 socket
    linux/socket.h$_{inc}$
    Berkeley sockets

## Network storage

🚀 advanced topic
🔧 TODO
⚲ API:
    man 2 sendfile ↪ do_sendfile$_{id}$.
See also Zero-copy between file descriptors
    Application layer: Network File System
    NFS$_{doc}$
       init_nfs_fs$_{id}$, nfs4_fs_type$_{id}$, nfs_fs_type$_{id}$,
       init_nfsd$_{id}$, nfsd_fs_type$_{id}$
    CIFS$_{doc}$
       init_cifs$_{id}$
       cifs_fs_type$_{id}$, smb3_fs_type$_{id}$ cifs_smb3_do_mount$_{id}$
    target and iSCSI Interfaces Guide$_{doc}$

## Logistics
### Names

⚲ API: man 2 uname, man 2 sethostname, man 2 gethostname, man 2 setdomainname man 2 getdomainname
    ↪ utsname$_{id}$
⚙️ Details
    utsname$_{id}$ returns writable pointer to new_utsname$_{id}$ from uts_namespace$_{id}$ from nsproxy$_{id}$ from current$_{id}$ task_struct$_{id}$.
    CLONE_NEWUTS$_{id}$, setns$_{id}$
    kernel/utsname.c$_{src}$
📚 References:
    man 7 namespaces
    man 7 network_namespaces
    man 7 uts_namespaces

### Address families

⚲ API
    man 2 getsockname
    man 2 getpeername
    Address Family (AF) domain defines address format and address length socklen_t.
    man 3 inet_ntop, man 3 inet_pton (derive socklen_t from AF)
Common AF: AF_UNIX$_{id}$, AF_INET$_{id}$, AF_NETLINK$_{id}$.
PF - Protocol Family index (PF_MAX$_{id}$) actually is the same as Address Family index (AF).
⚙️ Internals of some AF
    man 7 unix ↪ unix_family_ops$_{id}$ - sockets for local IPC
       unix_create$_{id}$
    man 7 ip ↪ inet_family_ops$_{id}$ - IPv4
       inet_create$_{id}$
    man 7 netlink ↪ netlink_family_ops$_{id}$ - communication between kernel and user space
       netlink_create$_{id}$
    man 7 vsock ↪ vsock_family_ops$_{id}$ - communication between VM and hypervisor
       vsock_create$_{id}$
    man 7 packet ↪ packet_family_ops$_{id}$ - device level interface
       packet_create$_{id}$
    bt_sock_family_ops$_{id}$ - Bluetooth
       bt_sock_create$_{id}$
Totally there are more than 40 AFs (see AF_MAX$_{id}$)
⚙️ Internals
    sock_register$_{id}$ - registers net_proto_family$_{id}$. See references to this identifiers to find more than 30 protocol families.
    __sock_create$_{id}$
📚 Further reading
    man 8 ip-address – protocol address management
    Internet layer
    man 7 address_families

### Protocols

Each Protocol Family (PF, same index as Address Family AF) consists of several protocol implementations.
Directory /proc/net contains various files and subdirectories containing information about the networking layer. File /proc/net/protocols lists available and used protocols.
In each PF protocols are classified to different types sock_type$_{id}$, for example stream, datagram and raw socket. TCP is type of stream, UDP is type of datagram, raw and ping are type of raw.
    proto_register$_{id}$ - registers struct proto$_{id}$ - protocol implementations:
    In inet_init$_{id}$ initcall, inetsw_array$_{id}$, proto_ops$_{id}$ and proto$_{id}$ :
       inet_stream_ops$_{id}$ & tcp_prot$_{id}$ tcp_sendmsg$_{id}$ ...
       inet_dgram_ops$_{id}$ & udp_prot$_{id}$ udp_sendmsg$_{id}$ ...
       inet_sockraw_ops$_{id}$
          raw_prot$_{id}$ raw_sendmsg$_{id}$ ...
          ping_prot$_{id}$ ping_v4_sendmsg$_{id}$ ...
    In af_unix_init$_{id}$ initcall:
       unix_family_ops$_{id}$
          unix_create$_{id}$
             unix_stream_ops$_{id}$ unix_stream_sendmsg$_{id}$ ...
             unix_dgram_ops$_{id}$ unix_dgram_sendmsg$_{id}$ ...
             unix_seqpacket_ops$_{id}$ unix_seqpacket_sendmsg$_{id}$ ...
📚 References:
    man 7 tcp
    man 7 udp
    man 7 raw
    Transport layer and TCP

### RDMA

🚀 advanced topic
🗝️ Acronyms:
    IB — InfiniBand, an interconnect standard, competes with Ethernet, Fibre Channel
    IPoIB — IP network emulation layer over InfiniBand networks
    SRP — SCSI RDMA Protocol
    ULP — Upper-layer protocols
    iSER — iSCSI Extensions for RDMA
⚲ Interfaces:
    https://github.com/linux-rdma/rdma-core
    man 8 rdma
    man 7 rdma_cm — RDMA communication manager
    include/uapi/rdma$_{src}$
    include/rdma$_{src}$
⚙️ Internals:
    drivers/infiniband$_{src}$
       drivers/infiniband/ulp$_{src}$ — Upper-layer protocols
       drivers/infiniband/sw$_{src}$ — software drivers
       drivers/infiniband/hw$_{src}$ — hardware device drivers
📚 References:
    InfiniBand$_{doc}$
    InfiniBand and RDMA Interfaces$_{doc}$

## Netfilter

🚀 advanced topic
⚲ Interface:
    man 8 ebtables-nft
    man 8 arptables-nft
    man 8 xtables-nft
    man 8 iptables
    man 8 ip6tables
    man 8 ebtables
    man 8 arptables
    ipset
    linux/netfilter.h$_{inc}$
    uapi/linux/netfilter$_{inc}$
    net/netfilter$_{inc}$
    net/netns/netfilter.h$_{inc}$
    linux/netfilter$_{inc}$
⚙️ Internals:
    net/netfilter$_{src}$
📚 References:
    Netfilter Sysfs variables$_{doc}$
    Netfilter Conntrack Sysfs variables$_{doc}$
    Netfilter’s flowtable infrastructure$_{doc}$
    nftables
    https://wiki.nftables.org/
    https://lwn.net/Kernel/Index/#Networking-Packet_filtering

## Network device interfaces

⚲ Interfaces
    ip -brief link show
    ls -l /sys/class/net
    devm_register_netdev$_{id}$ registers net_device$_{id}$, net_device_ops
    sk_buff$_{id}$ socket buffer (skb)
       dev_queue_xmit$_{id}$ queues socket buffers into transmit queue
    linux/netdevice.h$_{inc}$
    linux/skbuff.h$_{inc}$
👁 Example: drivers/net/loopback.c$_{src}$ - the most famous and simple interface lo
⚙️ Internals
    net/core/dev.c$_{src}$
    function loopback_xmit$_{id}$ receives skb and passes it back with netif_rx$_{id}$
📚 Further reading
    man 8 ip-link – network device configuration
    man 8 ip-stats – manage and show interface statistics
    man 7 netdevice – low-level access to Linux network devices
    man 7 packet – packet interface on device level
    Queueing in the Linux Network Stack
💾 Historical
    LDP TLK Chapter 10 Networks

## Network drivers

    linux/etherdevice.h$_{inc}$
    netif_rx$_{id}$ - before NAPI
    input_pkt_queue$_{id}$
    NAPI
    NAPI Driver design
       ⚲ API:
          netif_napi_add$_{id}$ adds napi_struct$_{id}$
          napi_schedule$_{id}$ - called by an IRQ handler to schedule a poll
          netif_receive_skb$_{id}$ - instead netif_rx, finally calls ip_rcv$_{id}$
          napi_complete_done$_{id}$ - called from custom napi->poll
       ⚙️ Internals:
          net_dev_init$_{id}$
             net_rx_action$_{id}$
                napi_poll$_{id}$ calls custom napi->poll
       👁 example
          e1000_intr$_{id}$ calls __napi_schedule$_{id}$
          custom napi->poll e1000e_poll$_{id}$ calls napi_complete_done$_{id}$
    ether_setup$_{id}$ setups Ethernet network device
    👁 An example of Ethernet driver: e1000_probe$_{id}$
⚙️ Internals:
    drivers/net$_{src}$
    drivers/net/wireless$_{src}$
    drivers/net/ethernet$_{src}$
📚 References:
    man 8 ethtool – query or control network driver and hardware settings
    Data link layer: Ethernet
    GRO - Generic Receive Offload
    Segmentation Offloads$_{doc}$
    https://wireless.wiki.kernel.org
💾 Historical:
    LDD2:Network Drivers
    LDD3:Network Drivers
    Kernel Analysis: Networking, 2003
    network_overview
📖 Further reading about networking
    Networking interfaces$_{doc}$
       Networking$_{doc}$
    https://lwn.net/Kernel/Index/#Networking
    https://lartc.org/ – Linux Advanced Routing & Traffic Control
    man 8 ip – show / manipulate routing, network devices, interfaces and tunnels
    man 8 tc – show / manipulate traffic control settings
    bcc/ebpf networking tools
    eBPF-based Networking, Security, and Observability
    Retis – tracing packets in the Linux networking stack & friends