Mainlining

Category:Socs
Mainlining is the process of replacing the kernel provided by the device vendor (the "downstream" kernel), with a version close to the actively developed version released on kernel.org (the "mainline" kernel). It involves porting (and most importantly: cleaning up) device-specific code from the downstream kernel to the mainline kernel. For some , common support already exists in mainline, so you can focus on device-specific components.
== Why? ==
Mainlining is a fairly complicated process, especially if you're one of the first to start work on a given SoC. However through community effort there is a growing number of SoCs that have increasingly stable mainline ports (see below). In general it is relatively simple to take your existing Android kernel and boot it without any of the Android blobs, however from that point it can be a lot of working to create a working system and at the end of it you don't receive any security patches or general improvements.
Apart from purely practical insufficiencies, the cold hard truth is that the downstream code is simply not meant to be maintained or get used outside Android. The code quality is often several magnitudes worse, which makes maintenance rather complicated. Often large parts of the drivers are moved into proprietary parts in Android userspace - which are generally impossible to use without emulating the expected Android environment. At the end, the effort to make such drivers work properly on long term is often as much as rewriting them properly for mainline.
Successfully mainlining an SoC or device means that you benefit from every improvement the Linux kernel has to offer, you can run a fairly standard Linux distro (in theory even any Linux distro that has support for your device’s architecture) and benefit from kernel updates and improvements without needing to compile your own kernel or deal with frustrating patches. A notable easy-to-notice benefit of running mainline is minimal-effort-needed GPU support for almost all GPUs (except PowerVR unfortunately), which is needed for using modern phone DEs like Plasma Mobile or Phosh with acceptable performance. Lastly, mainlining is a terrific learning opportunity that you will never regret.
If your device's SoC is in the list below there's a very decent chance that you can get the same functionality working on your device and eventually upstream your patches to the kernel, helping to build a solid base for other devices too!
== Overview ==
=== Supported SoCs ===
Some SoCs are already supported quite well and are used for some devices in postmarketOS. Getting started with one of these might be easier for you, because you can get help from others working with the same SoC. The feature matrix below shows which features are already supported for a particular SoC. The pages for each SoC will provide more information, plus hints how to get started and who you can contact.
* CPU: SMP (bring up secondary CPU cores), CPU frequency scaling, CPUidle
* Storage: eMMC, SD cards, UFS, ...
* Video: Hardware-accelerated video de/encoding
* Modem: Calls, SMS, Internet
* Suspend: Can reach (all) low-power modes of the SoC during suspend (either system suspend or cpuidle).
* Unavailable (-) means that such a feature is not built directly into SoC. For example, WiFi/BT are also often used through SDIO/UART which can be chosen independently from the SoC.
Note: The statuses displayed below signify if a particular feature was tested on a device by someone in the postmarketOS community - not necessarily the components supported in the mainline kernel.
=== Getting help ===
Mainlining related to postmarketOS is primarily discussed in the #postmarketOS-mainline channel. For one of the SoCs above there should be someone around who is able to help you to some extent. Note that you will be on your own for some of the problems - there are often device specific problems or components that no one has experience with yet.
Below are some more chat channels / mailing list that may be helpful for further help or if you want to work on a SoC that is not listed above (yet!):
In Matrix, you can browse most of Linux kernel development related channels in #linux-kernel-devspace
SoC 	Where to ask for help
 	#postmarketOS-mainline (SoC-specific questions might be limited to those listed above)
Allwinner 	#linux-sunxi on OFTC, linux-sunxi@googlegroups.com
Amlogic 	#linux-amlogic on Libera.Chat, linux-amlogic@lists.infradead.org; general information is available at linux-meson.com.
Broadcom Kona 	ask knuxify, #bcm-kona-mainline:dissonant.dev on Matrix; bcm-kona-mainline on GitHub
Exynos 	#replicant (with focus on older variants) or #linux-exynos on Libera.chat (ask forkbomb, Putti, krzk), linux-samsung-soc@vger.kernel.org
Samsung Exynos 4 	Exynos entry + #exynos4xxx-mainline:matrix.org on Matrix
Samsung Exynos 5 	Exynos entry + #exynos5-mainline:matrix.org on Matrix
Marvell PXA1908 (incl. other PXA and MMP) 	#pxa1908-mainline on OFTC
MediaTek 	#linux-mediatek on Libera.chat, linux-mediatek@lists.infradead.org
MediaTek MT6735 	MediaTek entry + join #mt6735-foss:matrix.org
OMAP3 / OMAP4 	ask @pavelm, linux-omap@vger.kernel.org
Qualcomm (msm...) 	#linux-msm on OFTC, linux-arm-msm@vger.kernel.org
Qualcomm Snapdragon 400 (MSM8226) 	Qualcomm entry + join #msm8226-mainline:postmarketos.org
Qualcomm Snapdragon 210 (MSM8909) 	Qualcomm entry + join #msm8916:postmarketos.org
Qualcomm Snapdragon 410/412 (MSM8916) 	Qualcomm entry + join #msm8916:postmarketos.org
Qualcomm Snapdragon 450/625/626/632 (MSM8953) 	Qualcomm entry + join #msm8953-mainline:matrixorg
Qualcomm Snapdragon 600 (APQ8064) and Qualcomm Snapdragon S4 Plus (MSM8960)	Qualcomm entry + join #linux-apq8064:matrix.org
Qualcomm Snapdragon 615/616 (MSM8939) 	Qualcomm entry + join #msm8916:postmarketos.org
Qualcomm Snapdragon 660/630/636 (SDM660/SDM630/SDM636) 	Qualcomm entry + join #sdm660-mainline:matrix.org
Qualcomm Snapdragon 670 (SDM670) 	Qualcomm entry + join #sargo/bonito porting:erebion.eu
Qualcomm Snapdragon 730/730G/732G (SM7150) 	Qualcomm entry + join #sm7150-mainline:matrix.org
Qualcomm Snapdragon 820/821 (MSM8996) 	Qualcomm entry + join #msm8996-mainline:matrix.org
Qualcomm Snapdragon 845 (SDM845) 	Qualcomm entry + ask Caleb and join #sdm845:postmarketos.org
Qualcomm Snapdragon 865/865+/870 (SM8250) 	Qualcomm entry + join #sm8250-mainline:matrix.org
Qualcomm Snapdragon 8 Gen 1/8+ Gen 1/7+ Gen 2 (SM8450/SM8475/SM7475) 	Qualcomm entry + join #sm8450-mainline:matrix.org
Qualcomm Snapdragon 8 Gen 3 (SM8650) 	#linux-msm and #sm8650-mainline on Matrix or OFTC
Nvidia Tegra 	matrix.org:#postmarketOS-on-transformers:matrix.org (initially focused on the Asus Eee Pad Transformer family, now also offers help with other Tegra devices under PMOS), #tegra on Libera.chat, linux-tegra@vger.kernel.org
== Getting Started ==
* Collect as much information about your device as you can: downstream kernel sources, specifications, schematics (can be sometimes found with your favorite search engine) are all going to prove invaluable when attempting to port a device to mainline.
* Serial debugging will be helpful especially in the beginning (before USB or network is working). Some devices expose it through the USB port or the audio jack; others have test points available on the mainboard.
* Choose a kernel tree:
** Mainline: (the last kernel version released by Linus Torvalds) Usually a good compromise of being recent, yet quite stable. torvalds/linux.git
** linux-next: A collection of recently merged changes from many subsystems; use this if you are working on a very recent SoC. next/linux-next.git
** There may be more WIP changes available in Linux kernel trees specific to your SoC.
* Compiling kernels with envkernel.sh is very useful to integrate kernel development into postmarketOS.
* If you are attempting to port to a SoC listed above, check the SoC wiki page for further help and instructions.
* Do not attempt to copy any code as-is from downstream. In general this won't work, and most importantly: it won't be accepted for inclusion into the mainline kernel upstream.
** Instead, try to understand what the downstream code does, and rewrite it from scratch for mainline by looking at similar code.
== Materials to look into ==
This section include many useful sources to help you with mainlining as possible.
Note: Linux architecture regarding to embedded devices moving fast forward, so be aware that some specific information can get outdated in 2 – 3 years.
Books:
* Mastering Linux Device Driver Development: Write custom device drivers to support computer peripherals in Linux operating systems (2021, John Madieu) 
* Linux Driver Development for Embedded Processors - Second Edition: Learn to develop Linux embedded drivers with kernel 4.9 LTS (2018, Alberto Liberal De Los Ríos)
Intro:
* Porting mainline Linux to mobile phones, Luca Weiss, 2022 (40:00) FOSDEM
* Running Mainline Linux on Snapdragon 410, Nikita Travkin, 2022 (40:00) FOSDEM
* From Android to mainline on the Snapdragon 845, Caleb Connolly, 2022 (30:00) FOSDEM
* Qualcomm SoC upstreaming adventures in 2020 Youtube Odysee
* Porting Linux to your favorite obscure Arm SoC, 2020 media.ccc.de Youtube Odysee
* Upstreaming a Qualcomm SoC, 2020 YouTube Slides Overview of porting 2020-era SoC
* Introduction into how mainlining is done, 2017 (52:37) Youtube Odysee
Basics:
* Introduction into ARM architecture, 2017 (46:33) Youtube Odysee Slides
* ARM64 SoC Linux Support Check-List, 2017 (42:38) Youtube Odysee Slides Very useful step-by-step guide what should be done to bring in SoC support
* Upstreaming ARM64 SoC's easier than before, 2019 (19:03) Youtube Odysee Slides
* Mainlining blog by ichernev
* How the ARM32 kernel starts by Linus Walleij
Writing device trees:
* Device Tree 101, 2021 (1:54:58) Youtube
* Device Tree (DT) introduction, 2013 (1:12:14) Youtube Odysee Slides
* Device Tree: present, past, future, 2018 (37:28) Youtube Odysee Slides
* Device tree bindings (i.e. documentation how to specify devices in your device tree) are mandatory for all mainline drivers. You can check those in the Documentation/devicetree/bindings directory to see which options are supported by a particular driver.
* Device Tree Usage (beginner guide by elinux.org)
* Device Tree Specification
* DT Resources on elinux.org
Clocks:
* Common Clock Framework: How To Use It, 2013 (44:50) Youtube OdyseeNot so useful, old and only a basic overview
* What The Clock! - Linux Clock Subsystem Internals, 2020 (33:12) Odysee Youtube Slides
* Clock sources, Clock events, sched_clock() and delay timers at kernel.org
Power Management
* SAN19-421 Training: Device power management for idle Youtube
Camera
* Camera⧸ISP Drivers Using V4L2 Media Controller Framework (35m) Youtube Odysee
Various topics:
* Pin Control and GPIO, 2013 (48:03) Youtube Odysee Slides talk by Linus Walleij
* Timekeeping in the Linux Kernel - Stephen Boyd, Qualcomm Innovation Center, 2017 (34:41) Youtube Odysee About how clocksource works.
* How Dealing with Modern Interrupt Architectures can Affect Your Sanity Youtube Odysee Slides About interrupt controllers and their relationships
* Linux Kernel Display Architecture, 2013 (56:53) Youtube Odysee DRM/KMS, MIPI DSI, panels,... slightly old, need something newer.
* Demystifying Linux MIPI DSI Subsystem - Jagan Teki, 2019 (41:57) Youtube Odysee MIPI DSI, controllers, bridges, panels, DRM/KMS...
* Power Management Integrated Circuits: Keep the Power in Your Hands, 2017 Youtube Odysee Slides About PMICs, regulators, power supply, fuelgauge, ...
Development tools
* Using Visual Studio Code for Embedded Development - Michael Opdenacker, Bootlin, 2021 (31:37) Youtube
Note: for VSCodium, download extensions from https://marketplace.visualstudio.com/ manually.
Submitting patches:
* Submitting Patches wiki page
* Submitting patches: the essential guide to getting your code into the kernel
* Submitting Your First Patch to the Linux Kernel and Responding to Feedback
* Guide for git-send-email
== Upstreaming ==
Upstreaming your mainline efforts is key to reduce the maintenance burden of rebasing your patches on newer version of the Linux kernel.
Upstreaming patches requires sending to the Linux Kernel Mailing List using git send-email.
For a tutorial on git send-email have a look at https://git-send-email.io/.
See https://wiki.postmarketos.org/wiki/Submitting_Patches#Linux for a detailed overview for submitting patches upstream.
== See also ==
* Mainlining Guide (very generic, parts are slowly incorporated into this page)
* MSM8916 Mainlining (specific to MSM8916, but might be helpful for similar Qualcomm SoCs)
* MSM8996 Mainlining
* SDM845 Mainlining
* Mainlining FAQ
* The Mainline Kernel
* Qualcomm mainline porting
* Exynos mainline porting
* Google Galaxy Nexus (samsung-maguro): Mainlining progress (initial tricks with RAM and some sleep calls before the reboots in the code to figure out where it was failing will also be useful for other devices)
* Google Pixel 3 XL (google-crosshatch): mainlining attempt: progress and questions (useful for newer SoCs, useful info about force-reboot code, stub dtbo partition)