[Android General] mirfatif:
LINUX vs. ANDROID

. . . continued . . .

LINUX BE POSSIBLY INSTALLED ON ANDROID DEVICES?​

It's both: YES AND NO.

Though there is no (or negligible) proprietorship in Linux, there are technical obstructions as discussed above which make it difficult to install Linux on Android device. "Ubuntu for Android" is a stance of such failed attempt. It made use of Linux Containers (LXC) to run Linux with Android kernel. Ubuntu did their homework but abandoned due to no support from Android side. "Ubuntu Touch" and "Ubuntu Phone" are subsequent projects, almost abandoned. But there is somewhat proprietorship involved i.e. dependence on closed-source drivers.

And there is SHR.

Lets have a look at Linux vs Android to know why and why not:

Linux for ARM

Due to its open-source nature, many Linux distributions are available for ARM architecture. ArchLinux ARM port is available mainly for mini single board computers and tablets. Ubuntu ports are available for arm/arm64 as well as officially available for servers. Fedora supports desktops, laptops and workstations as well. Debian supports development boards, network storage and QEMU (virtual machine / hardware emulator) too. There are many others as well, mostly in development or trial phases.

Booting process of Android and Linux resembles up to some degree. Linux mainly focuses on PC-like systems or specially customized embedded solutions. PC's have a standard BIOS/UEFI to take control after power on unlike Android devices as discussed above in "PC vs Mobile" section. Android devices, being mobile phones / tablets, don't have any analogous of BIOS.

Since there is no BIOS/UEFI, there is no standard ACPI (power management) on Android; apps use wake locks for ON state, otherwise device is sleeping.

Android uses ARM devices mostly with SoC's manufactured by Qualcomm, which have very least support for mainline Linux kernel contrary to other silicon vendors like Allwinner and Atmel etc.

Read ANDROID BOOT PROCESS for further explanation.

Bootloader

Most commonly used bootloader in Linux is GRUB which comes bundled with OS and often resides with OS on the same partition. It's open source and you can even configure it. Android's most common used bootloader is aboot (or its derivative like hboot) which is closed source and signed by Qualcomm (SoC vendor) and it lives on a separate partition with it's suppportive components residing on multiple other partitions. It's to secure the device from booting any other bootloader, again for commercial purposes or to protect user's privacy. Also aboot comes locked and can't boot unsigned kernel images other than officially provided by OEM. However, in most cases, it's unlockable. Thus, Linux bootloader can't replace Android's bootloader. There's u-boot, the traditional embedded Linux bootloader but that too is helpless here coming to the "signed" part.

Kernel

Kernel being the core of OS, manages all hardware resources particularly memory and CPU. It uses a scheduler to arbitrate which tasks take priority at any given moment, creating the illusion of many tasks being executed simultaneously.

We have an agreement on this part. Android kernel is inherited from Linux and both are open source (at least by GPL T&C) and similar enough. Though again Android kernel lives on an independent partition named "boot", similar to partition or directory "/boot" on Linux which also contains grub. But the differences in implementation of kernel are enough for incompatibility and the reason is non-standard nature of Android devices. Linux kernel is designed for standard PC's and include drivers (or at least drivers support) for almost all hardware vendors as does Windows. On the other hand, Android kernel, as explained above, is different for every device because every vendor includes its own hardware drivers which are not open source. Practically it's impossible to develop a universal Linux kernel of a few hundred MB's that works with such a diverse range of mobile phones.

In summary, we can run Linux on Android device if:

Android kernel provides all functionality of a Linux kernel (specifically compatible with glibc)

This condition is impossible to meet. Neither we can modify Android kernel to meet all needs of Linux glibc, neither we can modify userspace libraries to be compatible with Android kernel. Later will cut down many standard functions of Linux.

Or we have a fully working Linux kernel for the device (specifically drivers)

Compiling Linux kernel for Android is possible but the proprietary part (drivers) is almost impossible.

However, one may try to compile Linux kernel (native booting) for a specific Android device:

Get:

Android kernel source code

Firmware blobs / source from Android

Kernel modules from Linux (matching with kernel source version)

Compile Linux kernel (/boot/zImage and /boot/System.map-*) and initramfs (/boot/initrd.img.gz)

Let's say we are going to install Linux on second partition of SD card. Unpack Android boot.img and repack with Linux kernel, Linux initramfs, Android kernel base and cmdline something like "root=/dev/mmcblk1p2 console=tty0". Or pass root argument to bootloader while testing with fastboot:

fastboot -c "root=/dev/mmcblk1p2 rootwait" boot boot.img

"rdinit=" or "init=" might also need to be specified in cmdline.

"Framebuffer Console Support" and "Virtual Terminal Support" need to be enabled while compiling Linux kernel to enable "console=tty0" which is disabled by default on Android kernel.

But it's not very simple to achieve because all above requirements are rarely met. Getting all hardware work aside, a lot of kernel patches would be needed to hardly boot the device. Simple is to make some compromise and share Android kernel to start up a basic Linux environment.

Init (Startup service manager), sysV/systemD vs Android init

Init, being the very first process (PID 1) started by kernel, has very deep integration with kernel design. Linux traditionally has (almost had) sysV, then came, for a while, Upstart (by Ubuntu) and a few others, and now systemD (by RedHat) is most accepted service manager. Android init is derived from sysV init but it has additional responsibilities of ueventd (analogous of udev in Linux) and watchdog. It's open source and creates somewhat similar filesystems as those are on Linux. Services started by Android init are totally different than started on Linux. So, we cannot use Android init to start standard Linux services. But we can start a few of them (non-systemD, only from legacy paths /etc/init.d and /etc/rc[N].d, or direct execute binary) manually as daemons, once we have a basic Linux environment sharing Android kernel. However, init and particularly systemD starts services at boot in a defined order with entangled chain of dependencies on other services going up to kernel. Therefore, altering init's behavior (changing code and recompiling shared libraries blobs) is equal to writing a new OS.

Android init creates only one root filesystems from initramfs in which all other filesystem are mounted. While on Linux, usually rootfs created by initramfs is just for loading drivers etc. and it's subsequntly replaced by a persistent storage (partition) rootfs.

Linux init provides a platform to add or remove init services when new packages are installed or removed, by adding simple shell scripts to /etc/init.d on sysV or .service files to /etc/systemd.

But Android init has no such flexibility and it starts a fixed set of services at start, defined when ROM was made. Init configuration files found in initramfs/init*.rc and in /system/etc/*.rc or /vendor/etc/*.rc are meant to be used by vendors or ROM developers and not at user level. Though we can add a custom init .rc file, on Linux any user or package can modify and update initramfs at any time. We can add startup scripts to "/etc/init.d" on some rooted Android devices or custom ROMs which have init.d support enabled by modifying initramfs in boot.img. To start custom services/processes on boot, SELinux policy also needs to be modified such as daemon-su does.

Some other tasks which aren't part of init implementation on Linux, are handled by Android's init, such as configuring kernel parameters at boot. Android does not support /etc/sysctl.conf, so the updates to the /proc/sys/ are part of the "init.rc" file.

On a Linux system init along with scripts, binaries kernel drivers and modules (in initrd.img), kernel (vmlinuz executable) and bootloader configuration along with modules, they all reside on root or a separate partition (mounted) at /boot. While on Android, init along with a few binaries and configuration files and kernel reside in a separate partition named "boot" with a special filesystem.

A simple comparison of Ubuntu 16.04 (systemD) and Android 8.1 processes is given here.

Filesystems and Directories

As long as kernel part is concerned, /sys, /proc and /dev filesystems are same on Linux and Android with negligible differences. Both depend on a single root "/" filesystem. But other filesystems related to runtime environment and userspace differ i.e. /bin vs /system/bin, /home vs /data/media, /var, /usr, /media etc. But we can make use of resemblance in kernel and its filesystems to run Linux on Android device, not independently, but depending on Android OS for hardware handling.

A comparison of Android and Ubuntu is given here.

Virtual Terminals and Consoles

Linux kernel supports a native console and muxes multiple virtual consoles and terminals (accessible through Fn+F[N] keys) which are a primary method for user logins. Android uses modified Linux kernel, with only one /dev/tty which is the current terminal and the controlling terminal of any running process requiring a terminal. Like many eLinux systems, kernel console is either disabled on Android by default or not connected to any device and it's not muxing any virtual terminals because no text mode user login is intended and only GUI mode is used. However, support for both Virtual Terminal and Console in frambuffer mode can be enabled while compiling kernel.

For more details, have a look here.

Runtime Environment (Standard C library and Linker)

Let's think in another direction. Due to above said differences between Linux and Android, why don't we skip the installation part and run a complete Linux environment inside Android when it's completely booted? We can't.

As said at the start of this post, OS is a set of binaries/libraries compiled for a target OS and placed at specific paths to get maximum performance. These binaries/libraries execute in a certain order to perform required operations. For maximum utilization of hardware resources (i.e. to save space and minimize processing time), each program/biary/library isn't compiled independently (i.e. statically). In most cases, they are linked and dependent on each other (dynamically linked). A "linker" program (which is itself an OS specific static binary) links these binaries and libraries when they need to run. Dynamic linking makes executable files smaller and saves disk space, because one copy of a library can be shared between multiple programs.

Static linking is performed at build time by linker after compilation of libraries / binaries. If all required functions are statically linked to a library / binary, it doesn't need any support from OS at runtime, with a few exceptions. It's loaded in RAM as a standalone program just needing syscalls to kernel. However, in order to facilitate developers updating code and to save storage as well, static linking is not desirable in most cases. If we compile every application as a whole package, we'll be including same code hundred of times in many applications. Instead shared/dynamic libraries are referenced (using symbols, pointers) in program code at compile time but not included in executable. These symbols are resolved by linker i.e. libraries are searched recursively (searching libraries needed by already loaded libraries and so on) and linked to the program by creating addresses/offsets table in RAM before the program is executed. So the required libraries are loaded directly in RAM at runtime instead of linking in an archive file (static library) or binary (executable program) at compile time.

For dynamic linking, linker location is hard-coded in program (binary) code i.e. absolute path is set (can be specified by "--dynamic-loader" at compile time). Shared libraries are rarely hard-coded, instead those are searched in LD_PRELOAD, LD_LIBRARY_PATH, DT_RUNPATH directories, linker cache file or default directories.

Dynamic linking is often, but not always, accompanied by Dynamic Loading. It's the responsibility of program to load the required library when needed (not always at the start of process). Such programs (compiled with "-ldl" option) use dlopen syscall through libdl.so library for loading mechanism. In static loading, all libraries are loaded in RAM before execution of program starts.

Libraries are implemented through library files (lib*.so) and header files (.h). Header files (used at compile time) include function names while actual functions are defined in .c files. After adding the header file in a program code, functions provided by that library can be called instead of making syscalls to kernel.

One most important of these libraries is "libc" (Standard C Library) which is responsible for very core functions like memory management and input/output processing etc. Almost every single click or tap of yours on screen goes through the traditional libc API. libc provides very basic communication with kernel (through syscalls), acting as a bridge between kernel and userspace, so must be compatible with kernel (to make kernel understand syscalls).

C is the choice of many OS's because it's small in size, speedy and more human firendly as compared to other languages for low level programming. Libc is the most frquently linked library, almost every programs links to it for interaction with kernel. So, linker is implemented as a part of libc. Dynamically linked programs make use of std C API (header + .so files) while statically linked programs and kernel work independent of libc.

Similarly programs can make use of C++ Standard library which itself make use of all C header files (around 30 in no.) and a few additional. libz is also one of most excessively linked libraries.

Android and Linux have a different linkers implementation (/system/bin/linker and /lib/ld-linux.so respectively) and a different libc (/system/lib/libc.so and /lib/libc.so respectively).

Android owns a minimal set of Bionic (what Google calls it) libc while mainstream Linux uses glibc (GNU Library C) for its applications. They can't replace each other due to difference in functions they can perform and the locations too (filesystem hierarchy) where they reside along with other binaries/libraries. As mentioned in Filesystems, Android's major binaries and libararies reside in /system/bin, /system/vendor and /system/lib while that of Linux live in /bin, /usr/bin, /lib and /usr/bin. Therefore their linkers are compiled to look in these locations accordingly for the required libraries to link. But the main difference is in their capabilities.

Bionic is a trimmed+modified version of glibc and OpenBSD/FreeBSD C library, small in size and providing a different set of functions than glibc, just enough to get Dalvik working. Bionic was designed keeping in view the space limits, slower CPU's, bloated glibc and open-source license issues. It implements libc, libm (math.h), libdl (dynamic loader) and libpthread (CPU/RAM management for running processes). Android linker has also been designed in accordance with C library, not only different in name and location but also links libraries of different format.

Due to differences in libc and linker, dynamic binaries/libraries compiled for Linux won't always run on Android even if the architecture is same. Static libraries which have all functions (from different libraries) built-in and work independently of dynamic linker, even those have some dependencies on shared libraries at runtime. For instance, functions like dlopen (libdl.so), related to locales and Name Services (getuid, getpwent, getpwnam, getservent, getaddrinfo etc.) require at runtime the shared libraries (such as libnss_compat.so and libnss_files.so etc.) from the same glibc version used at the time of static linking, because paths are hard coded.

Some other differences:

User/group management on Android is different than Linux Name Services (NSS, NSCD, /etc/passwd, /etc/group. . .). Programs like whoami and id, if compiled on Linux, don't work on Android for the same reason.

Android's login process is quite different than on Linux. Bionic doesn't provide glibc's standard utmp/wtmp functionality. But same do other Linux libc's such as musl libc.

Locales (Native Language Support) are not supported by Bionic at all, glibc implements locale through gconv modules

No wide characters support (no iconv.h, use only Unicode in coding)

Limited libpthread (POSIX incompatibility) and libm functionality

Not using Linux syslog, instead uses __libc_android_log_print. /dev/log interface is also no more used on Android.

Domain Name Resolver (DNS) doesn't support Linux Name Service Swith (NSS)

No SysV IPC (managing data sharing between apps)

Bionic additionally provides a large number of System Properties through interface (/dev/__properties__/) using init's property service.

Unlike glibc, Bionic doesn't support C++ fully because it wasn't needed for Dalvik (Java). Android uses llvm libc++ (previuosly using libgnustl/libstdc++ and libstlport) to compensate this shortcoming for programmers. libandroid_support is a bridge to make libc++ compatible with Bionic.

For apps part, Android uses Java instead of native languages (C / C++ etc.). Dalvik VM is a kind of emulator running on top of native libc (and other libraries) to translate Java bytecodes into ARM understandable (native) code. This is how Java works; "write once, run anywhere" (WORA).

Root User and Linux Capabilities

Display Server and Display Manager

While binaries/libraries work at back end, a GUI (graphical user interface) is to facilitate the user at front end i.e. just click on a good looking button and OS will handle the execution of commands at back end, providing you with the required results. It frees the user from the trouble of remembering commands and their syntax.

Linux being developed for PC's comes with a desktop environment backed by a windows server (display server) like most popular X server and Wayland. Display server (protocol) is directly talking to kernel through input and graphics drivers as explained in next section. On the client side, display server is accompanied by a session manager which handles user login/logout, a desktop manager which draws a desktop and a window manager which draws and moves windows on screen. Additionally there can be panels, launchers etc. in a typical desktop environment. It consists of a few hundred packages, like main utilities being settings manager, file manager, power manager, network manager etc. Sound isn't a part of X server and is implemented separately.

Android's display server is SurfaceFlinger which draws surfaces (windows) in a quite different manner than Linux. It offers Java based apps. A launcher is an app equivalent to Desktop, a minimal UI to manage installed apps. File manager and settings are too separate apps. So the differences here too are large. Android's windowing system is totally different than that of Linux. "Mir", an amalgam of X, Wayland and Android, was a proposed display server for Ubuntu but the project is almost dead like "Ubuntu Phone". Wayland is closer to Android, a new project not designed for remote access, eliminating unnecessary parts of X windowing system. It's a compositor consisting of server and window manager as a single unit. It doesn't include input handling either, but implemented through "libinput" separately. However, the differences are large.

Display server and Display manager (login manager) are directly associated to Linux console and virtual terminals. Linux init lands us on a login manager or a fully working virtual console where we can use network related services, install packages, start X server, launch DE/GUI apps etc. Android has almost no concept of using kernel consoles or virtual terminals as discussed above. This is also a major difference limiting the use of display managers and servers on Android.

To conclude, we can't run Linux X server on Android in normal way, because init is different, libc is different and kernel is also different when it comes to display server. Android developers tried their luck to develop an X server for Android, like XServer XSDL. Though functional, these are very limited in functionality, mainly due to app sandboxing mechanism of Android. X server provides a basic security mechanism when started with "-auth" argument to use a cookie file (~/.Xauthority) i.e. using X authority, but Android app doesn't provide this options. Other option is Xvnc server which is X server + vnc server in a single package and unlike other vnc servers, it doesn't need a physical display. It runs X server in headless mode on a virtual display without need of a console connected to actual display. X server directs it output to vnc server which can be accessed from a vnc client (viewer) on localhost or remotely by entering the IP and port/display if phone is connected to network (wi-fi / hotspot).

Graphics, Input and Sound

DRM, evdev and ALSA drivers work in parallel inside Linux kernel.

Graphics:

Framebuffer (fbdev)

Traditional methods in which /dev/(graphics)/fb[N] device is provided by kernel which directly writes to GPU / graphics memory

Kernel is built with framebuffer driver (text console and video mode driver; fbcon), otherwise vesa generic driver can be used

Kernel Mode Settings / Direct Rendering Manager (KMS/DRM)

They live in kernel and exposed to userspace by device /dev/dri/card0

Gives more control over the graphics hardware

Mostly provided by hardware specific drivers as well

Xorg

Most commonly used method

Searches installed drivers for hardware specific drivers (Intel, AMD, Nvidia etc.); open source (mesa) or proprietary. Open source drivers also make use of libdrm for talking to kernel.

If not found, uses vesa generic driver, without acceleration

If not found, uses frambuffer device driver (fbdev)

If not found, uses KMS

xlib or xcb are used by clients for graphics output to X server; either directly or through widget toolkit libraries such as GTK or QT for the actual drawing of buttons, menus etc.

Android supports fbdev somewhat and X server can be configured to use fbdev driver. However, Qualcomm doesn't want you use msm framebuffer freely.

Input:

Hardware specific drivers such as for Synaptic Touchpad can be used but mostly two methods are used:

libinput

A userspace library developed by Wayland, but now implemented by Xorg as well (pointer, touchpad, touchscreen etc.)

evdev

Traditional Linux kernel generic driver (keyboard, mouse through USB, PS/2, PCI etc.) exposes input device through /dev/input/*

Android supports evdev and touchscreen can be used to operate Linux.

Sound:

Advanced Linux Sound Architecture (ALSA) is a set of kernel modules + libraries that exposes sound card status and configuration interface through /proc/asound.

ALSA kernel module (drivers) for sound card is loaded at boot ('cat /proc/asound/modules' or 'lsmod | grep ^snd') which controls device.

Most distros run a sound server PulseAudio which receives audio streams from client libraries ( used by music apps / mic etc.) and directs the output to hardware device (speaker / headphone etc.) through ALSA drivers. PulseAudio client can also forward sound to other PulseAudio servers on network or stream to network audio devices through local PulseAudio server.

Android supports ALSA but devices on Android are quite different than on PC's. There is also difference in ioctl calls both kernel support.

Android kernel is often compiled without LKMs support (CONFIG_MODULES=y).

Proprietary drivers of Android sound cards (msm*sndcardm on Qualcomm devices, 'cat /proc/asound/cards') and their configurations (/usr/share/alsa/cards) are not supported by ALSA on Linux distros.

Therefore sound won't work out of the box and manual configuration is needed to natively enable Linux sound on Android device. Android kernel populates control and PCM sound device interfaces in /dev/snd from /sys/class/sound which can be manipulated to direct ALSA towards Android sound card.

Native support for Linux multimedia (display and sound) can be enabled on running Android device by modifying framebuffer and PCM devices. However, it usually involves disabling functionality of these devices for Android when we are using them for Linux. It's due to the fact that though the base is same, there are differences in implementations.

. . . continued . . .