[Android General] mirfatif:
This post is not an authority on the subject, but just a few reminder notes for me and for those who are interested in grasping things in simple way.

WHAT IS SHELL

Shell is the commandline interpreter i.e. it takes commands from user as text and facilitates the execution of relevant binaries and libraries through kernel to convert commands into system functions.

It provides a platform to find, run, change, remove or install applications and programs in CLI environment just as file managers or windows managers provide in a GUI environment. For dynamic binaries, linker, libc and libdl along with $LD_LIBRARY_PATH and $LD_PRELOAD variables are used by shells to find and execute programs, other than built-in paths within bins and libs. $PATH variable contains a list of directories with possibility of having executable binaries. Shell itself is an executable program with a few built-in commands. We can execute single commands on shell in interactive mode or through a script otherwise.

There are many shells available historically; bash, ksh, zsh, csh, ash, dash, fish and so on, bash being most common, starting from early UNIX days. Shell developed in early days for distant access was rsh (remote shell), later evolved to ssh (secure shell) protocol with added encrypted communication for security, widely used till date.

There can be multiple shells on a device, having different ways of entering commands, redirecting input or output, managing complicated tasks involving running multiple programs simultaneously, etc. Accessing necessary documentation of programs and help files, usually called MANual pages, is also managed by shells.

ANDROID SHELL

Android currently uses mksh, a derivative of KornSHell. mksh is somewhat lightweight than bash and less resources hungry, thus preferable for mobile devices with smaller RAMs. Shell login configuration file is located at (/system)(/vendor)/etc/mkshrc.

~$ echo $0
/system/bin/sh

~$ set | grep VERSION=
KSH_VERSION='@(#)MIRBSD KSH R55 2017/04/12'

~$ file /system/bin/sh
/system/bin/sh: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /system/bin/linker64, stripped

Read here further about shell.

Standard in (stdin), out (stdout) and error (stderr) streams are used by shell to communicate with terminals through File Descriptors 0, 1 and 2 respectively.

WHAT IS TERMINAL

HARDWARE TERMINALS

TeleTYpewriters (TTY) used to be real electro-mechanical devices, you can say a combination of keyboard and printer, sometimes placed miles away to communicate a single character at a time with each other. With the invention of computer in 20th century, these TTY's got use of being connected to a computer through serial cable or a telephone line and modem, for textural input/output. Even interactive commandline interface (CLI) wasn't invented by then and computers ran only in batch mode.

Later, combining video displays with computer, these printing terminals were replaced with "keyboard and glass screen (CRT)" dumb text Terminals or thin clients with no microprocessor inside. With the invention of microprocessor in early 1970's, intelligent or smart terminals started processing user input before sending it to main computer. TTY hardware drivers and more explicitly serial port (UART) or modem drivers were used for connection with computer.

However, computers, being a costly and large-sized item in 1960's, could not be provided to each individual user and many users from their terminals shared the mainframe computer placed in a server room somewhere. UNIX shell was the program that lived on these mainframe computers to handle the input/output from/to terminals. getty program was run on computer to differentiate between those different terminals by authenticating the user through login credentials. Consoles were sophisticated terminals directly connected to mainframe computer with more controls on them, now replaced by separate or built-in keyboard and monitor.

Those initial TTY devices are now obsolete but the nomenclature persists.

VIRTUAL TERMINALS

TTY devices /dev/tty[0-63] in UNIX-like OS's (including Android) still denote Virtual Terminals, emulated inside kernel to behave like some hardware terminal through bidirectional character device files. But in fact, for input/output, they communicate to physical keyboard and monitor/LCD (VGA cards loaded in RAM by BIOS at very early boot stage) provided by kernel using relevant hardware drivers. Since there is no physical terminal connected to serial port (and even there are no serial ports now), terminal hardware part is now inside kernel eliminating serial port connection. With the easy access of microprocessors, race for personal computers (PC) started among hardware vendors and OS developers. Therefore, the need of hardware terminals vanished and even if required, PC's with terminal emulators were being used instead of hardware terminals to connect to mainframe computer.

In short, terminals (real / virtual) are to take input from user (keyboard), communicate to CPU (through different channels) and provide feedback to user on screen (monitor) with less or more processing in between (ioctl).

LINUX CONSOLE

In this phase of virtualization, System Administrator's console has now turned into Linux console. system console (/dev/console) is Linux/Android kernel's "personal" terminal, the default CLI, setup by bootloader before calling kernel, which can be directed to multiple ouput sources configuring "console=" parameter in kernel cmdline. If the parameter is not set otherwise, /dev/console is connected to (or replaced by) some virtual terminal or PTY. Default is the foreground virtual terminal (/dev/tty0), the VGA console (monitor on VGA port + keyboard on USB/PS2 port) set up by BIOS. However if no hardware (like VGA) is available to offer virtual terminals, first serial port device i.e. ttyS0 (analogous of COM1 on Windows) (or UART on embedded devices; ttyMSM on Android) becomes default console. We can manually set console=ttyMSM (on Android) or console=ttyS0 (on Linux) provided that kernel is built with appropriate flags and hardware supports it too. Similarly we can connect to some hardware (or emulated) terminal over USB or parallel serial ports.

When system boots, before some terminal becomes available to Kernel Console, kernel saves it's messages in ring buffer. When init starts in kernel console, it is connected to stdin, stdout and stderr FD's. Now kernel log can be written to syslog / logcat in userspace or accesses by dmesg (read kernel buffer) later on.

However modern systems don't enable kernel console text log output and show a logo, animation or progress bar instead by using an application which takes ownership of /dev/console. Like on Android, '/system/bin/bootanimation' and on Ubuntu, Plymouth app shows boot animation while saving kernel console output to "/var/log/boot.log". Then instead of a CLI console, a GUI (display manager and display server) is loaded directly. Display manager takes user's login credentials and server provides windows environemnt, like Xorg server or Wayland on Linux and SurfaceFlinger on Android.

At the end of init process (the stage when zygote starts on Android), kernel console muxes/multiplexes virtual consoles, each of which connect to some of other /dev/tty* virtual terminals in text mode. This is to allow multiple user logins. In single user mode, only tty0 is available as kernel console for input/output. At a time we can use only one virtual terminal, switching ALT+CTL+F[1-7] on PC. On each virtual terminal, a getty process is run, which in turn runs /bin/login to authenticate a user i.e. make the user owner of that virtual terminal's device file. After authentication, a command shell program is run.

Read here more about ANDROID BOOT PROCESS

CURRENT / CONTROLLING TERMINAL

In an environment of multiple consoles (or multiple GUI terminals), /dev/tty0 (which is usually assigned to /dev/console) is always the currently active virtual terminal i.e. it is connected to VGA card. While /dev/tty is always controlling terminal where process starts i.e. the current console if we are not switching consoles. It ignores all re-directions i.e. following command will always print on the console from where command was issued.

echo TEST >/dev/tty

While the following will always appear on the current console (anyone of /dev/tty[1-7]).

echo TEST >/dev/console
echo TEST >/dev/tty0

We can start a process in background with no controlling terminal.

PSEUDO TERMINALS

As the technology evolved, graphical user interface (GUI) emerged in mid 1980's after Intel designed first 32-bit microprocessor and MS / Apple developed OS's with GUI's. Menu based Window System was developed for easy access by mouse movements, easy for any user instead of memorizing commands. In contrast to text terminals, Graphical Terminals can draw lines, display pictures and change font, size or color of text.

Every app (more precisely, every windows that appears on screen) acts as a client that talks to Display Server using a certian protocol. Initially server lived on each terminal individually, communicating to X display manager and X applications (clients) running on the mainframe computer using XDMCP protocol (TCP/UDP). Display sever is run on a certain display device and every client has a corresponding value for $DISPLAY=host:display.screen ("display" is the sequence number if server has multiple displays, "screen" is the number if there are multiple screens on each display) i.e. server with host name or IP address: "host" listens at TCP port 6000+display from client on mainframe computer. Same concept was transferred to virtual terminals and then to PC's. On PC's communication method is not TCP but a UNIX Domain Socket i.e. on same machine. The GUI we see on a PC screen is also in fact a display server running over a virtual terminal, e.g. tty7 on Ubuntu. Hostname for this display server is localhost i.e. localhost/unix:display.screen or simply :display.screen, X server is listening at UNIX domain socket /tmp/.X11-unix/X[N] where N=display and every window (open applications) is sending its display to this socket through XCB.

Now all GUI applications running inside this desktop environment will be using the same virtual terminal (i.e. talking to the same X server for input and output). This works perfect in a pure GUI environment. But what if we need to run a CLI mode application (like shell) within GUI, which needs a terminal?

In order to have a hardware terminal-like interaction with computer within X window system, contrary to kernel-level terminal emulation, another type of emulation; Pseudo Terminal (PTY) was invented. Programs like GUI terminal emulator applications (e.g. Konsole, Termux etc.), multiplexer applications (e.g. screen) and remote shell applications (e.g. ssh) emulate the terminals in userspace using PTY. Pseudo Terminal MultipleXer device /dev/ptmx creates a pair of PTM (master end, a file descriptor under /proc/<pid>/fd/) and PTS (slave end, a character device file under /dev/pts/). Replacing the actual hardware drivers, PTM now handles the input/output from some software or daemon. Instead of a serial port connection, input/output is communicated with a software program like terminal emulator application which itself handles the output to video display and input from either a hardware keyboard (like on a laptop) or from a virtual keyboard software (like keyboard app on Android phone) through pipes / sockets. PTS interacts with shell as a classical terminal, emulating a real terminal (very rare now) or the already kernel-emulated virtual terminal (the one being already used by GUI session). It takes user input from PTM and writes shell output back to PTM which the users see on application GUI.

So, these terminal emulator apps, when launched, load default shell with them automatically because it's their sole purpose of existence.

Some example cases:

# simple shell running on terminal emulator
terminal emulator <--fd--> /dev/ptmx <--> /dev/pts/[N] <--fd--> process (shell)

# Linux su:
keyboard/touch/display(X) <--socket--> terminal emulator <--fd 56--> /dev/ptmx <--> /dev/pts[N1] <--fd 0,1,2--> su (non-root) | su/bash (root) | processes

# Android su
keyboard/touch/display(SF) <--socket--> terminal emulator <--fd 56--> /dev/ptmx <--> /dev/pts[N1] <--fd 0,1,2--> su (non-root) <--fd 4--> /dev/ptmx <--> /dev/pts/[N2] <--fd 0,1,2--> su/bash (root) | processes

# sshd
terminal emulator <--fd--> /dev/ptmx <--> /dev/pts/[N] <--fd--> ssh <--socket on client--> network <--socket on server--> sshd <--fd--> /dev/ptmx <--> /dev/pts/[N] <--fd--> process

WHAT IS $TERM

This environment variable is defined to announce the capabilities of virtual consoles and terminal emulator applications. Since all the dumb terminals manufactured in early days were different and had to be driven differently by a computer, a database of terminal instructions was created, called terminfo. This database is used depending on the TERM environment variable. These values refer to some hardware device (like vt-100) which were designed with certain specifications or an application (like xterm, the standard terminal emulator for the X Window System), designed to emulate the capabilities of some hardware terminal. linux is the standard terminal for Linux consoles. dumb terminal tells you to expect the worst i.e. just input/output functions and no processing.

So, if you set $TERM manually, interacting applications would be misguided about the capabilities of the terminal emulator they are running in and may crash or behave unexpectedly.

Details of supported terminal emulators can be found under /usr/share/terminfo/ on Linux machines.

ANDROID TERMINAL

Android uses modified Linux kernel, with only one /dev/tty which is the current terminal and the controlling terminal of any running process requiring a terminal. Like many eLinux systems, kernel console is either disabled on Android by default or not connected to any device and it's not muxing any virtual terminals because no text mode user login is intended and only GUI mode is used. However, "Framebuffer Console Support" and "Virtual Terminal Support" can be enabled while compiling Android kernel.

Terminal emulator apps we use on Android device also communicate through PTS files.

~$ lsof /dev/ptmx
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.termux 12032 u0_a122 35u CHR 5,2 0t0 6455 /dev/ptmx
jackpal.androidterm 15457 u0_a89 50u CHR 5,2 0t0 6455 /dev/ptmx

~$ lsof /dev/pts
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sh 15482 u0_a89 0u CHR 136,0 0t0 3 /dev/pts/0
sh 15482 u0_a89 1u CHR 136,0 0t0 3 /dev/pts/0
sh 15482 u0_a89 2u CHR 136,0 0t0 3 /dev/pts/0
bash 15792 u0_a122 0u CHR 136,1 0t376 4 /dev/pts/1
bash 15792 u0_a122 1u CHR 136,1 0t376 4 /dev/pts/1
bash 15792 u0_a122 2u CHR 136,1 0t376 4 /dev/pts/1

Though an app com.android.terminal has been part of AOSP, we don't find a built-in terminal application on stock ROM's. Users aren't expected to use CLI on Android directly but through adb tool on PC, if needed. Since there is no terminal, there is no value set for TERM. Terminal application we install on our device bring their own values of $TERM depending on their capabilities. /etc/mkshrc had TERM=vt100 in old releases of AOSP but not in new ones.

The most commonly used Jack's Terminal Emulator which uses Android mksh shell shows screen terminal type.

~$ ps -p $(ps -p $$ -o ppid=) -o args=
jackpal.androidterm

~$ set | grep -E "TERM=|VERSION="
KSH_VERSION='@(#)MIRBSD KSH R55 2017/04/12'
TERM=screen

~$ type type; # whence is ksh built-in command
type is an alias for '\\builtin whence -v'

The newer Termux app that comes with a Linux environment and bundled with dash and bash has xterm-256color default value for $TERM variable.

~$ ps -p $(ps -p $$ -o ppid=) -o args=
com.termux

~$ set | grep -E "TERM=|VERSION="
BASH_VERSION='4.4.18(1)-release'
TERM=xterm-256color

~$ type type; # type is bash built-in command
type is a shell builtin

Termux also has a few terminals' terminfo under $PREFIX/share/terminfo/.

ADB shell shows simply dumb terminal as set in adb shell_service.cpp.

To summarize, TERM should not be set by user, but by terminal emulator itself. You may read here further about terminal emulators.

WHERE IS WINDOWS' TERMINAL?

Contrary to "always-been" multi-user UNIX OS, MS DOS (Microsoft Disk Operating System) was designed for the floppy-based single user IBM PC in early 1980's and COMMAND.COM was its command-line shell program. MS DOS was purchased and modified from 86DOS which was a clone of very first commercial CP/M OS designed for Intel's 8-bit / 16-bit microprocessors in 1970's. Only the multi-user aspect of CP/M was ignored in DOS in favor of MS Xenix, a Unix OS. A GUI program called Windows was added to DOS in 1986 in response to Apples's MacOS. After ending partnership with IBM in 1990 before OS/2 was released, Windows became an independent GUI OS in 1993 with the release of 32-bit Windows NT (MS's name of OS/2) which contained only emulated DOS, that too dropped in 64-bit versions. Ever since, Windows has been a GUI-focused OS, so it never needed to introduce terminal concept. Windows 95, 98 and ME included DOS just as an additional part, though independently bootable. Later with the release of Windows XP (2001), DOS was merged into Windows, losing its originality.

Windows calls its explorer.exe (desktop program) a graphical shell and provides supplementary cmd.exe, just to name a text mode shell, slightly better than COMMAND.COM, but with no concept of remote access like UNIX/Linux terminals, just a local commandline shell. Single user thing always kept Windows away from concepts like terminal emulator and ssh, compensated somehow by puTTY and other third party solutions. MS also realized the lack of native support for these UNIX concepts and tried to empower the users by providing a better PowerShell in Vista, though not comparable to Linux shell. A limited but expected to grow bash shell (as an application) is included in Windows 10 which enables it to host Windows Subsystem for Linux (WSL), a downloadable CLI Linux environment within Windows GUI. Windows now owns cmd.exe, PowerShell and bash, but not a Linux-like pure terminal+shell environment.