Creating a background process on RPi / Linux - Page 1
Forum section: electronics / general

author: John B
author_rank: Frequent Contributor
timestamp: January 06, 2025, 08:36:30 pm
content: I've been having a lot more difficulty with this than I thought would be the case. Basically I would like a python script to start when rebooting the RPi. More specifically I need a few bash commands to run, ie running the python script in a virtual environment. The script also has stdin/stdout/stderr functionality, so I've been manually running it within a TTY.The first idea is using crontab to create a startup event where it will run a few bash commands on boot. However I can't seem to figure whether it's possible to run a command in one TTY or desktop, and have that command run in another TTY as if it had been run in that TTY. I can manage to print stdout from one TTY to another one, but that doesn't give the same stdin/stderr functions and doesn't take exclusive control over the TTY process. So to clarify, is there a way to run a bash command in another TTY (assuming things like it's already logged in)?The second idea is to create a systemd process, however I will run into a similar issue, though I think I can attach a TTY to a systemd process.

author: abeyer
author_rank: Frequent Contributor
timestamp: January 06, 2025, 11:44:38 pm
content: I'm a little confused as what you're trying to do, as what you're saying sounds like the exact opposite of normal background process behaviour on linux: normally part of your daemonizing steps if you do that yourself is to explicitly detach yourself from a tty if you have one. (the NOTTY ioctl)The systemd setup is probably the right direction... but again, expecting a tty is not the normal approach, but rather something that is intended to run the in background should handle whatever it's given as stdin/stout/stderr and let systemd handle that.

author: abeyer
author_rank: Frequent Contributor
timestamp: January 06, 2025, 11:48:49 pm
content: also, as to your specific question (even though I suspect it's not the right approach) you might look athttps://github.com/nelhage/reptyr

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 07, 2025, 02:11:25 am
content: If you want to run an interactive service that you can "connect" to, install thescreenpackage, so you can usescreento create a new session and run your script in detached mode, with something likesetpriv --reuiduser--init-groups --reset-env screen -d -m -Ssessionnamepython3/path/to/script.pyin your init script. This command starts the script as a separate process as useruser, and returns immediately.Afterwards, userusercan runscreen -rsessionnameto reconnect to the script, from any terminal; even from an SSH connection. To detach from the script, press Ctrl+A then Ctrl+D.screenand other similar utilities work by creatingpseudoterminalswhen needed. These are character device pairs, a slave (which acts basically indistinguishably from a TTY device), and a master (that controls the input and output to the slave). In your particular case, Linuxptssupport, also known as UNIX 98 pseudoterminal support, will be used and the character devices reside under/dev/pts/.You can also run a script or program using a specificconsole terminalusingopenvtas root:openvt -c8-- python3/path/to/script.pyoropenvt -c8-- setpriv --reuiduser--init-groups --reset-env python3/path/to/script.pyThis starts the Python script connected to/dev/tty8as root (first) or useruser(second), but without changing the ownership of the tty device. It will fail if the tty is already in use. These too will start the script in a separate process, and return immediately, without waiting for the script to complete.You can switch the active terminal using e.g. Ctrl+Alt+F8, or runningchvt 8, but only from the console terminal, and not when connecting using e.g. SSH.This kind of stuff is normally done only when you replace the entireinitsystem, though; with say passwordless login shells in a couple of TTYs, but one or more primary ones providing some kind of a text-based full-screen interface (TUI), so that the user can switch between them using Ctrl+Alt+Fnor Ctrl+Alt+◀ and Ctrl+Alt+▶, but only from the local console terminal, not over SSH etc.

author: John B
author_rank: Frequent Contributor
timestamp: January 07, 2025, 02:38:30 am
content: Thanks, I guess describing it as a background process is a little confusing. Put another way, when I start the RPi, I'll manually enterbash ./startscript.shinto tty1. The sh script then switches to the virtual environment and starts the python script with parameters. I would just like to automate that so it happens on reboot.The script is a serial/i2c/mqtt bridge, and is supposed to run all the time, weeks or months at a time. I think of it in terms of background processes, as for a while RPi has had a bug where going into screen power saving mode could crash the display server, and therefore kill any processes in that desktop environment. The ttys seemed to be unaffected by that crashing. It may be fixed now with RPi moving to a wayland based system, but separating the script from the usual desktop environment seems like a good idea.I will give openvt another try. I think I tried that but maybe I was unsuccessful in trying to reference an existing terminal

author: John B
author_rank: Frequent Contributor
timestamp: January 07, 2025, 03:17:29 am
content: The openvt was a success, I was able to start the process on tty8.However most of the parameters that you mentioned don't work, at least on the RPi version. I used:sudo openvt -c 8 commandI've still got a lot to learn about the structure and terminology. I guess an "in use" tty means a login is attached to that tty?

author: Nominal Animal
author_rank: Super Contributor
timestamp: January 07, 2025, 03:41:29 am
content: When you start a process from the command line, the process inherits thecontrolling terminalfrom your shell. When the controlling terminal is closed, all processes having that terminal as their controlling terminal receives aHUPsignal; it is this signal that kills lingering processes. To execute a Python script as the current user, but detaching from the terminal, you can use( setsid -f python3/path/to/script) </dev/null 1>/dev/null 2>/dev/nullin a Bash script to redirect standard input, output, and error to/dev/nulldiscarding them, fork a new process group, and start running the Python script as the process group leader (process group ID being the same as the process ID). It will not have a controlling terminal, so will not be killed by a HUP signal. The command will return immediately, and not wait for the script to complete/exit.The difference to the methods in my previous post is that there is no way to provide standard input to the script, or read the standard output or error from the script. If you use Pythonsyslogmodule, you can usesyslog.openlog("name")exactly once to set the script/service name, andsyslog.syslog("message")to log messages to the system log. In Debian derivatives these should appear in/var/log/system.log, unless you redirect them to their own log files; and are logged via systemd-journald, syslogd, rsyslogd, or a similar service.Quote from: John B on January 07, 2025, 03:17:29 amsetprivcommand is part of util-linux, intended for dropping privileges when switching from root to a less privileged user.Quote from: John B on January 07, 2025, 03:17:29 amMore generally, thatioctl(ttyfd, TIOCSCTTY, 0)succeeds, meaning it is not already the controlling terminal of some process. This includes logins and processes started using openvt.

author: abeyer
author_rank: Frequent Contributor
timestamp: January 07, 2025, 04:25:46 am
content: Quote from: John B on January 07, 2025, 02:38:30 amI still think it sounds from what you've said that you're neither trying to avoid/replace systemd nor actually relying on needing a tty so would argue that tieing your script arbitrarily to some vt is not the right approach. That's kind of a kludge, itwillwork, it's sometimes necessary to do something like that... but I would say that a simple systemd service definition would give you the behavior you've described in a much more native, understandable, and reproducible way.I think something like this installed as a system service would do what you want, while also giving you systemd's service control tools, logging, automatic restart on failures, etc...Code:Description=serial/i2c/mqtt bridgeRestart=on-failureRestartSec=5StartLimitIntervalSec=300StartLimitBurst=3Type=simpleExecStart=/bin/bash /path/to/startscript.shWantedBy=multi-user.targetAlso... if you'reonlyusing bash to activate a venv, but otherwise running everything in python, it's usually fine to skip activation and directly call thepythoninterpreter in the venv. (the common exception would be if your code or dependencies need to be able to shell out to something else installed in the venv'sbindirectory.)eg you could change the above to something likeCode:ExecStart=/path/to/venv/bin/python /path/to/startscript.py ...

author: JohanH
author_rank: Frequent Contributor
timestamp: January 07, 2025, 08:06:48 am
content: Agree on the systemd approach. I use it for everything nowadays.You can also run services as user, which is handy if it doesn't need root access and only needs to run in the user's environment. Here's a simple guide that I found through a google search:https://www.baeldung.com/linux/systemd-create-user-servicesEdit. When running services as a user, I do prefer putting everything under the user's home. This article shows how it's done:https://wiki.archlinux.org/title/Systemd/User

author: DiTBho
author_rank: Super Contributor
timestamp: January 07, 2025, 12:44:10 pm
content: Quote from: abeyer on January 06, 2025, 11:48:49 pmapp-misc/reptyris somewhat likeapp-misc/screen.Very useful, in a different way, for "oops, I didn't start the program under Screen".Unfortunately not present in 2009 and 2010 (gentoo) steps. Recently added, v0.10.0.I created a backwards compatibility ebuild, EABI-v1 overlay for 2009 stagesIt's now on my rb532a router

author: DiTBho
author_rank: Super Contributor
timestamp: January 07, 2025, 01:22:23 pm
content: Code:sys-apps/util-linux caps cramfs hardlink logger ncurses nls pam readline su suid unicode -audit -build -cryptsetup -fdformat -kill -magic -python -rtas -selinux -slang -static-libs -systemd -test -tty-helpers -udev -verify-sig#+cramfs build mkfs/fsck helpers for cramfs filesystems#+hardlink build hardlink program#+logger build the logger program#+readline Enable support for libreadline, a GNU line-editing library that almost everyone wants#+su build the su program#+suid Install some programs with suid bit set to provide additional functionality.# mount/umount: non-root users may mount/umount devices wall/write# non-root users can notify other users su: non-root users may become root#audit Use sys-process/audit to emit audit messages about system changes#build !!internal use only!! DO NOT SET THIS FLAG YOURSELF!# used for creating build images and the first half of bootstrapping [make stage1]#caps build setpriv helper run programs with diff capabilities#cryptsetup Use sys-fs/cryptsetup to have built-in dm-verity in libmount#fdformat build fdformat floppy disk format#kill build the kill program#magic Add support for file type detection via magic bytes usually via libmagic from sys-apps/file#ncurses Add ncurses support console display library#nls Add Native Language Support using gettext - GNU locale utilities#pam build runuser helper#python Add optional support/bindings for the Python language#rtas Add support for the Run Time Abstraction Services RTAS#selinux !!internal use only!!# Security Enhanced Linux support, this must be set by the selinux profile or breakage will occur#slang Add support for the slang text display library it's like ncurses, but different#static-libs Build static versions of dynamic libraries as well#systemd Enable use of systemd-specific libraries and features like socket activation or session tracking#test Enable dependencies and/or preparations necessary to run tests# usually controlled by FEATURES=test but can be toggled independently#tty-helpers install the mesg/wall/write tools for talking to local users#udev Enable virtual/udev integration device discovery, power and storage device support, etc#unicode Add support for Unicode#uuidd build uuidd daemon#verify-sig Verify upstream signatures on distfilesGentoo,sys-apps/util-linux: setpriv needs "caps" use flags, disabled by default

author: pcprogrammer
author_rank: Super Contributor
timestamp: January 07, 2025, 02:47:05 pm
content: Quote from: John B on January 07, 2025, 02:38:30 amIf it is just about running that shell script on startup, there is the init.d directory, at least if that still is a thing under the latest of the Linux setups. I have used this method to start my programs that needed to run for my central heating system. That was based on minibian from 2015 and in the init.d directory I placed shell scripts to allow starting and stopping of my programs.Might be that there is a bit more to it than just placing your script in this directory, but below is a copy of the script I used for the main processing program I wrote.Code:#!/bin/sh### BEGIN INIT INFO# Provides: centralheatingprocessing# Required-Start: $remote_fs $syslog mysql owt2db db2iic# Required-Stop: $remote_fs $syslog mysql# Default-Start: 2 3 4 5# Default-Stop: 0 1 6# Short-Description: central heating control process# Description: Start and stop the program needed for handling the central heating control processing.### END INIT INFOname="centralheatingprocessing"pid_file="/var/run/$name.pid"stdout_log="/var/log/$name.log"stderr_log="/var/log/$name.err"dir="/root/CentralHeatingPrograms/"cmd=$nameget_pid() {cat "$pid_file"}is_running() {[ -f "$pid_file" ] && ps `get_pid` > /dev/null 2>&1}case "$1" instart)if is_running; thenecho "Already started"elseecho "Waiting 10 seconds"sleep 10echo "Starting $name""$dir$cmd" >> "$stdout_log" 2>> "$stderr_log" &echo $! > "$pid_file"if ! is_running; thenecho "Unable to start, see $stdout_log and $stderr_log"exit 1fifi;;startd)if is_running; thenecho "Already started"elseecho "Starting $name""$dir$cmd" >> "$stdout_log" 2>> "$stderr_log" &echo $! > "$pid_file"if ! is_running; thenecho "Unable to start, see $stdout_log and $stderr_log"exit 1fifi;;stop)if is_running; thenecho -n "Stopping $name.."kill -9 `get_pid`for i in {1..10}doif ! is_running; thenbreakfiecho -n "."sleep 1doneechoif is_running; thenecho "Not stopped; may still be shutting down or shutdown may have failed"exit 1elseecho "Stopped"if [ -f "$pid_file" ]; thenrm "$pid_file"fifielseecho "Not running"fi;;restart)$0 stopif is_running; thenecho "Unable to stop, will not attempt to start"exit 1fi$0 start;;status)if is_running; thenecho "Running"elseecho "Stopped"exit 1fi;;*)echo "Usage: $0 {start|stop|restart|status}"exit 1;;esacexit 0Found some info about this herehttps://www.elithecomputerguy.com/2019/09/linux-start-stop-restart-services-systemd-systemctl-service-init-d/or herehttps://superuser.com/questions/1087983/init-d-how-to-start-a-service-in-init-d-when-networking-fully-worksLooking through that it seems to be old but might still work and maybe easier then systemd.

author: abeyer
author_rank: Frequent Contributor
timestamp: January 07, 2025, 08:19:59 pm
content: Quote from: pcprogrammer on January 07, 2025, 02:47:05 pmStrong disagree there. IMHO, there are a great many things Idon'tlike or agree with about systemd... but never having to manually write an initd script again is one thing that I won't miss. There is no good reason for every single service to have to have that script where it rerolls its own solution to start/stop/restart, pid files, file handles, etc...If you're a solid bash user, the script might feel slightly "easier", but I posit that's merely familiarity and stops being true as soon as you get a bit of experience writing unit files.

author: pcprogrammer
author_rank: Super Contributor
timestamp: January 08, 2025, 07:18:31 am
content: Quote from: abeyer on January 07, 2025, 08:19:59 pmThat is why I wrote "maybe easier".I have not used systemd and maybe never will. I only once used Linux to build an embedded control system and that was in 2016. The need for auto starting something from myself on my desktop Linux systems has not yet arisen.

author: John B
author_rank: Frequent Contributor
timestamp: January 09, 2025, 09:43:47 pm
content: After a bit of extra reading, I've successfully created a systemd unit. It seems to be the most modern approach, was quite easy and is the winner of this contest.I've attached standard in/out/err to tty8, which was also quite easy. I thought there might be some issues with the python script starting via a bash script, but the systemd unit seems to list and handle child processes connecting to a tty just fine.

author: radiolistener
author_rank: Super Contributor
timestamp: January 13, 2025, 05:10:55 am
content: I'm not entirely sure what you're aiming to achieve. Are you looking to run a console application in the background while being able to access its TTY at any time from any terminal, virtual terminal, or SSH session?If that's the case, I believe tmux is exactly what you need. tmux is a powerful terminal multiplexer that allows you to run console applications in the background while maintaining full access to their input, output, and error streams. You can attach to the session at any time from any terminal or SSH session, view all the output generated while the application was running in the background, and provide input just like in a regular terminal. Additionally, you can detach from the session without interrupting the application’s execution.tmux also offers advanced features such as managing multiple sessions, displaying several terminal windows on the screen simultaneously, switching between sessions, creating new ones, and closing them as needed.Here’s how you can get started:1) Start a tmux session, simply run:Code:tmuxInside the session, start your script or application as you normally would.2) Detach from the session. To detach from the session while keeping it running in the background, press Ctrl+B, then D.3) Reattach to the session. From any terminal or SSH session, you can reconnect to the tmux session using:Code:tmux a -t 0Here, 0 is the default session name if you started tmux without arguments. If you have multiple sessions, you can list them with:Code:tmux ls4) Automate the script with tmux on boot. To run your script automatically on startup, you can create a simple shell script and configure it to start tmux with your application:Code:tmux new-session -d -s TEST 'path/to/your/script'then you can connect to this session from any terminal with:Code:tmux a -t TESTWith tmux, you get the flexibility to manage your application as if it were running in a dedicated TTY, while still being able to access and control it from anywhere. This approach is robust and convenient, especially for remote management.I like tmux, it's small lightweight and allows you to run console apps in background the same as if you run it on dedicated TTY.It also allows you to connect to your console app from local terminal and from remote ssh simultaneously.