Repository: static-web-server/static-web-server (language: rust)

Issue:
socket activation (accept tcp listener file descriptor instead of binding to a tcp port)
Firstly, thanks for writing static-web-server - it's something I've been wanting for a while!

Whilst looking at #35 I investigated how easy it would be to offer a socket activation model, as an alternative to binding to TCP port manually.

In this context it would mean that the parent process of static-web-server (e.g. inetd, systemd, or possibly Apple launchd) would open a TCP socket listener, and then pass the file descriptor for this listener to static-web-server.  In it's simplest form (and the model use by inetd), static-web-server would inherit the file descriptor for the TCP listener as it's standard input file descriptor (FD 0).  There is a Rust crate called `listenfd` designed to handle this situation (making the passed-in FD available as an `io::Result<Option<TcpListener>>`).

In the context of #35, this would allow:

- Static-web-server to bind to a port number below 1024 whilst not running as root (I know there are other methods to do this e.g. using firewalls and proxies, but this is probably the lowest overhead one).
- Restarting of the static-web-server without service interruption (e.g. for configuration change, or on SSL certificate renewal etc.) via https://www.freedesktop.org/software/systemd/man/systemd.socket.html#FlushPending=
- The use of various advanced socket binding options supported by systemd (e.g. `FreeBind`) without adding more code to static-web-server or warp to handle these.
- Further sandboxing of static-web-server from the host's network e.g. ref https://www.freedesktop.org/software/systemd/man/systemd.socket.html :

> All network sockets allocated through .socket units are allocated in the host's network namespace (see network_namespaces(7)). This does not mean however that the service activated by a configured socket unit has to be part of the host's network namespace as well. It is supported and even good practice to run services in their own network namespace (for example through PrivateNetwork=, see systemd.exec(5)), receiving only the sockets configured through socket-activation from the host's namespace. In such a set-up communication within the host's network namespace is only permitted through the activation sockets passed in while all sockets allocated from the service code itself will be associated with the service's own namespace, and thus possibly subject to a much more restrictive configuration.

Unfortunately using this model with Warp isn't as straightforward as I'd hoped, although it can be done via hyper as shown in this example:

https://github.com/rusty-crab/warp-api-starter-template/blob/833ca24994bd0a8e197332aee7df24eb687c84d1/src/main.rs#L172

(this uses the listenfd crate: https://docs.rs/listenfd/0.3.3/listenfd/index.html ).

Because static-web-server is already using a forked version of warp (and I'm not at all familiar with hyper and warp), I thought it best to just note this investigation at this point, rather than ploughing in and adding this feature to static-web-server.

Any thoughts?


Discussion / hints:
> In the context of #35, this would allow:
> 
> * Static-web-server to bind to a port number below 1024 whilst not running as root (I know there are other methods to do this e.g. using firewalls and proxies, but this is probably the lowest overhead one).
> * Restarting of the static-web-server without service interruption (e.g. for configuration change, or on SSL certificate renewal etc.) via https://www.freedesktop.org/software/systemd/man/systemd.socket.html#FlushPending=
> * The use of various advanced socket binding options supported by systemd (e.g. `FreeBind`) without adding more code to static-web-server or warp to handle these.
> * Further sandboxing of static-web-server from the host's network e.g. ref https://www.freedesktop.org/software/systemd/man/systemd.socket.html :
> 
> > All network sockets allocated through .socket units are allocated in the host's network namespace (see network_namespaces(7)). This does not mean however that the service activated by a configured socket unit has to be part of the host's network namespace as well. It is supported and even good practice to run services in their own network namespace (for example through PrivateNetwork=, see systemd.exec(5)), receiving only the sockets configured through socket-activation from the host's namespace. In such a set-up communication within the host's network namespace is only permitted through the activation sockets passed in while all sockets allocated from the service code itself will be associated with the service's own namespace, and thus possibly subject to a much more restrictive configuration.

As being me not a systemd confident dev, I see socket activation feature in general as an interesting alternative.
I also saw the advantages described such as significant improvements specially for running non-on-demand or non-Internet services (please correct me if I'm wrong ?) along with the ability of rootless by default.

So why not. 
Regarding that I would like to highlight one thing. That possible feature "could" also involve to have to provide some .deb and .rpm packages in near future for Linux users (systemd) and maybe for Mac users as well similarly (launchd). But of course this would come later. What do you think?

> Unfortunately using this model with Warp isn't as straightforward as I'd hoped, although it can be done via hyper as shown in this example:
> 
> https://github.com/rusty-crab/warp-api-starter-template/blob/833ca24994bd0a8e197332aee7df24eb687c84d1/src/main.rs#L172
> 
> (this uses the listenfd crate: https://docs.rs/listenfd/0.3.3/listenfd/index.html ).

I'm also no aware about if Warp provides that kind of support (or perhaps [could this kind of help us?](https://github.com/seanmonstar/warp/blob/master/examples/unix_socket.rs#L13)) however since Warp is built upon Hyper I think would not be impossible.
BTW I saw that warp-api-starter is using systemfd + listenfd to provide debug mode with auto-reload feature. Is that correct? 

> Because static-web-server is already using a forked version of warp (and I'm not at all familiar with hyper and warp), I thought it best to just note this investigation at this point, rather than ploughing in and adding this feature to static-web-server.

Yes, static-web-server is using a synced fork of latest Warp but basically due to few features needed by this server that I have introduced to Warp's built-in filters only, concretely on headers, compression and dir ones (you can see them under [Jan 20, 2021 commits](https://github.com/joseluisq/warp/commits/master) if you are interested).
Honestly I have no plans to maintain those changes for long-time as a fork but since the server v2 is under beta so it means we can still continue hacking and of course it's not a blocking condition for future additions/improvements to this server.

--

Summing up, we can try to play around with listenfd/other along with Warp in order to see how could be addressed, from my side I will try to give a test. So feel free to let me know if there is progress about it. Thanks.

I think it would be quite straightforward to implement the same design as the one referred to in the warp-api-starter-template above.  I think socket activation would be a useful feature for `static-web-server` even if it's not shipped as an OS package, but socket activation would probably be the logical way to package `static-web-server` (to allow maximum security in the default setup).  I haven't created a Debian package for a Rust binary before, but I've written some Rust, and done some Debian (and rpm) packaging, so I'd be happy to look at putting together a Debian package if it's implemented.
Great! in that case let's try it out. Just pinging me if you need review.
Just to let it know I was re-thinking the idea of Warp adoption for `static-web-server` project so [here](https://github.com/joseluisq/static-web-server/issues/37) the result.
OK thanks.  I think it's probably better to wait on #37 (whatever eventual outcome), and then decide how to implement this feature afterwards.
#38 is finally merged so it leaves the way clear to address this feature issue.
How would you like to divide the work on this?  I'm happy to have a go if you like?  It may be a simple revision...
Feel free to start when you want from whatever place that you feel is suitable.
I think you also want to have a look at [server.rs](https://github.com/joseluisq/static-web-server/blob/master/src/server.rs).

From my side I will support with review and changes if they are needed.
Since I'm focusing on other pending features like cors and directory listing. BTW those doesn't interfere with this feature.
OK, sounds good.  Didn't want to duplicate effort if you were already looking at this.
I have a maximally hacky (minimal effort) proof of concept built (and serving successfully), so I'll try and find some time to finish it up and create a PR.

Fix patch:
diff --git a/Cargo.lock b/Cargo.lock
index 1e509705..94ff9166 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -484,6 +484,17 @@ version = "0.2.95"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "789da6d93f1b866ffe175afc5322a4d76c038605a1c3319bb57b06967ca98a36"
 
+[[package]]
+name = "listenfd"
+version = "0.3.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "492158e732f2e2de81c592f0a2427e57e12cd3d59877378fe7af624b6bbe0ca1"
+dependencies = [
+ "libc",
+ "uuid",
+ "winapi",
+]
+
 [[package]]
 name = "log"
 version = "0.4.14"
@@ -858,6 +869,7 @@ dependencies = [
  "humansize",
  "hyper",
  "jemallocator",
+ "listenfd",
  "mime_guess",
  "nix",
  "num_cpus",
@@ -1116,6 +1128,15 @@ version = "0.7.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "a156c684c91ea7d62626509bce3cb4e1d9ed5c4d978f7b4352658f96a4c26b4a"
 
+[[package]]
+name = "uuid"
+version = "0.6.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e1436e58182935dcd9ce0add9ea0b558e8a87befe01c1a301e6020aeb0876363"
+dependencies = [
+ "cfg-if 0.1.10",
+]
+
 [[package]]
 name = "version_check"
 version = "0.9.3"
diff --git a/Cargo.toml b/Cargo.toml
index b3dc9f02..a51c49b1 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -45,6 +45,7 @@ pin-project = "1.0"
 tokio-rustls = { version = "0.22" }
 humansize = "1.1"
 time = "0.1"
+listenfd = "0.3.3"
 
 [target.'cfg(not(windows))'.dependencies.nix]
 version = "0.14"
diff --git a/README.md b/README.md
index 5876c540..0287a267 100644
--- a/README.md
+++ b/README.md
@@ -27,6 +27,7 @@
 - Configurable using CLI arguments or environment variables.
 - First-class [Docker](https://docs.docker.com/get-started/overview/) support. [Scratch](https://hub.docker.com/_/scratch) and latest [Alpine Linux](https://hub.docker.com/_/alpine) Docker images available.
 - MacOs binary support thanks to [Rust Linux / Darwin Builder](https://github.com/joseluisq/rust-linux-darwin-builder).
+- The ability to accept a socket listener as a file descriptor for use in sandboxing and on-demand applications.
 
 ## Releases
 
@@ -45,6 +46,7 @@ Server can be configured either via environment variables or their equivalent co
 | --- | --- | --- |
 | `SERVER_HOST` | Host address (E.g 127.0.0.1). | Default `[::]`. |
 | `SERVER_PORT` | Host port. | Default `80`. |
+| `SERVER_LISTEN_FD` | Optional file descriptor number (e.g. `0`) to inherit an already-opened TCP listener on (instead of using `SERVER_HOST` and/or `SERVER_PORT` ). |
 | `SERVER_ROOT` | Root directory path of static | Default `./public`. |
 | `SERVER_LOG_LEVEL`          | Specify a logging level in lower case. (Values `error`, `warn`, `info`, `debug`, `trace`). | Default `error` |
 | `SERVER_ERROR_PAGE_404`     | HTML file path for 404 errors. | If path is not specified or simply don't exists then server will use a generic HTML error message. Default `./public/404.html`. |
@@ -82,6 +84,12 @@ OPTIONS:
     -z, --directory-listing <directory-listing>
             Enable directory listing for all requests ending with the slash character (‘/’) [env:
             SERVER_DIRECTORY_LISTING=]  [default: false]
+    -f, --fd <fd>
+            Instead of binding to a TCP port, accept incoming connections to an already-bound TCP socket listener on the
+            specified file descriptor number (usually zero). Requires that the parent process (e.g. inetd, launchd, or
+            systemd) binds an address and port on behalf of static-web-server, before arranging for the resulting file
+            descriptor to be inherited by static-web-server. Cannot be used in conjunction with the port and host
+            arguments [env: SERVER_LISTEN_FD=]
     -a, --host <host>
             Host address (E.g 127.0.0.1 or ::1) [env: SERVER_HOST=]  [default: ::]
 
@@ -115,6 +123,16 @@ OPTIONS:
             [default: 1]
 ```
 
+## Use of file descriptor socket passing
+
+Example `systemd` unit files for socket activation are included in the [`systemd/`](systemd/) directory.  If
+using `inetd`, its "`wait`" option should be used in conjunction with static-web-server's `--fd 0`
+option.
+
+Alternatively, the light-weight [`systemfd`](https://github.com/mitsuhiko/systemfd) utility may be
+useful - especially for testing e.g.
+`systemfd --no-pid -s http::8091 -- path/to/static-web-server --fd 0`
+
 ## Docker stack
 
 Example using [Traefik Proxy](https://traefik.io/):
diff --git a/src/config.rs b/src/config.rs
index f77f09e7..5966f1a2 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -11,6 +11,21 @@ pub struct Config {
     /// Host port
     pub port: u16,
 
+    #[structopt(
+        long,
+        short = "f",
+        env = "SERVER_LISTEN_FD",
+        conflicts_with_all(&["host", "port"])
+    )]
+    /// Instead of binding to a TCP port, accept incoming connections to an already-bound TCP
+    /// socket listener on the specified file descriptor number (usually zero). Requires that the
+    /// parent process (e.g. inetd, launchd, or systemd) binds an address and port on behalf of
+    /// static-web-server, before arranging for the resulting file descriptor to be inherited by
+    /// static-web-server. Cannot be used in conjunction with the port and host arguments. The
+    /// included systemd unit file utilises this feature to increase security by allowing the
+    /// static-web-server to be sandboxed more completely.
+    pub fd: Option<usize>,
+
     #[structopt(
         long,
         short = "n",
diff --git a/src/logger.rs b/src/logger.rs
index b2b1f753..b2e62715 100644
--- a/src/logger.rs
+++ b/src/logger.rs
@@ -1,3 +1,4 @@
+use std::io;
 use tracing::Level;
 use tracing_subscriber::fmt::format::FmtSpan;
 
@@ -7,6 +8,7 @@ use crate::Result;
 pub fn init(level: &str) -> Result {
     let level = level.parse::<Level>()?;
     match tracing_subscriber::fmt()
+        .with_writer(io::stderr)
         .with_max_level(level)
         .with_span_events(FmtSpan::CLOSE)
         .try_init()
diff --git a/src/server.rs b/src/server.rs
index 4751a7ea..97637a11 100644
--- a/src/server.rs
+++ b/src/server.rs
@@ -1,7 +1,8 @@
 use hyper::server::conn::AddrIncoming;
 use hyper::server::Server as HyperServer;
 use hyper::service::{make_service_fn, service_fn};
-use std::net::{IpAddr, SocketAddr};
+use listenfd::ListenFd;
+use std::net::{IpAddr, SocketAddr, TcpListener};
 use std::sync::Arc;
 use structopt::StructOpt;
 
@@ -58,8 +59,26 @@ impl Server {
 
         tracing::info!("runtime worker threads: {}", self.threads);
 
-        let ip = opts.host.parse::<IpAddr>()?;
-        let addr = SocketAddr::from((ip, opts.port));
+        let (tcplistener, addr_string);
+        match opts.fd {
+            Some(fd) => {
+                addr_string = format!("@FD({})", fd);
+                tcplistener = ListenFd::from_env()
+                    .take_tcp_listener(fd)?
+                    .expect("Failed to convert inherited FD into a a TCP listener");
+                tracing::info!(
+                    "Converted inherited file descriptor {} to a TCP listener",
+                    fd
+                );
+            }
+            None => {
+                let ip = opts.host.parse::<IpAddr>()?;
+                let addr = SocketAddr::from((ip, opts.port));
+                tcplistener = TcpListener::bind(addr)?;
+                addr_string = format!("{:?}", addr);
+                tracing::info!("Bound to TCP socket {}", addr_string);
+            }
+        }
 
         // Check for a valid root directory
         let root_dir = helpers::get_valid_dirpath(&opts.root)?;
@@ -111,7 +130,12 @@ impl Server {
                     }
                 });
 
-                let mut incoming = AddrIncoming::bind(&addr)?;
+                tcplistener
+                    .set_nonblocking(true)
+                    .expect("Cannot set non-blocking");
+                let listener = tokio::net::TcpListener::from_std(tcplistener)
+                    .expect("Failed to create tokio::net::TcpListener");
+                let mut incoming = AddrIncoming::from_listener(listener)?;
                 incoming.set_nodelay(true);
 
                 let tls = TlsConfigBuilder::new()
@@ -124,9 +148,9 @@ impl Server {
                     HyperServer::builder(TlsAcceptor::new(tls, incoming)).serve(make_service);
 
                 tracing::info!(
-                    parent: tracing::info_span!("Server::start_server", ?addr, ?threads),
+                    parent: tracing::info_span!("Server::start_server", ?addr_string, ?threads),
                     "listening on https://{}",
-                    addr
+                    addr_string
                 );
 
                 server.await
@@ -153,14 +177,15 @@ impl Server {
                     }
                 });
 
-                let server = HyperServer::bind(&addr)
+                let server = HyperServer::from_tcp(tcplistener)
+                    .unwrap()
                     .tcp_nodelay(true)
                     .serve(make_service);
 
                 tracing::info!(
-                    parent: tracing::info_span!("Server::start_server", ?addr, ?threads),
+                    parent: tracing::info_span!("Server::start_server", ?addr_string, ?threads),
                     "listening on http://{}",
-                    addr
+                    addr_string
                 );
 
                 server.await
diff --git a/systemd/etc_default_static-web-server b/systemd/etc_default_static-web-server
new file mode 100644
index 00000000..1e0e952c
--- /dev/null
+++ b/systemd/etc_default_static-web-server
@@ -0,0 +1,5 @@
+SERVER_ROOT=/var/www/html
+SERVER_HTTP2_TLS=true
+SERVER_HTTP2_TLS_CERT=/etc/static-web-server/example_org_fullchain.pem
+SERVER_HTTP2_TLS_KEY=/etc/static-web-server/example_org_privkey.pem
+SERVER_LOG_LEVEL=warn
diff --git a/systemd/static-web-server.service b/systemd/static-web-server.service
new file mode 100644
index 00000000..e9a7f023
--- /dev/null
+++ b/systemd/static-web-server.service
@@ -0,0 +1,107 @@
+# Example systemd service unit file (see systemd.service(5) man page) for use
+# with the --fd option of static-web-server.  This allows e.g. binding the
+# server to a TCP port number 0 - 1023 without running the server as root,
+# and/or running sws in an isolated network name space.
+#
+# This also allows sws to be started on-demand.  If sws is restart (e.g. after
+# updating its SSL certificates, or reconfiguring its content directory), new
+# inbound connections will be queued until sws is up and running again.
+#
+# A comprehensive description can be found in:
+# http://0pointer.de/blog/projects/socket-activation.html
+# ...and the linked articles.
+
+[Unit]
+Description=Static Web Server
+Wants=static-web-server.socket
+After=static-web-server.socket
+
+# The options below reflect a reasonably comprehensive sandboxing based on the
+# features available in systemd v247.  Newer versions of systemd may offer
+# additional options for sandboxing.
+#
+# The options below focus on security, when making changes to this unit file
+# you may wish to evaluated the output of:
+# systemd-analyze security static-web-server.service
+#
+# Beyond the limits used here, additional limits can be placed on CPU, memory,
+# and disk I/O, as well as network traffic filters (via eBPF and other
+# mechanisms), and implemented for this server using the systemd override
+# facilities.  See systemd.resource-control(5) for details.
+
+[Service]
+Type=simple
+
+# An example environment file for static-web-server is included in the file:
+# systemd/etc_default_static-web-server
+EnvironmentFile=/etc/default/static-web-server
+
+# File descriptor 0 corresponds to the standard input...
+ExecStart=/usr/local/bin/static-web-server --fd 0
+
+# ...so the following line attaches fd 0 of the static web server process to
+# the socket defined by the corresponding `static-web-server.socket` unit file.
+# Each instance of static-web-server currently only supports listening on a
+# single socket.
+StandardInput=fd:static-web-server.socket
+
+# Debug and tracing output goes to stderr, and can be viewed with e.g.
+# `journalctl -u static-web-server.service`.
+StandardError=journal
+
+Restart=always
+RestartSec=5
+DynamicUser=true
+SupplementaryGroups=www-data
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProtectSystem=strict
+ProtectHome=yes
+CapabilityBoundingSet=
+RestrictNamespaces=true
+
+#RestrictAddressFamilies=none
+# ☟ workaround to implement ☝in older versions of systemd.
+#   see: https://github.com/systemd/systemd/issues/15753
+RestrictAddressFamilies=AF_UNIX
+RestrictAddressFamilies=~AF_UNIX
+
+PrivateDevices=true
+PrivateUsers=true
+PrivateNetwork=true
+ProtectClock=true
+ProtectControlGroups=true
+ProtectKernelLogs=true
+ProtectKernelModules=true
+ProtectKernelTunables=true
+ProtectProc=invisible
+ProcSubset=pid
+RestrictSUIDSGID=true
+SystemCallArchitectures=native
+RestrictRealtime=true
+LockPersonality=true
+RemoveIPC=true
+MemoryDenyWriteExecute=true
+UMask=077
+ProtectHostname=true
+
+# Restrict the use of exotic system calls (bugs in seldom-used syscalls are a
+# historical source of kernel vulnerabilities)...
+SystemCallFilter=@system-service
+# ... It may be possible to restrict this further.  e.g.
+#SystemCallFilter=@signal @basic-io @io-event @network-io @process statx fstat sched_getaffinity getrandom
+# but a process to discover the set of system calls used (e.g. as part of the
+# unit tests) will probably be needed to avoid regressions e.g. due to changes
+# in crates which are used by static-web-server. The following may be useful to
+# record system calls performed:
+# "/usr/bin/strace --summary-only -o sws.syscallstats -- static-web-server [...]"
+# You can view the sets of system calls defined by systemd using:
+# "systemd-analyze syscall-filter"
+
+DevicePolicy=strict
+DeviceAllow=/dev/null rw
+DeviceAllow=/dev/random r
+DeviceAllow=/dev/urandom r
+
+[Install]
+WantedBy=multi-user.target
diff --git a/systemd/static-web-server.socket b/systemd/static-web-server.socket
new file mode 100644
index 00000000..d2f8b265
--- /dev/null
+++ b/systemd/static-web-server.socket
@@ -0,0 +1,18 @@
+# Example systemd socket unit file (see systemd.socket(5) man page) for use
+# with the --fd option of static-web-server.  This allows e.g. binding the
+# server to a TCP port number 0 - 1023 without running the server as root,
+# and/or running sws in an isolated network name space.
+#
+# A comprehensive description can be found in:
+# http://0pointer.de/blog/projects/socket-activation.html
+# ...and the linked articles.
+
+[Unit]
+Description=Static Web Server Socket
+
+[Socket]
+ListenStream=443
+Accept=no
+
+[Install]
+WantedBy=sockets.target


Test patch: