Handling Unix Kill Signals in Rust
Tags: rust, programming, linux
Published: 2021-02-20 20:27:39

## Intro

Like many of you, I am a software developer. For the past few years, I've been working with Python, both at work, and writing small hobby projects at home.

One of the most common things I do with Python is write Linux services/daemons. A linux daemon is a program, in our case written in Python, that runs in a loop, usually by SystemD, and only exits when it receives a kill signal.

A few months ago, I decided to teach myself Rust, and after reading the [Rust book](https://doc.rust-lang.org/book/title-page.html) (which I highly recommend), and watching lots of youtube videos, I tried to write a Rust Linux daemon.

Part of the process was figuring out how to handle kill signals in Rust, which is very different than how it is handled in Python. I expected this to be relatively easy, but found it to be rather difficult. In this article, we're going to take a look at the `signal_hook` crate, and the different ways Rust allows us to react to a kill signal. Hopefully, my experience will help ease the journey for some of you.

## Linux Kill Signals

The 2 most common kill signals in Linux are SIGINT, which gets sent to your program when you run the program manually, and hit Ctrl+C in your terminal, or SIGTERM, which SystemD sends to your program when you run `systemctl stop my_program`.

Unlike Python, which throws an ugly traceback when it receives an unhandled kill signal, Rust quietly exits. If your Rust program does not need to do anything special when it terminates, you can probably get away with not doing any kill signal handling at all, but if you need to do any kind of cleanup, you'll likely need to catch these kill signals, and do your cleanup before exiting.

## Crate Options

Unlike Python, Rust doesn't have built-in kill signal handling. You'll need to use an external crate to add this functionality. There are 2 main crates that allow you to handle kill signals:

- [ctrlc](https://docs.rs/ctrlc/3.1.7/ctrlc/)
- [signal_hook](https://docs.rs/signal-hook/0.3.4/signal_hook/)

The `ctrlc` crate is fairly easy to use, but only supports a limited number of kill signals:

- SIGINT
- SIGTERM ([requires enabling the termination feature](https://docs.rs/ctrlc/3.1.7/ctrlc/#handling-sigterm))

Here, I'll be taking a look at the `signal_hook` crate. Once we break down the `signal_hook` crate, the `ctrlc` crate becomes easy to understand and use as well.

## Signal Hook

This may change, but as of now, the crate documentation has 2 examples on the front page - one that is [very simple](https://docs.rs/signal-hook/0.3.4/signal_hook/index.html#using-a-flag-to-terminate-a-loop-based-application), which we will take a closer look at below, and one that is [overly complex](https://docs.rs/signal-hook/0.3.4/signal_hook/index.html#a-complex-signal-handling-with-a-background-thread), which uses features that are unlikely to be commonly needed, and [won't compile without enabling extra features](https://github.com/vorner/signal-hook/issues/93). We'll slowly work our way from the simple example, to a more complex one that resembles the one in the documentation.

## Cargo.toml

The first thing you'll want to do is add this to the dependencies section of your Cargo.toml file:

```toml
signal-hook = "0.3.4"
```

While this will work for ALL of our examples, if you want to compile the [complex](https://docs.rs/signal-hook/0.3.4/signal_hook/index.html#a-complex-signal-handling-with-a-background-thread) example from the documentation, you'll want your toml file to say this instead:

```toml
signal-hook = { version = "0.3.4", features = ["extended-siginfo"] }
```

## Simple Example

This is a slightly modified version of the simple example from the documentation:

```rust
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread;
use std::time;
use signal_hook::flag;

fn main() -> Result<(), Error> {
    // Declare bool, setting it to false
    let term = Arc::new(AtomicBool::new(false));

    // Ask signal_hook to set the term variable to true
    // when the program receives a SIGTERM kill signal
    flag::register(signal_hook::consts::SIGTERM, Arc::clone(&term))?;

    // Do work until the term variable becomes true
    while !term.load(Ordering::Relaxed) {
        println!("Doing work...");
        thread::sleep(time::Duration::from_secs(1));
    }

    // Since our loop is basically an infinite loop,
    // that only ends when we receive SIGTERM, if
    // we got here, it's because the loop exited after
    // receiving SIGTERM
    println!("Received SIGTERM kill signal. Exiting...");

    Ok(())
}
```

Lets have a look at the different parts. First, we have this:

```rust
let term = Arc::new(AtomicBool::new(false));
```

This line creates a special boolean variable that can be used safely from different threads. It can be set in one thread, by the `signal_hook` crate, and checked in your main thread.

Next, we have this line:

```rust
flag::register(signal_hook::consts::SIGTERM, Arc::clone(&term))?;
```

This tells `signal_hook` that we would like to register the SIGTERM kill signal to be monitored, and when we receive this kill signal, to have the `term` boolean set to true.

Lastly, we have the main loop:

```rust
while !term.load(Ordering::Relaxed) { ... }
```

The way we check if the special boolean we are using is true or false is slightly different than how we check regular booleans. We use the `.load()` method on it. Because the loop starts out false, and we are waiting for it to turn true, and `while` loops only run while something is true, we use `!` here. All we are saying is that we want the loop to run until the boolean turns true.

To test this program, compile it, run it, and after a few seconds, run:

```bash
killall -SIGTERM my_program
```

where `my_program` is the name of your program. That's the name you gave it when you ran `crate new my_program`. It's also the name of your binary under `target/debug/` of your project. This will send your program a SIGTERM signal, causing the boolean to get set to true, and the loop to exit. Simple.

## Double SIGINT

This is a slightly more complex example. Here, if the user hits Ctrl+C (sending our program the SIGINT kill signal), our program will begin to clean up and try to exit. If the user hits Ctrl+C again, possibly because our cleanup procedure locked up, our program will terminate immediately.

```rust
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread;
use std::time;

// This is just a collection of ints that represent kill signals.
// More specifically, they are the common kill signals used to
// terminate a program
// You can do println!("{:?}", TERM_SIGNALS) to see them
// They are just SIGINT(2), SIGTERM(15) and SIGQUIT(3)
use signal_hook::consts::TERM_SIGNALS;

// Module that sets boolean flags when kill signal is received
use signal_hook::flag;

fn main() -> Result<(), Error> {
    // A special boolean that can be used across threads
    // It will be passed to flag::register, which will
    // set it to true the first time a kill signal is received
    let term_now = Arc::new(AtomicBool::new(false));

    // Register all kill signals
    // Note: You COULD specify other, specific kill signals here
    // rather than the 3 in TERM_SIGNALS. You just need a vector
    // of constants from signal_hook::consts::signal
    for sig in TERM_SIGNALS {
        // When terminated by a second term signal, exit with exit code 1.
        // This will do nothing the first time (because term_now is false).
        flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
        // But this will "arm" the above for the second time, by setting it to true.
        // The order of registering these is important, if you put this one first, it will
        // first arm and then terminate ‒ all in the first round.
        flag::register(*sig, Arc::clone(&term_now))?;
    }

    // Main process that does work until term_now has been set
    // to true by flag::register
    while !term_now.load(Ordering::Relaxed) {
        println!("Doing work...");
        thread::sleep(time::Duration::from_secs(1));
    }

    // If we ended up here, the loop above exited because of a kill signal
    println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");

    // This simulates a long cleanup operation
    // If you wait this long, the program will exit
    // If you hit Ctrl+C again before this is done, flag::register_conditional_shutdown will kill
    // the process without waiting for it to finish. This means double Ctrl+C kills the process
    // immediately
    thread::sleep(time::Duration::from_secs(10));

    println!("Exited cleanly");

    Ok(())
}
```

This code is actually very similar to the first example, but with one additional part. Now, the `term_now` bool is used for 2 different purposes.

The first is defined here:

```rust
for sig in TERM_SIGNALS {
    flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
    flag::register(*sig, Arc::clone(&term_now))?;
}
```

As before, the `register` function will set `term_now` to true when it receives a kill signal. The new `register_conditional_shutdown` on the other hand will kill your program, but only if `term_now` is set to true. 

The first time you hit Ctrl+C, `register_conditional_shutdown` will do nothing because `term_now` will be false, but `register` will set `term_now` to true. The second time you hit Ctrl+C, `register_conditional_shutdown` will terminate the program no matter what else the program might be working on.

These 2 lines in a `for` loop will make the double-Ctrl+C work (terminate your program), but you still need your main code to watch for, and react to a kill signal to shut down your program gracefully. As before, we do this with a `while` loop that runs until `term_now` becomes true.

## Complex Example

Building on the previous examples, the following is a slightly more complex example that reacts differently to different kill signals:

```rust
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::thread;
use std::time;
use std::sync::atomic::Ordering;

use signal_hook::flag;
use signal_hook::consts::signal::*;
use signal_hook::consts::TERM_SIGNALS;
use signal_hook::iterator::Signals;

fn main() -> Result<(), Error> {
    // A special boolean that can be used across threads
    // The first time a kill signal is received, it will be set to
    // true by flag::register
    // The second time a kill signal is received, our process will
    // be killed by flag::register_conditional_shutdown
    let term_now = Arc::new(AtomicBool::new(false));

    for sig in TERM_SIGNALS {
        // When terminated by a second term signal, exit with exit code 1.
        // This will do nothing the first time (because term_now is false).
        flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
        // But this will "arm" the above for the second time, by setting it to true.
        // The order of registering these is important, if you put this one first, it will
        // first arm and then terminate ‒ all in the first round.
        flag::register(*sig, Arc::clone(&term_now))?;
    }

    // Our actual work thread
    let t = thread::spawn(move || {
        while !term_now.load(Ordering::Relaxed)
        {
            println!("Doing work...");
            thread::sleep(time::Duration::from_secs(1));
        }

        println!("\nThread exiting...");
    });

    // Create iterator over signals
    let mut signals = Signals::new(TERM_SIGNALS)?;

    // This loop runs forever, and blocks until a kill signal is received
    'outer: loop {
        for signal in signals.pending() {
            match signal {
                SIGINT => {
                    println!("\nGot SIGINT");
                    break 'outer;
                },
                SIGTERM => {
                    println!("\nGot SIGTERM");
                    break 'outer;
                },
                term_sig => {
                    println!("\nGot {:?}", term_sig);
                    break 'outer;
                },
            }
        }
    }

    // Wait for thread to exit
    t.join().unwrap();

    // Cleanup code goes here
    println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");
    thread::sleep(time::Duration::from_secs(10));
    println!("Exited cleanly");

    Ok(())
}
```

Here, a few things are going on that we are already familiar with. Just like before, we create a new bool that `signal_hook` will set to true for us when it receives a kill signal.

We also, once again, use `flag::register_conditional_shutdown` to make sure that a second SIGINT will kill our program immediately.

Something that's new here is this:

```rust
// Our actual work thread
let t = thread::spawn(move || {
    while !term_now.load(Ordering::Relaxed)
    {
        println!("Doing work...");
        thread::sleep(time::Duration::from_secs(1));
    }

    println!("\nThread exiting...");
});
```

Here, we are spawning a new worker thread where our actual work will be done, whatever that happens to be, and moving our `term_now` variable to this thread, so that the worker thread can monitor the boolean, and know when it's time to exit (when we've received a kill signal).

The worker thread looks very similar to our previous examples' main thread - it runs an infinite while loop that only exits if the boolean `signal_hook` will set for us becomes true. When it does exit, the statements following the while loop will be executed on its way out.

The next new code in this example is this:

```rust
let mut signals = Signals::new(TERM_SIGNALS)?;
```

This will let us use an iterator to iterate over any incoming signals.

The main example in the `signal_hook` documentation uses `SignalsInfo` instead. The main difference is that `Signals` will simply return the kill signal received, whereas `SignalsInfo` returns some additional info about every kill signal.

As stated above, using `Signals` doesn't require anything special, while using `SignalsInfo` requires the `extended-siginfo` feature to be enabled in your Cargo.toml, so please be aware.

After that, we have this code:

```rust
    'outer: loop {
        for signal in signals.pending() {
            match signal {
                SIGINT => {
                    println!("\nGot SIGINT");
                    break 'outer;
                },
                SIGTERM => {
                    println!("\nGot SIGTERM");
                    break 'outer;
                },
                term_sig => {
                    println!("\nGot {:?}", term_sig);
                    break 'outer;
                },
            }
        }
    }
```

This is a loop within a loop. The inner loop iterates over any kill signals in queue, while the outer loop is an infinite loop that keeps checking the queue.

When we do receive a kill signal, we can match it against the constants found in `signal_hook::consts::signal`, like SIGINT and SIGTERM, take action, and break out of both loops.

In order to easily break out of both loops, we give the outer loop a name, and our break statement tells rust that we want to break out of the loop with the given name, rather than the first enclosing loop.

After we are out of the loop, we have this line:

```rust
t.join().unwrap();
```

Because the only way to get out of the loop is by receiving a kill signal, and our worker thread will also see this kill signal, we can assume at this point that our thread is terminating, so here, we simply wait for it to return (finish its cleanup).

Once the thread returns, we can do our own cleanup here:

```rust
println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");
thread::sleep(time::Duration::from_secs(10));
println!("Exited cleanly");
```

This can be whatever you need to do before your program exits.

Because we used `flag::register_conditional_shutdown` at the beginning, if this cleanup gets stuck, the user has the option of hitting Ctrl+C again in order to terminate our program.

## Conclusion

Now that we've slowly worked up from a basic example, to a complex example, hopefully you are ready to read, and understand the example in the `signal_hook` documentation that uses `SignalsInfo`, and a few extra features we haven't covered here.

It took me a few days to figure this out - I hope with this guide, you'll be able to get going faster.

If you have any questions, please let me know. I'm still learning Rust, and definitely don't know everything about the `signal_hook` crate, but I'll try to answer them as best as I can.