Repository: hubblo-org/scaphandre (language: rust)

Issue:
Add option to display more than five processes (stdout exporter)
## Problem

Currently, the stdout exporter only display the first five processes.

## Solution

An interesting feature would be to add an option to the exporter, so that we can view an arbitrary number of processes.

Like ```./scaphandre stdout -p 10```

The difficulty is that making this change require a modification in two files (example with a fixed value):

- One in [src/exporters/stdout.rs](https://github.com/hubblo-org/scaphandre/blob/main/src/exporters/stdout.rs):
    ```rust
    let consumers = self.topology.proc_tracker.get_top_consumers(10);
    ```
    The exporter then display 10 lines, but only the first five are not empty, as scaphandre still send only five of them.

- The second modification in [src/sensors/mod.rs](https://github.com/hubblo-org/scaphandre/blob/main/src/sensors/mod.rs)
    ```rust
    pub fn new() -> Topology {
             Topology {
                 sockets: vec![],
    -            proc_tracker: ProcessTracker::new(5), 
    +            proc_tracker: ProcessTracker::new(10),
                 stat_buffer: vec![],
                 ...
    ```

    Now we really get the ten processes.

For what I understand, it will be required to override the Topology structure inside stdout.rs, but I'm not sure if it is possible, and if it is a good thing (I'm not experienced in Rust).



Discussion / hints:
@BasicAcid, I will have a look if I can add this feature. I think it could be also interesting to have a switch to pass a regexp and display the processes that will match it.
Hi !

> The second modification in src/sensors/mod.rs
> 
> pub fn new() -> Topology {
>          Topology {
>              sockets: vec![],
> -            proc_tracker: ProcessTracker::new(5), 
> +            proc_tracker: ProcessTracker::new(10),
>              stat_buffer: vec![],
>              ...

This is not about the number of processes returned in the top consumers list. This is about the number of statistics records we store in the internal "buffer" for each process. You don't need to touch this part to configure the number of process returned by the stdout exporter. This only resides in `src/exporters/stdout.rs` and could look like what I recently proposed on the json exporter:

[https://github.com/hubblo-org/scaphandre/pull/97/files#diff-d4e02f0252f3f8939ffb5643c89bacb8cf4aaeb06ad8f999f02cba1250235335R72](https://github.com/hubblo-org/scaphandre/pull/97/files#diff-d4e02f0252f3f8939ffb5643c89bacb8cf4aaeb06ad8f999f02cba1250235335R72)

and
[https://github.com/hubblo-org/scaphandre/pull/97/files#diff-d4e02f0252f3f8939ffb5643c89bacb8cf4aaeb06ad8f999f02cba1250235335R180](
https://github.com/hubblo-org/scaphandre/pull/97/files#diff-d4e02f0252f3f8939ffb5643c89bacb8cf4aaeb06ad8f999f02cba1250235335R180)

@BasicAcid feel free to look at this code and propose something similar for stdout :)

> I will have a look if I can add this feature. I think it could be also interesting to have a switch to pass a regexp and display the processes that will match it.

@uggla the regexp would be awesome :)

Fix patch:
diff --git a/docs_src/references/exporter-stdout.md b/docs_src/references/exporter-stdout.md
index 74496655..af569898 100644
--- a/docs_src/references/exporter-stdout.md
+++ b/docs_src/references/exporter-stdout.md
@@ -6,19 +6,37 @@ You can launch the stdout exporter this way (running the default powercap_rapl s
 
 	scaphandre stdout
 
-Default behavior is to measure and show metrics periodically during 10 seconds. You can change that timeout with `-t`. Here is how to display metrics during one minute:
+Default behavior is to measure and show metrics periodically during 10 seconds. You can change that timeout with `-t`.
+A value of `-t 0` will display top consumers infinitely and must be interrupted with ctrl-c.
+
+Here is how to display metrics during one minute:
 
     scaphandre stdout -t 60
 
-You can change as well the step measure duration with -s. Here is how to display metrics during one minutes with a 5s step:
+You can change as well the step measure duration with `-s`. Here is how to display metrics during one minutes with a 5s step:
 
     scaphandre stdout -t 60 -s 5
 
+You can change the number of top consumers displayed with `-p`. Here is how to display the first 20 top consumers:
+
+    scaphandre stdout -p 20
+
+You can filter the processes to display with `-r`. A warning will be risen if this option is used with `-p` at the same time.
+In such case, `-p` behavior is disabled.
+
+The `-r` expected parameter is a regular expression. Details can be found [here](https://docs.rs/regex/1.4.5/regex/#syntax) and tested [here](https://rustexp.lpil.uk/).
+
+Here is how to display power data for the 'scaphandre' process:
+
+    scaphandre stdout -r 'scaphandre'
+
+Note
+
 As always exporter's options can be displayed with `-h`:
 
 	$ scaphandre stdout -h
     scaphandre-stdout
-    Stdout exporter allows you to output the power consumption data in the terminal.
+    Stdout exporter allows you to output the power consumption data in the terminal
 
     USAGE:
         scaphandre stdout [OPTIONS]
@@ -28,5 +46,10 @@ As always exporter's options can be displayed with `-h`:
         -V, --version    Prints version information
 
     OPTIONS:
-    -s, --step <step_duration>    Set measurement step duration in seconds. [default: 2]
-    -t, --timeout <timeout>       Maximum time spent measuring, in seconds. [default: 10]
+        -p, --process <process_number>    Number of processes to display. [default: 5]
+        -r, --regex <regex_filter>        Filter processes based on regular expressions (e.g: 'scaph\w\wd.e'). This option
+                                          disable '-p' or '--process' one.
+        -s, --step <step_duration>        Set measurement step duration in seconds. [default: 2]
+        -t, --timeout <timeout>           Maximum time spent measuring, in seconds. 0 means continuous measurement.
+                                          [default: 10]
+
diff --git a/src/exporters/stdout.rs b/src/exporters/stdout.rs
index 8e32fd43..8d646b80 100644
--- a/src/exporters/stdout.rs
+++ b/src/exporters/stdout.rs
@@ -2,6 +2,8 @@ use clap::Arg;
 
 use crate::exporters::*;
 use crate::sensors::{Record, Sensor, Topology};
+use colored::*;
+use regex::Regex;
 use std::collections::HashMap;
 use std::thread;
 use std::time::{Duration, Instant};
@@ -23,7 +25,7 @@ impl Exporter for StdoutExporter {
         let mut options = Vec::new();
         let arg = Arg::with_name("timeout")
             .default_value("10")
-            .help("Maximum time spent measuring, in seconds.")
+            .help("Maximum time spent measuring, in seconds. 0 means continuous measurement.")
             .long("timeout")
             .short("t")
             .required(false)
@@ -39,6 +41,23 @@ impl Exporter for StdoutExporter {
             .takes_value(true);
         options.push(arg);
 
+        let arg = Arg::with_name("process_number")
+            .default_value("5")
+            .help("Number of processes to display.")
+            .long("process")
+            .short("p")
+            .required(false)
+            .takes_value(true);
+        options.push(arg);
+
+        let arg = Arg::with_name("regex_filter")
+            .help("Filter processes based on regular expressions (e.g: 'scaph\\w\\wd.e'). This option disable '-p' or '--process' one.")
+            .long("regex")
+            .short("r")
+            .required(false)
+            .takes_value(true);
+        options.push(arg);
+
         options
     }
 }
@@ -54,26 +73,59 @@ impl StdoutExporter {
 
     /// Runs iteration() every 'step', during until 'timeout'
     pub fn runner(&mut self, parameters: ArgMatches) {
-        let timeout = parameters.value_of("timeout").unwrap();
-        if timeout.is_empty() {
-            self.iterate();
-        } else {
-            let now = Instant::now();
+        // Parse parameters
+        // All parameters have a default values so it is safe to unwrap them.
+        // Panic if a non numerical value is passed except for regex_filter.
 
-            let timeout_secs: u64 = timeout.parse().unwrap();
+        let timeout_secs: u64 = parameters
+            .value_of("timeout")
+            .unwrap()
+            .parse()
+            .expect("Wrong timeout value, should be a number of seconds");
 
-            // We have a default value of 2s so it is safe to unwrap the option
-            // Panic if a non numerical value is passed
-            let step_duration: u64 = parameters
-                .value_of("step_duration")
-                .unwrap()
-                .parse()
-                .expect("Wrong step_duration value, should be a number of seconds");
+        let step_duration: u64 = parameters
+            .value_of("step_duration")
+            .unwrap()
+            .parse()
+            .expect("Wrong step_duration value, should be a number of seconds");
 
-            println!("Measurement step is: {}s", step_duration);
+        let process_number: u16 = parameters
+            .value_of("process_number")
+            .unwrap()
+            .parse()
+            .expect("Wrong process_number value, should be a number");
+
+        let regex_filter: Option<Regex>;
+        if !parameters.is_present("regex_filter")
+            || parameters.value_of("regex_filter").unwrap().is_empty()
+        {
+            regex_filter = None;
+        } else {
+            regex_filter = Some(
+                Regex::new(parameters.value_of("regex_filter").unwrap())
+                    .expect("Wrong regex_filter, regexp is invalid"),
+            );
+        }
+
+        if parameters.occurrences_of("regex_filter") == 1
+            && parameters.occurrences_of("process_number") == 1
+        {
+            let warning =
+                String::from("Warning: (-p / --process) and (-r / --regex) used at the same time. (-p / --process) disabled");
+            eprintln!("{}", warning.bright_yellow());
+        }
+
+        println!("Measurement step is: {}s", step_duration);
+        if timeout_secs == 0 {
+            loop {
+                self.iterate(&regex_filter, process_number);
+                thread::sleep(Duration::new(step_duration, 0));
+            }
+        } else {
+            let now = Instant::now();
 
             while now.elapsed().as_secs() <= timeout_secs {
-                self.iterate();
+                self.iterate(&regex_filter, process_number);
                 thread::sleep(Duration::new(step_duration, 0));
             }
         }
@@ -99,12 +151,12 @@ impl StdoutExporter {
         }
     }
 
-    fn iterate(&mut self) {
+    fn iterate(&mut self, regex_filter: &Option<Regex>, process_number: u16) {
         self.topology.refresh();
-        self.show_metrics();
+        self.show_metrics(regex_filter, process_number);
     }
 
-    fn show_metrics(&self) {
+    fn show_metrics(&self, regex_filter: &Option<Regex>, process_number: u16) {
         let host_power = match self.topology.get_records_diff_power_microwatts() {
             Some(record) => record.value.parse::<u64>().unwrap(),
             None => 0,
@@ -145,27 +197,42 @@ impl StdoutExporter {
                     to_print.push_str("---- \t\t");
                 }
             }
-            println!("{}", to_print);
+            println!("{}\n", to_print);
         }
 
-        println!("Top 5 consumers:");
-        println!("Power\tPID\tExe");
 
-        let consumers = self.topology.proc_tracker.get_top_consumers(5);
-        for c in consumers.iter() {
-            if let Some(host_stat) = self.topology.get_stats_diff() {
-                let host_time = host_stat.total_time_jiffies();
-                println!(
-                    "{} W\t{}\t{:?}",
-                    ((c.1 as f32 / (host_time * procfs::ticks_per_second().unwrap() as f32))
-                        * host_power as f32)
-                        / 1000000.0,
-                    c.0.pid,
-                    c.0.exe().unwrap_or_default()
-                );
-            }
+        let consumers: Vec<(procfs::process::Process, u64)>;
+        if let Some(regex_filter) = regex_filter {
+            println!("Processes filtered by '{}':", regex_filter.as_str());
+            consumers = self
+                .topology
+                .proc_tracker
+                .get_filtered_processes(regex_filter);
+        } else {
+            println!("Top {} consumers:", process_number);
+
+            consumers = self.topology.proc_tracker.get_top_consumers(process_number);
         }
 
+        println!("Top {} consumers:", consumers.len());
+        println!("Power\tPID\tExe");
+        if consumers.is_empty() {
+            println!("No processes found yet or filter returns no value.");
+        } else {
+            for c in consumers.iter() {
+                if let Some(host_stat) = self.topology.get_stats_diff() {
+                    let host_time = host_stat.total_time_jiffies();
+                    println!(
+                        "{} W\t{}\t{:?}",
+                        ((c.1 as f32 / (host_time * procfs::ticks_per_second().unwrap() as f32))
+                            * host_power as f32)
+                            / 1000000.0,
+                        c.0.pid,
+                        c.0.exe().unwrap_or_default()
+                    );
+                }
+            }
+        }
         println!("------------------------------------------------------------\n");
     }
 }
diff --git a/src/sensors/utils.rs b/src/sensors/utils.rs
index 0e4a97da..a48fe28b 100644
--- a/src/sensors/utils.rs
+++ b/src/sensors/utils.rs
@@ -1,4 +1,5 @@
 use procfs::process::Process;
+use regex::Regex;
 use std::time::{Duration, SystemTime};
 
 #[derive(Debug, Clone)]
@@ -195,16 +196,23 @@ impl ProcessTracker {
         None
     }
 
+    /// Returns the CPU time consumed between two measure iteration
+    fn get_cpu_time_consumed(&self, p: &[ProcessRecord]) -> u64 {
+        let last_time = p.first().unwrap().total_time_jiffies();
+        let previous_time = p.get(1).unwrap().total_time_jiffies();
+        let mut diff = 0;
+        if previous_time <= last_time {
+            diff = last_time - previous_time;
+        }
+        diff
+    }
+
+    /// Returns processes sorted by the highest consumers in first
     pub fn get_top_consumers(&self, top: u16) -> Vec<(Process, u64)> {
         let mut consumers: Vec<(Process, u64)> = vec![];
         for p in &self.procs {
             if p.len() > 1 {
-                let last_time = p.first().unwrap().total_time_jiffies();
-                let previous_time = p.get(1).unwrap().total_time_jiffies();
-                let mut diff = 0;
-                if previous_time <= last_time {
-                    diff = last_time - previous_time;
-                }
+                let diff = self.get_cpu_time_consumed(p);
                 let higher: Vec<&(Process, u64)> = consumers
                     .iter()
                     .filter(|x| ProcessRecord::new(x.0.to_owned()).total_time_jiffies() > diff)
@@ -221,6 +229,23 @@ impl ProcessTracker {
         consumers
     }
 
+    /// Returns processes filtered by a regexp
+    pub fn get_filtered_processes(&self, regex_filter: &Regex) -> Vec<(Process, u64)> {
+        let mut consumers: Vec<(Process, u64)> = vec![];
+        for p in &self.procs {
+            if p.len() > 1 {
+                let diff = self.get_cpu_time_consumed(p);
+                let process_name = p.last().unwrap().process.exe().unwrap_or_default();
+                if regex_filter.is_match(process_name.to_str().unwrap_or_default()) {
+                    consumers.push((p.last().unwrap().process.clone(), diff));
+                    consumers.sort_by(|x, y| y.1.cmp(&x.1));
+                }
+            }
+        }
+
+        consumers
+    }
+
     /// Drops a vector of ProcessRecord instances from self.procs
     /// if the last ProcessRecord from the vector is of state Terminated
     /// (if the process is not running anymore)


Test patch: