Package: topplot
topplotMunge logs from the commandline utilitytopin to useful graphs[For latest updates seeRELEASE.md][[TOC]]Introductiontopplotproduces graphs from information it munges fromtoplogs. It can select which processes to focus on, and it can split out information by cpu core (iftopwas configured to record the cpu core column, and/or display the cpu summary info by core).As of version0.0.5topplotruns on Linux and Windows. (It should run on any platform that suports Python's tkinter module. Please
send reports if you can test this.)topplotcan save the graphs as PNG files. It can also print information derived from the logs to stdout, with or without emitting the graphs.There may be better, more efficient ways of collecting live system information, but if for some reason you've hundreds of thousands of lines oftoplogs and you want toseewhat's in them,topplotcan help.(I wrotetopplotwhen one ofCodethink's clients asked us to investigate an issue which had 300,000 lines oftoplogs attached to it.)Turn this...Snipped first iteration of 300 in the log file:top - 13:35:22 up 1 min,  0 users,  load average: 0.71, 0.47, 0.18
Tasks: 203 total,   1 running, 202 sleeping,   0 stopped,   0 zombie
%Cpu0  :  5.9 us,  5.9 sy,  0.0 ni, 88.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  5.6 us,  0.0 sy,  0.0 ni, 94.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  15717.0 total,  13778.2 free,   1030.7 used,    908.1 buff/cache
MiB Swap:  15792.0 total,  15792.0 free,      0.0 used.  14327.6 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ P COMMAND
 2430 jonatha+  20   0    9164   3704   3120 R   6.2   0.0   0:00.02 0 top -bd 1 -n 300
    1 root      20   0  167036  10768   7800 S   0.0   0.1   0:01.33 3 /sbin/init
    2 root      20   0       0      0      0 S   0.0   0.0   0:00.00 3 [kthreadd]
    3 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 0 [rcu_gp]
    4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 0 [rcu_par_gp]
    .
    .
------->8  snip 193 lines of process information
    .
    .
 2057 jonatha+  20   0  270244  35968  32508 S   0.0   0.2   0:00.04 2 /usr/bin/plasma-browser-integration-host /usr+
 2064 jonatha+  20   0  341308  38504  32424 S   0.0   0.2   0:00.07 2 /usr/lib/x86_64-linux-gnu/libexec/vvvvvvvvvvv
 2069 jonatha+  20   0    6640   3136   2888 S   0.0   0.0   0:00.00 3 /bin/bash /home/jonathansambrook/.config/wwww+
 2070 jonatha+  20   0  657204  43148  26504 S   0.0   0.3   0:00.18 3 /home/jonathansambrook/xxxxxxxxxxxxxxxx/yyyyy+
 2098 jonatha+  20   0 2414952  77112  54068 S   0.0   0.5   0:00.27 2 /opt/firefox/firefox-bin -contentproc -childI+... in to these:The overview figure:Processes of interest (POI) by cpu core:Note that the overview figure is a screen capture, hence the coloured background, whilst the POI figure was produced bytopplot. Click on either image to embiggen it.You can see that some plotted lines have been turned off (by clicking on a legend line), and that various points have been annotated with text in yellow boxes in the second figure (by clicking on the line itself).Installing topplotDependenciesDependencies should mostly be handled bypipor your package manager.The only currently known exception is that your Python3 installation must have theTkintermodule available.On Debian this can be done by running:sudo apt install python3-tkInstalling for normal usageIf you just want to havetopplotinstalled so you can use it:If you're running on Debian Linux:sudo apt install topplotOtherwise:pip3 install topplotNote: some distros only have Python3 installed, so where I'm usingpip3you may need to use the unadornedpip.Installing for hacking ontopplotIf you want to hack ontopplot, clone the repo and setup a Python virtual environment:# clone the repo
git clone https://gitlab.com/eBardie/topplot
cd topplot

# create and activate a Python virtual environment to isolate dependencies from your base system
python3 -m venv .venv
source .venv/bin/activate

# install dependencies inside the virtual environment
python3 -m pip install -e .

# hack on topplot

# .
# .
# .

# When done, leave the virtual environment
deactivateFor subsequent sessions all you'll need to do is:# cd to where ever you cloned the topplot repo
cd topplot

# Active the Python virtual environment
source venv/bin/activate

# hack some more on topplot

# .
# .
# .

# When done, leave the virtual environment
deactivateSee also:External issues: Glitches and fixes in dependencies.Getting the logs from topSeegenerating top logs.Specifying which log file to useBy defaulttopplotexpects the log to be a file calledtop.login the current working directory. You can use the-fcommandline option to specify a file path.Filtering the inputLimit the range of log entries by timestamp:topplot -s 18:38:00 -S 18:39:15-s TIMESTAMP, --start TIMESTAMP      Start with time stamp ([D:]HH:MM:SS)
  -S TIMESTAMP, --stop TIMESTAMP       Stop with time stamp  ([D:]HH:MM:SS)These arguments select processes of interest for graphing:-c [N], --acc-cpu [N]        Top N processes ranked by accumulated CPU use (default: 10)
  -m [N], --acc-mem [N]        Top N processes ranked by accumulated MEM use (default: 10)

         --peak-cpu [N]        Top N processes ranked by peak CPU use (default: 10)
         --peak-mem [N]        Top N processes ranked by peak MEM use (default: 10)

        --pct-cpu [PCT]        Any process using more than PCT% of memory will be graphed (default: 20)
        --pct-mem [PCT]        Any process using more than PCT% of cpu will be graphed (default: 3)

        --prio [cmpPRIO]       Any process with priority =, <=, >=, <, or > to PRIO (default: '=RT', note the prefixed comparison operator)These two arguments can make the processes graph clearer by plotting only one or the other of CPU or MEM related information:-C, --only-proc-cpu         Don't plot processes' mem info
    -M, --only-proc-mem         Don't plot processes' cpu infoFiltering by process name:REGEX                        Python style regex for names of processes to graph
  -I REGEX, --ignore REGEX     Python style regex for names of processes to completely ignore

  -i                           Use case insensitive matchingTextual outputUse one or more instances of the--listargument, or-lor-llor-lll, to display increasing levels of information about processes.Use-vto increase the verbosity of other optional filtering arguments such as--peak-cpu.Use--no-graphor-Gto surpress graphing.More commandline optionsTo see the full set of commandline options:-h, --help                   show this help message and exitThe GUIOncetopplothas parsed and munged the data, by default it will display the overview graph.Press the1key to display the top left graph in a separate window,2to display the top right,3lower left,4lower right, or0to re-open the overview graph from another window.Iftopwas configured to output the process data with teh last core it ran on, press uppercasecto display the graphs for processes running on individual cores.Iftopwas configured to output the CPU data for each individual core, press uppercaseCto display the CPU graphs for individual graphs.Presshto display helpful infomation about usingtopplot.Limiting the displayed dataIf you want to narrow down the data displayed, click on the items in a graph's legend to toggle their visibility.Click on a legend's title to toggle all of its lines.Right click on a legend's title to make all of its lines visible.Caveat: the "mem data" graph's legend doesn't need or implement toggling.LegendsPressl(lowercase 'ell') to toggle legend visibility. If the mouse pointer is over a particular graph, then only the legend(s) on that graph will be affected. If the mouse pointer is between graphs, the legends on all graphs on that figure will be toggled.Legends can be dragged around within their windows but be careful to not leave a legend from one graph entire within a separate graph - it will not be possible to interact with it any more, including moving it off of that graph!Saving to png filesPresspto 'print' an image of the current figure to a PNG file to the current working directory.Press uppercasePto 'print' images of all the open figures to PNG files to the current working directoty.Presssto save an image of the current figure via a file dialogue window.Zooming intl;dr: Click on the Pan/Zoom button (the arrow-headed cross), then whilst keeping the CONTROL key depressed,rightclick on the area of a graph you wish to zoom in on, and drag the mouse around.For full details see:https://matplotlib.org/3.1.1/users/navigation_toolbar.html. [Note thattopplotoverrides some keypresses.]Special features of the "processes of interest" graphPresstswap between having the mem axis, the cpu axis, or both axes visible.Special features of the "cpu data" graphFor top logs with per core cpu data available, the "cpu (grouped)" legend toggles lines across all cores.Caveat emptortop versionstopplotis known to work with log formats generated bytopfrom theprocps-ngpackage versions3.2.8and3.3.15.Handling further formats may be as simple as adding new regexes to the Re_Variants instances.Android uses Toybox to providetop, and is not yet handled bytopplot. Busybox'stophas not been tried, but seeing as Toybox
was forked from Busybox I'd guess Busybox'stopwon't be compatible either.External Issues: Glitches and fixes in dependenciesCurrently the only known issue is Matplotlib not (yet) having adapted to changes in Python's Tk library. The effect for Topplot is that keyboard control doesn't work on Windows if you're using Python 3.10 or a clean install of Python 3.9.9. (If your Python 3.9.9 was upgraded from an earlier install of Python 3.9.x where x<=8, you may be in luck if the Python installer didn't upgrade the Tk DLL.)Seethis Topplot bug reportfor a quick fix. Seethis Matplotlib PRfor details of the proposed full fix.