Package: iosnoop-tool
iosnoop-toolparse and visualize iosnoop outputWhat is iosnoop?iosnoopis a tracing tool for disk I/O events and created byBrendan D. Gregg. Originally, it is for Solaris/MacOS X using DTrace.iosnoop - A program to print disk I/O events as they happen, with useful details such as UID, PID, filename (if available), command, etc. Written using DTrace (Solaris 10 3/05, MacOS X 10.5).http://www.brendangregg.com/DTrace/iosnoopAnd then, iosnoop was ported for Linux usingftrace. Refer to his blog for iosnoop detail.I'm probably dreaming. I just ported my popular iosnoop tool to Linux:http://www.brendangregg.com/blog/2014-07-16/iosnoop-for-linux.htmlLinux ported iosnoop is a simple bash script and can be downloaded from here.https://github.com/brendangregg/perf-tools/blob/master/iosnoopThe output is like this. iosnoop requires root permission to run. iosnoop summarizes block device I/O (disk I/O) latency (LATms) with 1 line.$sudopath/to/iosnoop-ts
TracingblockI/O.Ctrl-Ctoend.
STARTsENDsCOMMPIDTYPEDEVBLOCKBYTESLATms14545946.33211514545946.344366<...>17185WS8,1604840587442409612.2514545946.33103614545946.347082main17197WS8,1604840666914409616.0514545946.33213614545946.348852<...>17185WS8,1604840643666409616.7214545946.33106714545946.352752main17197WS8,1604844855458409621.6914545946.33214614545946.355893<...>17185WS8,1604844863762409623.7514545946.35658414545946.358920<idle>0WS8,160484485972240962.3414545946.35623814545946.361128<idle>0WS8,160484485546640964.8914545946.35694314545946.364070<idle>0WS8,160484486377040967.13Why iosnoop-tool?iosnoop-toolis a visualizing tool for iosnoop output.Brendan also has provided a visualization tool named trace2heatmap.pl, and introduced how to make Heatmap from iosnoop output.Linux iosnoop Latency Heat MapsI confirmed this tool works and know heatmap seems good enough. However, I am not good at reading/writing perl script, so I created my visualization tool in order to customize for my purpose.RequirementsPython >= 3.4SeabornSupport output of iosnoop for Linux onlyneed-tsarguments$sudoiosnoop-tsHow to installfor developerCreate virtualenv to installiosnoop-tool.$mkdirvirtualenvs
$virtualenv-ppython3virtualenvs/venv
$sourcevirtualenvs/venv/bin/activate(venv)$Installiosnoop-toolpackage from the repository.(venv)$gitclonehttps://github.com/t2y/iosnoop-tool.git(venv)$cdiosnoop-tool(venv)$pythonsetup.pydevelopBase CLI optionsConfirmiosnoop-cliworks as below.(venv)$iosnoop-cli--help
usage:iosnoop-cli[-h][--basedateBASEDATE]--dataDATA[--columnsCOLUMNS[COLUMNS...]][--io-commandsIO_COMMANDS[IO_COMMANDS...]][--io-deviceIO_DEVICE][--io-pidsIO_PIDS[IO_PIDS...]][--io-typesIO_TYPES[IO_TYPES...]][--sinceSINCE][--untilUNTIL][-v][--version]{csv,plot}...

positionalarguments:{csv,plot}optionalarguments:-h,--helpshowthishelpmessageandexit--basedateBASEDATEsetbasedatetimetoconvertkerneltimestamptolocaltime,format:yyyymmddHHMISS--dataDATAsetpathtoiosnoopoutputfile--columnsCOLUMNS[COLUMNS...]setcolumnsnameiniosnoopoutput--io-commandsIO_COMMANDS[IO_COMMANDS...]setcommandsiniosnoopoutput--io-deviceIO_DEVICEsetdeviceiniosnoopoutput--io-pidsIO_PIDS[IO_PIDS...]setprocessidsiniosnoopoutput--io-typesIO_TYPES[IO_TYPES...]settypesiniosnoopoutput--sinceSINCEsetsecondssincerelativedifferencefromstart--untilUNTILsetsecondsuntilrelativedifferencefromstart-v,--verbosesetverbosemode--versionshowprogramversionSub CommandscsvCreate csv/tsv file from output of iosnoop.(venv)$iosnoop-clicsv--help
usage:iosnoop-clicsv[-h][--dialect{excel,excel-tab,unix}][--outputOUTPUT][--separatorSEPARATOR]optionalarguments:-h,--helpshowthishelpmessageandexit--dialect{excel,excel-tab,unix}setdialectforcsvwriter,defaultisexcel--outputOUTPUTsetpathtosavecsvfileofiosnoop--separatorSEPARATORsetseparator(choosefromcomma,tab),defaultiscommaplotCreate image file rendered by seaborn from output of iosnoop.(venv)$iosnoop-cliplot--help
usage:iosnoop-cliplot[-h][--backendBACKEND][--colormapCOLORMAP][--fig-outputFIGOUTPUT][--fig-sizeFIGSIZE][--hspaceHSPACE][--plot-type{heatmap}][--square][--subplot-conditionsSUBPLOT_CONDITIONS[SUBPLOT_CONDITIONS...]][--x-intervalX_INTERVAL][--x-maxX_MAX][--y-intervalY_INTERVAL][--y-maxY_MAX]optionalarguments:-h,--helpshowthishelpmessageandexit--backendBACKENDsetbackendformatplotlib,useTkAggtomonitorintheforeground--colormapCOLORMAPsetcolormapforseabornheatmap--fig-outputFIGOUTPUTsetpathtosavegraph--fig-sizeFIGSIZEsetfiguresize--hspaceHSPACEsethspaceforsubplot--plot-type{heatmap}setplottype("heatmap"bydefault)--squaresetsquaremodeforheatmap--subplot-conditionsSUBPLOT_CONDITIONS[SUBPLOT_CONDITIONS...]setDataFrameconditionstofilter--x-intervalX_INTERVALsetvalueofintervalforxbins--x-maxX_MAXsetmaximumvalueforx-axis--y-intervalY_INTERVALsetvalueofintervalforybins--y-maxY_MAXsetmaximumvaluefory-axisHow to useThis is sample heatmap rendered by seaborn.(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logplot--y-max600--y-interval10--fig-outputtests/fixtures/iosnoop-sample.png
...(venv)$filetests/fixtures/iosnoop-sample.png
tests/fixtures/iosnoop-sample.png:PNGimagedata,1024x800,8-bit/colorRGBA,non-interlacedSpecifying --basedate options converts kernel timestamp to localtime.(vent)$iosnoop-cli--basedate20180702230100--datatests/fixtures/iosnoop-sample.logplot--y-max600--y-interval10--fig-outputtests/fixtures/iosnoop-sample-with-basedate.pngExamplesplotshow graphical window instead of saviing image file(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logplot--backendTkAggadjust x-axis and y-axis as needed, default settings might not appropriate depends on output of iosnoop(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logplot--x-max100--x-interval2.0--y-max300--y-interval5compare multiple subplotsUse header name to retrieve particular data inpandas.DataFrame.(venv)$headtests/fixtures/iosnoop-sample.log
STARTsENDsCOMMPIDTYPEDEVBLOCKBYTESLATms14545946.33211514545946.344366<...>17185WS8,1604840587442409612.2514545946.33103614545946.347082main17197WS8,1604840666914409616.0514545946.35694314545946.364070<idle>0WS8,160484486377040967.13retrieved by exact match and contains string, --hspace is used for adjusting height space for each subplots(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logplot--y-max300--y-interval30--subplot-conditions"COMM == 'main'""COMM.str.contains('jbd2|kblockd|flush')""COMM == '<idle>'"--hspace0.9--fig-outputtests/fixtures/iosnoop-sample-comm.pngretrieved by bytes between, --hspace is used for adjusting height space for each subplots(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logplot--subplot-conditions"BYTES.between(0, 131072)""BYTES.between(131073, 262144)""BYTES.between(262145, 524287)""BYTES == 524288"--hspace1.2--fig-outputtests/fixtures/iosnoop-sample-bytes.pngcsvwrite contents parsed from iosnoop output as csv file(venv)$iosnoop-cli--datatests/fixtures/iosnoop-sample.logcsv--outputiosnoop.csv(venv)$headiosnoop.csv
STARTs,ENDs,COMM,PID,TYPE,DEV,BLOCK,BYTES,LATms,STARTs_DIFF14545946.332115,14545946.344366,<...>,17185,WS,"8,160",4840587442,4096,12.25,014545946.331036,14545946.347082,main,17197,WS,"8,160",4840666914,4096,16.05,-0.001079000532627105714545946.332136,14545946.348852,<...>,17185,WS,"8,160",4840643666,4096,16.72,2.099946141242981e-05