Package: iperflexer
This is a program to extract bandwidths and data transfers fromiperfoutput. To install it you can use pip.pip install iperflexerIt is meant to be used with files with data-reporting intervals shorter than the total run-time (e.g. 1 second intervals). Once installed it creates a command-line command namedparseiperfand can also be used as a library (see theSumParserclass documentation for an example of using it this way).Running ItThe original idea behind it was to use it as part of a pipeline so you can pipe iperf output to it directlyiperf -c 192.168.10.5 -i 1 -P 4 | parseiperfWhich will then send the total bandwidth for each interval to standard-output (it does this so that it can then be piped to another command). You can alsocatan existing file to it:cat test.iperf | parseiperfThe Command-Line OptionsYou can get a list of the options using the help option.parseiperf-husage: parseiperf [-h] [-g GLOB] [-u UNITS] [-i INTERVAL]
                  [--tolerance TOLERANCE] [-s] [-v] [--pudb] [--pdb] [-t]
                  [-p THREADS] [-m MAXIMUM] [-l] [-d]

optional arguments:
  -h, --help            show this help message and exit
  -g GLOB, --glob GLOB  A file-glob to match input file names. (default=None)
  -u UNITS, --units UNITS
                        Output units per second
                        [bits,Bytes,KBits,KBytes,Mbits,MBytes,Gbits,GBytes]
                        (default=Mbits)
  -i INTERVAL, --interval INTERVAL
                        Reporting intervals (default=1)
  --tolerance TOLERANCE
                        Interval tolerance (if parsed interval differs from
                        expected by more than tolerance, assume summation
                        lines) (default=0.1)
  -s, --save            If glob is provided, save to a file instead of sending
                        to stdout. (default=False)
  -v, --voodoo          Add threads yourself instead of using 'sum' lines.
                        (default=False)
  --pudb                Enable pudb (if installed). (default=False)
  --pdb                 Enable pdb (default=False)
  -t, --tee             Send lines standard error as they come in.
                        (default=False)
  -p THREADS, --threads THREADS
                        Specify thread count (default=4)
  -m MAXIMUM, --maximum MAXIMUM
                        Maximum allowed bandwidth (default=1000000)
  -l, --lastlinebandwidth
                        Get Iperf's final bandwidth value (default=False)
  -d, --lastlinetransfer
                        Get Iperf's final reported data transfer
                        (default=False)GlobAlthough I said above that you pipe input to theparseiperfcommand you can also use a file-glob to pass in file-names to it. It descends into all sub-directories to look for matches so you need to make sure the glob is restrictive enough to not match unexpected files and you also need to surround it with quotation marks (so that the shell’s interface doesn’t expand the glob before passing it to the command). The output for all the files will be concatenated and output to standard output.parseiperf -g "*.iperf"Although my example uses a glob you could just pass in a file name.UnitsThe--unitstells theiperflexerwhat units the bandwidths should use. One of the reasons for this code was to allow the iperf raw-data to be output in bits per second and then converted later to the more human-readable Megabits or Gigabits per second. The names for the units is meant to match the iperf headers (so to get megabits-per-second useMbits).SavesAs noted above, the default behavior is to send output to standard output, concatenating the values for multiple data-files, but if you passed in a file-name-glob you can use the--saveoption and it will save the output for each input file to a separate file. The output-file-names will be the input-file-names with_parsed.csvappended to them.VooDooThe--voodooflag will telliperflexerto re-add the individual threads (otherwise it uses the summed-values that iperf provides). In retrospect this should have been the default, but I didn’t have a complete enough understanding of how iperf’s reporting worked when I created the command-line interface. If you are doing a time-series analysis, the--voodoointerval sums will catch reporting intervals that are missing their summation lines and so should be considered better for that case. The intervals aren’t quiet right for re-calculating the total values though, so you should use the--lastlinebandwidthand--lastlinetransfersif you are trying to find the values for the entire session.DebuggingThe--pudband--pdboptions will start the chosen debugger before running the command.TeeThe--teeoption tells the command to send the un-parsed iperf lines to standard error. This way if you use it in the originally intended way (piping to another command) you can still see what happened or you can pipe it to another command (including another instance of theparseiperfcommand).ThreadsBy default, it is assumed that only one thread is being used. If more than one thread was used the number of threads has to be specified by passing in the--threadsoption. I might change it to make it infer them but I originally chose to do it this way because it seemed safest in the event of incomplete data output.MaximumSometimes when the bandwidth gets really small iperf seems to react by generating an unbelievably large number. When theiperflexersees a bandwidth that’s larger than what’s set as the--maximum(default is\(10^6\)) it will report the value as 0.Last Line BandwidthIf set, this will grab the last sum-line whose interval is bigger than the interval given (which is assumed to be the bandwidth-rate that iperf calculated for the session). This will generally be different from the value you would get if you were to re-add the interval counts, especially if any of the threads doesn’t report back in time for the interval (it looks like iperf drops the value for the interval’s reporting but adds it in once the thread does report back). Since it’s usually the last line you could probably get it with some clevertailandsedcombining or some other command-line combination, but this will also convert the units for you and allow you to process a set of files. Suppose you wanted to create a column of the bandwidth summaries for files that start with ‘client’ and end with ‘iperf’ and you want to store the outcome in ‘client_summaries.csv’. You could do it with:parseiperf -g "client*iperf" -l > client_summaries.csvLast Line TransferAs you might suspect, this tries to grab the last reported data-transfer (the transfer for the entire interval). It is currently mutually-exclusive to the last line bandwidth. This should be changed eventually. An example to gather the server-side information (assuming the names are meaningful):parseiperf -g "server*iperf" -l > server_transfer_summaries.csvIntervalThe expected reporting interval (e.g. every second is-i1). If the interval found on a line exceeds the expected interval, then it is assumed to be a total summation line and not used by the interval values, but is used by the SumParser.last_line_bandwidth and last_line_transfer.ToleranceThe amount that the calculated reporting interval is for a line can differ from the expected interval. This was created to prevent float rounding problems, but probably doesn’t need to be set.Building the DocumentationI try to keep an updated build of the documentation on the web atbitbucketandpypibut these instructions should help if you want to build them yourself.The code was written in a literate programming style usingPweaveso next to each python file you will also find a restructured-text file used for the documentation – the python package is also thesphinxsource for the documentation. If you want to edit or re-generate a file you need pweave:pip install pweaveNote that pweave has changed since I started using it, requiring Python 3 styleprintfunctions instead of the older pythonprintstatements so if you install it now some of the files may raise an error when weaving (generating the rst files). I don’tthinkthey will since I’m using stdout and stderr directly, not printing, but buyer beware.Pweave itself requires matplotlib if you want to generate the rst files (it’s meant to facilitatereproducible researchand it has some conveniences for creating plots with matplotlib). So if you want to create the rst files you need to install it:pip install matplotlibMatplotlib itself requires numpy and they can take a long time to build. If you’re not into building them just for this and you have a package manager it’s probably better to use it. For debian-based systems:sudo apt-get install python-matplotlibSince it’s a sphinx repository you’ll of course need sphinx:pip install sphinxI’m using thealabastersphinx theme so you need that too:pip install alabasterAt the moment I’m usingsphinxcontrib-plantumlfor the UML diagrams. If you want them you’ll need both plantuml and the sphinx extension:apt-get install plantuml
apt-get install graphviz
pip install sphinxcontrib-plantumlI think that’s everything. If it is, then when you’re next to theMakefileyou can build the (html) documentation with:make htmlAnd it will output the html files todoc/html.NoteIrsyncthe html files to a folder where my apache server can see it so if you build it before I remember to get rid of that in the Makefile you will likely see an error but that shouldn’t affect the built documentation (although if just by coincidence you have the exact same file structure as me it might clobber something (sorry)).Running the TestsI was originally using python’sunittestmodule but I’ve switched over to usingbehaveandpyhamcrestso the tests at the top of the package neednosetestsand the tests in theiperflexer/testsfolder needbehaveandPyHamcrest <https://pypi.python.org/pypi/PyHamcrest>.