bitcoin/bitcoin — pull request #2479 by brandondahler (2013-04-07 06:24:57+00:00)
Title: Allow capping of bandwidth on overall or command-specific basis, reset via cron-like expression

This is a first-pass change to begin resolving issues and #273, #1958, #2412. I doubt this change completely ready, I intend on using this request for feedback on the current implementation of code and usage.

____

## Basic
Allow users to set bandwidth limits on overall egress and/or command-specific egress bandwdith. Overall egress bandwidth and command-specific egress bandwidth counts are saved to disk (currently every packet queued), to allow counts to be saved across runnings of the bitcoin node. A simplified cron-expression is used to determine when to reset all bandwidth counts.

The cron-expression will cause start and reset times to "snap" to the closest values above and below the current time. This allows exact control over when the bandwidth usage is reset via the expression, instead of relying on the time the program was last started or other arbitrary limitations.

____

## Configuration options
For command-line (prepend '-') or configuration file:
* bandwidthsentmax - Overall egress bandwidth limit
* bandwidthsentmax_[command] - Command-specific bandwidth limit
* bandwidthresettimes - Cron-expression used to reset bandwidth counts

____

## Formats
* Bandwidth - As regular expression /[0-9]+[BbKkMmGgTtPp]?[Bb]?/ - Valid for any values 0 - 18,446,744,073,709,551,616 bytes (16,384 Peta-bytes)
* Cron - See [wiki](http://en.wikipedia.org/wiki/Cron#CRON_expression) for full cron expressions. Numerical representations allowed only. Special characters implemented: asterisk (*), slash (/) (with caveats), comma (,), and hyphen (-). Slash requires the end value be landed upon (which means */4 requires the possible range be divisible by 4).

____

## Sample configuration
Bitcoin.conf
````
# minutes hours dayOfMonth Month dayOfWeek
# Resets each month
# 0 0 1 * *
#
# Reset every sunday
# * * * * 0
# Resets each minute
# * * * * *
#
# Resets every 5 minutes
# */5 * * * *
bandwidthresettimes=0 0 0 * *

# Only allow 2 GByte of sent data
bandwidthsentmax=2G

# Only send out 1 GByte of sent "block" commands
bandwidthsentmax_block=1G
````

____

## Files this change-set:
### New
#### bandwidthman.h and bandwdithman.cpp
* New class to handle bandwidth management as needed, will serve a larger purpose as bandwidth management features are expanded.
* Public functions
* LoadConfiguration() - Uses mapArgs to read reset time cron expression, uses mapArgs to read overall bandwidth cap and specific command bandwidth caps.
* TestScheduling(string strCronLine, struct tm tmTestTime) - Parses a given cron expression and uses a specific time to calculate start time and reset time. Used only by the unit tester.
* AllowSendCommand(string strCommand, int nSize) - Determines if strCommand should be sent, given the command is nSize.

#### tests/bandwidthman_tests.cpp
* Contains unit tests for the calculation of start times and reset times used by the CBandwidthMan class.

### Changed
#### db.h
* Rename CAddrDB to CDatDB and make abstract, generalize specific functions to globally applicable functions, consider the general format used by CAddrDB as being a ".dat" file.
* Create templated wrapper for new CDatDB named CSerialDatDB. Extends CDatDB and allows for any serializable type to be used as the template type.
* Create CAddrDB by making it an alias of CSerialDatDB, construct the CSerialDatDB with the static filename of "peers.dat" (as previously hard-coded).
* Create CBandwidthDB by making it an alias of CSerialDatDB, construct CSerialDatDB with the static filename of "bandwidth.dat".

Note: All changes to CAddrDB result in no usability changes.

#### db.cpp
* Change CAddrDB functions to CDatDB functions.
* Modify write process to prepend message start and append hash due to change in serialization time of payload data (new lines 488-494).

#### init.cpp
* Insert new step 11: Load bandwidth manager.
* Renumber old steps 11 and 12 to 12 and 13 respectively.

#### makefile.*
* Add objs/bandwidthman.o as new object

#### net.h
* Export instance of bandwidth manager.
* Modify EndMessage to require the message name upon invocation and return success or failure to send message.
* EndMessage will return false if it is not allowed to send the message as determined by the bandwidth manager.
* If bandwidth manager does not allow the message to be sent, abort message.
* Modify all push messages to return success or failure, determined by EndMessage call.
* Success/failure result is not currently used but planned for use later.

#### net.cpp
* Add new global instance of CBandwidthMan.

____

## Extra data
While researching how to approach the problem, I collected command-specific egress data off of an unrestricted node that was on the latest block. [This spreadsheet](https://docs.google.com/spreadsheet/ccc?key=0AnpjY4_hVstbdHpqeVRXeFZDM1lydmdYdndaWGlLZXc&usp=sharing) shows the count and total bandwidth of each of the commands. I do not remember when or how long it took to collect this data, but I believe it was over a few hours with no local usage of the computer. The branch at [brandondahler/bitcoin branch bandwidth-test](https://github.com/brandondahler/bitcoin/tree/bandwidth-test) can produce a text file that can aid in plugging in your own data into this spreadsheet.