fix_message: implement fix_message_add_field; fix_get_field_{count,at}
Merge pull request #46 from StefanKarpinski/sk/fix_messages

fix_message: implement fix_message_add_field; fix_get_field_{count,at}
Micro-optimize buffer_find()

The buffer_find() function is used heavily by the FIX message parsing
code.  Micro-optimize the function by inlining it and rewriting the loop
to optimize for the case where the searched character is found. FIX
latency tests suggest that this reduces average latency by 1 usec in the
best case.

Before:

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 15.0/17.1/447.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1789.153872      task-clock (msec)         #    1.001 CPUs utilized
                  11      context-switches          #    0.006 K/sec
                   0      cpu-migrations            #    0.000 K/sec
                 606      page-faults               #    0.339 K/sec
       5,430,616,385      cycles                    #    3.035 GHz                     [83.32%]
       2,570,009,095      stalled-cycles-frontend   #   47.32% frontend cycles idle    [83.32%]
       1,613,862,841      stalled-cycles-backend    #   29.72% backend  cycles idle    [66.71%]
       6,878,023,152      instructions              #    1.27  insns per cycle
                                                    #    0.37  stalled cycles per insn [83.38%]
       1,278,707,409      branches                  #  714.700 M/sec                   [83.38%]
           6,080,104      branch-misses             #    0.48% of all branches         [83.27%]

         1.787487438 seconds time elapsed

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 16.0/17.3/435.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1801.566117      task-clock (msec)         #    1.001 CPUs utilized
                  13      context-switches          #    0.007 K/sec
                   0      cpu-migrations            #    0.000 K/sec
                 607      page-faults               #    0.337 K/sec
       5,450,041,559      cycles                    #    3.025 GHz                     [83.33%]
       2,629,636,390      stalled-cycles-frontend   #   48.25% frontend cycles idle    [83.33%]
       1,664,278,774      stalled-cycles-backend    #   30.54% backend  cycles idle    [66.70%]
       6,786,017,796      instructions              #    1.25  insns per cycle
                                                    #    0.39  stalled cycles per insn [83.38%]
       1,261,948,159      branches                  #  700.473 M/sec                   [83.38%]
           6,101,997      branch-misses             #    0.48% of all branches         [83.27%]

         1.799871025 seconds time elapsed

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 15.0/17.1/435.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1780.308839      task-clock (msec)         #    1.001 CPUs utilized
                  13      context-switches          #    0.007 K/sec
                   2      cpu-migrations            #    0.001 K/sec
                 606      page-faults               #    0.340 K/sec
       5,421,903,018      cycles                    #    3.045 GHz                     [83.35%]
       2,601,154,407      stalled-cycles-frontend   #   47.97% frontend cycles idle    [83.35%]
       1,629,581,224      stalled-cycles-backend    #   30.06% backend  cycles idle    [66.71%]
       6,789,101,246      instructions              #    1.25  insns per cycle
                                                    #    0.38  stalled cycles per insn [83.36%]
       1,262,191,825      branches                  #  708.974 M/sec                   [83.30%]
           6,106,546      branch-misses             #    0.48% of all branches         [83.30%]

         1.779002274 seconds time elapsed

After:

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 15.0/16.9/436.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1751.156973      task-clock (msec)         #    1.001 CPUs utilized
                  12      context-switches          #    0.007 K/sec
                   1      cpu-migrations            #    0.001 K/sec
                 606      page-faults               #    0.346 K/sec
       5,307,890,373      cycles                    #    3.031 GHz                     [83.31%]
       2,480,901,548      stalled-cycles-frontend   #   46.74% frontend cycles idle    [83.36%]
       1,565,645,741      stalled-cycles-backend    #   29.50% backend  cycles idle    [66.73%]
       6,734,974,617      instructions              #    1.27  insns per cycle
                                                    #    0.37  stalled cycles per insn [83.37%]
       1,252,607,759      branches                  #  715.303 M/sec                   [83.36%]
           5,804,166      branch-misses             #    0.46% of all branches         [83.25%]

         1.749597891 seconds time elapsed

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 15.0/16.3/249.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1699.117595      task-clock (msec)         #    1.001 CPUs utilized
                  11      context-switches          #    0.006 K/sec
                   1      cpu-migrations            #    0.001 K/sec
                 606      page-faults               #    0.357 K/sec
       5,350,388,781      cycles                    #    3.149 GHz                     [83.32%]
       2,514,394,493      stalled-cycles-frontend   #   46.99% frontend cycles idle    [83.32%]
       1,594,173,883      stalled-cycles-backend    #   29.80% backend  cycles idle    [66.71%]
       6,746,425,438      instructions              #    1.26  insns per cycle
                                                    #    0.37  stalled cycles per insn [83.38%]
       1,254,967,462      branches                  #  738.600 M/sec                   [83.38%]
           5,819,099      branch-misses             #    0.46% of all branches         [83.27%]

         1.697614047 seconds time elapsed

  [penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 15.0/16.3/434.0 μs
  Client Logout OK

   Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

         1690.081210      task-clock (msec)         #    1.001 CPUs utilized
                  13      context-switches          #    0.008 K/sec
                   0      cpu-migrations            #    0.000 K/sec
                 607      page-faults               #    0.359 K/sec
       5,315,892,110      cycles                    #    3.145 GHz                     [83.35%]
       2,531,245,591      stalled-cycles-frontend   #   47.62% frontend cycles idle    [83.35%]
       1,600,215,167      stalled-cycles-backend    #   30.10% backend  cycles idle    [66.71%]
       6,650,232,102      instructions              #    1.25  insns per cycle
                                                    #    0.38  stalled cycles per insn [83.36%]
       1,237,037,386      branches                  #  731.940 M/sec                   [83.35%]
           5,842,859      branch-misses             #    0.47% of all branches         [83.24%]

         1.688596264 seconds time elapsed

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Micro-optimize parse_tag()

Before:

[penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
Client Logon OK
Messages sent: 100000
Round-trip time: min/avg/max = 15.0/16.4/226.0 μs
Client Logout OK

 Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

       1693.787672      task-clock (msec)         #    0.998 CPUs utilized
                11      context-switches          #    0.006 K/sec
                 2      cpu-migrations            #    0.001 K/sec
               606      page-faults               #    0.358 K/sec
     5,342,018,512      cycles                    #    3.154 GHz                     [83.32%]
     2,478,820,731      stalled-cycles-frontend   #   46.40% frontend cycles idle    [83.32%]
     1,583,343,832      stalled-cycles-backend    #   29.64% backend  cycles idle    [66.75%]
     6,801,136,779      instructions              #    1.27  insns per cycle
                                                  #    0.36  stalled cycles per insn [83.38%]
     1,288,468,877      branches                  #  760.703 M/sec                   [83.37%]
         6,874,208      branch-misses             #    0.53% of all branches         [83.26%]

       1.696737850 seconds time elapsed

After:

[penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
Client Logon OK
Messages sent: 100000
Round-trip time: min/avg/max = 15.0/15.4/433.0 μs
Client Logout OK

 Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

       1621.309108      task-clock (msec)         #    0.998 CPUs utilized
                10      context-switches          #    0.006 K/sec
                 5      cpu-migrations            #    0.003 K/sec
               607      page-faults               #    0.374 K/sec
     5,097,502,920      cycles                    #    3.144 GHz                     [83.31%]
     2,399,211,788      stalled-cycles-frontend   #   47.07% frontend cycles idle    [83.35%]
     1,528,311,447      stalled-cycles-backend    #   29.98% backend  cycles idle    [66.74%]
     6,353,038,642      instructions              #    1.25  insns per cycle
                                                  #    0.38  stalled cycles per insn [83.37%]
     1,172,638,013      branches                  #  723.266 M/sec                   [83.37%]
         6,673,377      branch-misses             #    0.57% of all branches         [83.25%]

       1.624034802 seconds time elapsed

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: atoi, take #2

Before:

[penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
Client Logon OK
Messages sent: 100000
Round-trip time: min/avg/max = 15.0/15.5/485.0 μs
Client Logout OK

 Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

       1623.392443      task-clock (msec)         #    0.998 CPUs utilized
                12      context-switches          #    0.007 K/sec
                 4      cpu-migrations            #    0.002 K/sec
               605      page-faults               #    0.373 K/sec
     5,092,134,383      cycles                    #    3.137 GHz                     [83.33%]
     2,400,110,182      stalled-cycles-frontend   #   47.13% frontend cycles idle    [83.33%]
     1,514,628,090      stalled-cycles-backend    #   29.74% backend  cycles idle    [66.69%]
     6,314,026,809      instructions              #    1.24  insns per cycle
                                                  #    0.38  stalled cycles per insn [83.39%]
     1,148,801,918      branches                  #  707.655 M/sec                   [83.39%]
         6,626,195      branch-misses             #    0.58% of all branches         [83.27%]

       1.626275156 seconds time elapsed

After:

[penberg@localhost libtrading]$ perf stat taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
Client Logon OK
Messages sent: 100000
Round-trip time: min/avg/max = 14.0/15.4/434.0 μs
Client Logout OK

 Performance counter stats for 'taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000':

       1585.542173      task-clock (msec)         #    0.998 CPUs utilized
                11      context-switches          #    0.007 K/sec
                 3      cpu-migrations            #    0.002 K/sec
               607      page-faults               #    0.383 K/sec
     4,924,298,362      cycles                    #    3.106 GHz                     [83.31%]
     2,272,636,404      stalled-cycles-frontend   #   46.15% frontend cycles idle    [83.33%]
     1,413,842,968      stalled-cycles-backend    #   28.71% backend  cycles idle    [66.75%]
     6,211,535,097      instructions              #    1.26  insns per cycle
                                                  #    0.37  stalled cycles per insn [83.38%]
     1,124,905,511      branches                  #  709.477 M/sec                   [83.37%]
         5,976,068      branch-misses             #    0.53% of all branches         [83.25%]

       1.588288580 seconds time elapsed

Signed-off-by: Pekka Enberg <penberg@iki.fi>
Latest latency numbers

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: password support is added

It is quite natural to provide a password (or passphrase)
to establish a FIX session i.e. in a logon request. If a
particular exchange requires password to be provided we
have to support this

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: MICEX dialect is added

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Cancel request function is added

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: fix_version enumeration ordering

Make "enum fix_version" enumeration ordered by protocol versions in
ascending order so that you can use comparison operators for version
specific logic.

NOTE! This breaks ABI compatibility for applications built against older
versions of libtrading.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Fix EndSeqNo for FIX 4.1 and earlier

The EndSeqNo is specified as follows for FIX 4.1 and earlier:

  Message sequence number of last record in range to be resent. If
  request is for a single record BeginSeqNo = EndSeqNo. If request is for
  all messages subsequent to a particular message, EndSeqNo = "999999"

Fixes #49.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: MsgSeqNum should be ignored in Sequence Reset (Reset mode)

As per FIX specification MsgSeqNum in the header should be
ignored in the Reset mode. Use incoming sequence number instead.

This should fix the issue #48

Reported-by: @Poordeveloper
Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Order Mass Cancel Request and Mass Cancel Report introduced

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>