Clean up lib/order_book.c

  - Use unnecessary NULL assignments from ob_init()

  - Remove unnecessary return statement in ob_fini()

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Per-session dialect support

This adds support for using different FIX dialect per session at
runtime.

The current implementation allows dialects to specify tag types but can
be extended to support session level quirks, custom message headers,
custom message types, and message tag validation in the future.

Current generic FIX dialects are available as "fix_dialects" array
that's indexed by "enum fix_version" values.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: IEX dialect version 1.2

Add support for IEX FIX version 1.2 as specified in:

  https://s3.amazonaws.com/iextrading/docs/IEX+FIX+Specification.pdf

Please note that this patch also introduces a code generator that
simplifies introducing new FIX dialects to libtrading. The use of the
code generator is optional and you can also implement dialects manually.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
tools/fix: List protocols in usage message

Make fix_client and fix_server more user-friendly by listing supported
protocols in the usage message.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
tools/fix: Configurable SenderCompID and TargetCompID

Make SenderCompID and TargetCompID configurable with the "-s" and "-t"
command line options.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Remove test-fix-client and test-fix-server

This patch introduces a single version of FIX client and server
that are based on libtrading API. These are tools/fix/fix_client
and tools/fix/fix_server respectively. Both operate in multiple
modes.

A client operates in three possible modes (this could be changed
in the future):
* Running test scripts (-m 0 or -m script) (default option)
  $ tools/fix/fix_client -m 0 -h ip -p port -f script
* Sending session level messages (-m 1 or -m session)
  $ tools/fix/fix_client -m 1 -h ip -p port -s sender -t target
* Measuring an average latency of executions reports (-m 2 or -m order)
  $ tools/fix/fix_client -m 2 -h ip -p port -s sender -t target -n orders
There are only two mandatory options: FIX server host and port.
All other options are optional. If not specified, sender-comp-id
is set to "BUYSIDE" and target-comp-id is set to "SELLSIDE".

A server operates in two possible modes (this could be changed
in the future):
* Running test scripts (-m 0 or -m script) (default option)
 $ tools/fix/fix_server -m 0 -p port -f script
* Normal mode (-m 1 or -m session)
 $ tools/fix/fix_server -m 1 -p port
There are only one mandatory option, namely port (-p).
All other options are optional. If not specified, sender-comp-id
is set to "SELLSIDE" and target-comp-id is set to "BUYSIDE".

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Round-trip time reporting improvements

Improve round-trip time reporting:

  $ tools/fix/fix_server -m 1 -p 7070

  $ tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000
  Client Logon OK
  Messages sent: 100000
  Round-trip time: min/avg/max = 28.0/35.6/447.0 μs
  Client Logout OK

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Fix fix_client link error

Reported by Marat:

  tools/fix/fix_client.o: In function `fix_client_order':
  /home/maratrus/libtrading/tools/fix/fix_client.c:317: undefined reference to `fmin'
  /home/maratrus/libtrading/tools/fix/fix_client.c:318: undefined reference to `fmax'
  collect2: ld returned 1 exit status

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Fix link breakage #2

Commit bd207bb ("FIX tools: Fix fix_client link error") added "-lm" to
the server, not to the client which is the one that needs it.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Fix spaces vs. tabs formatting

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: Make a client print incoming messages

Print incoming messages in FIX client session mode.

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: latency file output

Make it possible to store latency data in a file for further analysis.

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX tools: a script to plot frequency distribution

This is a simple script to plot a histogram given the latency data stored in a
file. You can use it as follows:

$ ./tools/fix/fix_client -n 1000 -o out.csv -m order -p 9000 -h localhost
Client Logon OK
Messages sent: 1000
Round-trip time: min/avg/max = 101.0/188.9/14801.0 μs
Client Logout OK

$ gnuplot -e "file='out.csv'" -e "width=10" -e "max=1000" tools/fix/frequency.gp

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
[ penberg: Use "round-trip time" for the x-axis label. ]
Signed-off-by: Pekka Enberg <penberg@iki.fi>
Makefile: Use ctags instead of exuberant-ctags

Ctags is available on most systems, exuberant-ctags is not.

Signed-off-by: Pekka Enberg <penberg@iki.fi>
README updates

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Performance measuring statistics

On Linux systems SystemTap infrastructure can be used
to gather information of a running system. This patch
adds probes to a few places withing libtrading FIX
engine. A few examples show how to assess FIX engine
performance. In particular, we calculate the precise
amount of time spent on parsing messages and time spent
on sending or receiving data through the net.

The interface used can be extended to the systems with
other than SystemTap infrastructure e.g. dtrace. Probes
can also be switched off if there is no way to use any
technique of the above.

By default probes are switched off.

Usage example:

 $ sudo stap -c "./tools/fix/fix_client -m order -n 1000 -h localhost -p 9000" ./tools/fix/fix_client.stp -o out.txt
 $ cat out.txt | awk -f './tools/fix/fix_client.awk'
 fix_message_unparse_time_ret:      13479
 fix_message_unparse_ret:            1002
 fix_message_unparse_time_err:          0
 fix_message_unparse_err:               0
 fix_message_parse_time_ret:         8646
 fix_message_parse_ret:              1002
 fix_message_parse_time_err:         2292
 fix_message_parse_err:              1092
 fix_message_recv_time_ret:        102910
 fix_message_recv_ret:               1002
 fix_message_recv_time_err:          4111
 fix_message_recv_err:                 45
 fix_message_send_time_ret:         52249
 fix_message_send_ret:               1002
 fix_message_send_time_err:             0
 fix_message_send_err:                  0

Signed-off-by: Marat Stanichenko <mstanichenko@gmail.com>
Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Use non-blocking recv()

Switch to non-blocking recv() in FIX receive path.

Running the FIX engine latency tests:

  $ while true; do taskset -c 0 tools/fix/fix_server -m 1 -p 7070; done

  $ while true; do time taskset -c 2 tools/fix/fix_client -m 2 -h localhost -p 7070 -n 100000; done

Shows 30% reduction in round-trip times:

  Before:

    Round-trip time: min/avg/max = 30.0/31.9/2159.0 μs
    Round-trip time: min/avg/max = 30.0/31.4/458.0 μs
    Round-trip time: min/avg/max = 29.0/31.3/880.0 μs
    Round-trip time: min/avg/max = 29.0/31.3/2737.0 μs
    Round-trip time: min/avg/max = 29.0/31.2/1035.0 μs
    Round-trip time: min/avg/max = 30.0/31.3/973.0 μs

  After:

    Round-trip time: min/avg/max = 22.0/22.9/451.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/450.0 μs
    Round-trip time: min/avg/max = 22.0/22.6/448.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/451.0 μs
    Round-trip time: min/avg/max = 22.0/22.5/443.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/442.0 μs

Signed-off-by: Pekka Enberg <penberg@iki.fi>
Add timespec_diff() API and use it

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Optimize fix_field_unparse() function

Micro-optimize fix_field_unparse() to save 2 μs on average:

  Before:

    Messages sent: 100000
    Round-trip time: min/avg/max = 22.0/22.9/451.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/450.0 μs
    Round-trip time: min/avg/max = 22.0/22.6/448.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/451.0 μs
    Round-trip time: min/avg/max = 22.0/22.5/443.0 μs
    Round-trip time: min/avg/max = 22.0/22.4/442.0 μs

  After:

    Messages sent: 100000
    Round-trip time: min/avg/max = 19.0/21.1/434.0 μs
    Round-trip time: min/avg/max = 19.0/20.3/444.0 μs
    Round-trip time: min/avg/max = 19.0/20.2/444.0 μs
    Round-trip time: min/avg/max = 19.0/20.3/443.0 μs
    Round-trip time: min/avg/max = 19.0/20.2/434.0 μs
    Round-trip time: min/avg/max = 19.0/20.2/434.0 μs

The vfprintf() function drops from 6% down to 3.86% in 'perf report':

  Before:

     6.80%  fix_client  [kernel.kallsyms]     [k] _raw_spin_lock_bh
     6.15%  fix_client  libc-2.17.so          [.] vfprintf
     3.43%  fix_client  libc-2.17.so          [.] __libc_recv
     3.30%  fix_client  [kernel.kallsyms]     [k] ipt_do_table
     3.15%  fix_client  [kernel.kallsyms]     [k] sock_recvmsg
     2.94%  fix_client  [kernel.kallsyms]     [k] tcp_recvmsg
     2.56%  fix_client  [kernel.kallsyms]     [k] __audit_syscall_exit
     2.56%  fix_client  [kernel.kallsyms]     [k] sock_has_perm
     2.39%  fix_client  [kernel.kallsyms]     [k] system_call
     2.14%  fix_client  [kernel.kallsyms]     [k] avc_has_perm_flags
     2.09%  fix_client  libc-2.17.so          [.] __printf_fp
     2.00%  fix_client  [kernel.kallsyms]     [k] SYSC_recvfrom
     1.97%  fix_client  [kernel.kallsyms]     [k] system_call_after_swapgs
     1.96%  fix_client  libc-2.17.so          [.] _IO_default_xsputn
     1.60%  fix_client  fix_client            [.] fix_message_parse

  After:

     6.19%  fix_client  [kernel.kallsyms]     [k] _raw_spin_lock_bh
     4.70%  fix_client  [kernel.kallsyms]     [k] ipt_do_table
     3.86%  fix_client  libc-2.17.so          [.] vfprintf
     3.49%  fix_client  libc-2.17.so          [.] __printf_fp
     2.93%  fix_client  fix_client            [.] fix_field_unparse
     2.40%  fix_client  libc-2.17.so          [.] __GI_____strtoll_l_internal
     2.37%  fix_client  [kernel.kallsyms]     [k] sock_recvmsg
     2.12%  fix_client  libc-2.17.so          [.] __libc_recv
     2.03%  fix_client  [kernel.kallsyms]     [k] tcp_recvmsg
     2.01%  fix_client  libc-2.17.so          [.] __GI_____strtod_l_internal
     1.99%  fix_client  libc-2.17.so          [.] __strftime_internal
     1.75%  fix_client  fix_client            [.] fix_message_parse

Signed-off-by: Pekka Enberg <penberg@iki.fi>
FIX: Switch to sendmsg()

  Before:

    Messages sent: 100000
    Round-trip time: min/avg/max = 19.0/20.4/156.0 μs
    Round-trip time: min/avg/max = 19.0/19.5/61.0 μs
    Round-trip time: min/avg/max = 19.0/19.5/86.0 μs
    Round-trip time: min/avg/max = 19.0/19.4/72.0 μs
    Round-trip time: min/avg/max = 19.0/19.3/70.0 μs
    Round-trip time: min/avg/max = 19.0/19.4/78.0 μs

  After:

    Messages sent: 100000
    Round-trip time: min/avg/max = 18.0/19.6/118.0 μs
    Round-trip time: min/avg/max = 18.0/18.9/77.0 μs
    Round-trip time: min/avg/max = 18.0/18.9/64.0 μs
    Round-trip time: min/avg/max = 18.0/18.8/65.0 μs
    Round-trip time: min/avg/max = 18.0/18.8/64.0 μs
    Round-trip time: min/avg/max = 18.0/18.8/75.0 μs

  Before:

     6.29%  fix_client  [kernel.kallsyms]     [k] _raw_spin_lock_bh
     4.50%  fix_client  [kernel.kallsyms]     [k] ipt_do_table
     3.14%  fix_client  libc-2.17.so          [.] __libc_recv
     2.90%  fix_client  [kernel.kallsyms]     [k] sock_recvmsg
     2.69%  fix_client  [kernel.kallsyms]     [k] avc_has_perm_flags
     2.48%  fix_client  [kernel.kallsyms]     [k] tcp_recvmsg
     2.21%  fix_client  libc-2.17.so          [.] vfprintf
     2.19%  fix_client  fix_client            [.] fix_field_unparse
     2.13%  fix_client  [kernel.kallsyms]     [k] __audit_syscall_exit
     2.08%  fix_client  [kernel.kallsyms]     [k] system_call
     1.95%  fix_client  libc-2.17.so          [.] __printf_fp
     1.88%  fix_client  libc-2.17.so          [.] __GI_____strtoll_l_internal
     1.86%  fix_client  [kernel.kallsyms]     [k] sock_has_perm
     1.78%  fix_client  [kernel.kallsyms]     [k] SYSC_recvfrom
     1.67%  fix_client  libc-2.17.so          [.] __GI_____strtod_l_internal
     1.65%  fix_client  [kernel.kallsyms]     [k] system_call_after_swapgs
     1.62%  fix_client  fix_client            [.] fix_message_parse
     1.58%  fix_client  fix_client            [.] buffer_find
     1.39%  fix_client  [kernel.kallsyms]     [k] local_bh_enable
     1.39%  fix_client  [kernel.kallsyms]     [k] nf_iterate
     1.35%  fix_client  libc-2.17.so          [.] __strftime_internal

  After:

    10.69%  fix_client  [kernel.kallsyms]     [k] _raw_spin_lock_bh
     5.32%  fix_client  libc-2.17.so          [.] __libc_recv
     4.91%  fix_client  [kernel.kallsyms]     [k] sock_recvmsg
     3.96%  fix_client  [kernel.kallsyms]     [k] tcp_recvmsg
     3.85%  fix_client  [kernel.kallsyms]     [k] avc_has_perm_flags
     3.47%  fix_client  [kernel.kallsyms]     [k] sock_has_perm
     3.41%  fix_client  [kernel.kallsyms]     [k] __audit_syscall_exit
     3.04%  fix_client  [kernel.kallsyms]     [k] system_call
     2.69%  fix_client  [kernel.kallsyms]     [k] system_call_after_swapgs
     2.63%  fix_client  [kernel.kallsyms]     [k] fget_light
     2.56%  fix_client  [kernel.kallsyms]     [k] SYSC_recvfrom
     2.43%  fix_client  fix_client            [.] fix_message_parse
     2.13%  fix_client  [kernel.kallsyms]     [k] __audit_syscall_entry
     2.06%  fix_client  [kernel.kallsyms]     [k] lock_sock_nested
     1.91%  fix_client  [kernel.kallsyms]     [k] release_sock
     1.79%  fix_client  [kernel.kallsyms]     [k] ipt_do_table
     1.79%  fix_client  [kernel.kallsyms]     [k] local_bh_enable_ip
     1.63%  fix_client  [kernel.kallsyms]     [k] local_bh_enable
     1.58%  fix_client  fix_client            [.] fix_session_recv

Signed-off-by: Pekka Enberg <penberg@iki.fi>