Generate local kernel and UST indexes

When the consumer creates a new trace file, it now creates as well an
index file that the viewers can use to navigate in the trace.  For now
these indexes are only local and not sent to the relayd.

This patch requires additionnal ioctl and ustctl calls implemented in
lttng-modules and lttng-ust in 2.4.

For now, no indexes are sent over the network and no indexes are
generated on snaphots.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: David Goulet <dgoulet@efficios.com>
Bump version to 2.4.0-pre1

We need to bump the version early in the development cycle since we
extend the relayd protocol.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: David Goulet <dgoulet@efficios.com>
Live timer set up

Add a new option to create to enable the live_timer being --live at
session creation. A new API call is added also to create as session in
live mode like the snapshot feature.

This timer is responsible to check all streams of a live session and
make sure the viewer receives periodic updates even for low-throughput
channels.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: David Goulet <dgoulet@efficios.com>
HT support for keys with two uint64_t

Add the support in the hashtable abstraction layer to handle a key
composed of a two uint64_t.

The hash function is a simple XOR between the two hashes and the compare
function resolves the eventual collisions by comparing the two values.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: David Goulet <dgoulet@efficios.com>
Send indexes in streaming mode

To write an index on disk, we need to know all the fields sent by the
RELAY_SEND_INDEX command as well as the offset of the data in the
tracefile.

Since the control and data connection are not synchronized, this process
happens in two separate steps synchronized by an HT indexed by the
index_handle and the net_seq_num.

When we receive data and when we receive an index, we lookup in the HT
if an entry already exists. If it does, it means that we only need to
fill the fields we just received and write the index on disk, otherwise,
we allocate a new index, set the fields we know and store it in the HT.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
Signed-off-by: David Goulet <dgoulet@efficios.com>
Fix: return -errno value on error in run_as mkdir

Signed-off-by: David Goulet <dgoulet@ev0ke.net>
HT add_str (not unique)

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] Basic handling of the live timer

Make the metadata timer thread more generic, we now handle the live
timer in here as well.
Introduce the new LTTNG_CONSUMER_SIG_LIVE, for now handled as a simple
printf.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] Create relayd session from the sessiond

Move the relayd session creation from the consumer to the relayd. No
change in behaviour expected here, just prepare for the next patch to
allow the sessiond to pass additionnal parameters in the relayd session
creation required for live tracing (hostname, session name, flags, etc).

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] Create relayd session with live parameters

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] Implement the relayd live features

Relayd listen on 2 additionnal ports
Version exchange with the viewer
List the sessions
Attach to a session
List the streams of a session
Get the indexes per stream
Get data packets
Flag metadata streams
Match ctf_traces based on the trace path
Handle the synchronisation of the metadata (prevent the viewer from
reading a packet if it does not have the complete metadata)
Send the metadata
Ioctl to get the current timestamp
Inactive stream beacon

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] REVIEW CAREFULLY : test for new metadata at each packet

After sending each data packet in live, we need to check if new metadata
is available before sending the index informing the viewer it can read
the trace.
Since the data and the metadata are handled by two different threads,
this patch introduces a rendez-vous point : if new metadata is
available, the data thread flushes the metadata stream and waits on a
conditionnal variable. When the metadata thread finishes to send its
data, it wakes up the data thread which can send its index.

That way, the viewer is informed new metadata is available before
attempting to read a packet that might require an update of the
metadata.

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>
[HEAD MERGE] relicense lttng-index and lttng-viewer to BSD

Signed-off-by: Julien Desfossez <jdesfossez@efficios.com>