Delta

==Delta==
If delta is enable for this packet the packet-payload (after the 3
byte packet-header) is followed by the delta-header. The delta-header
is a bitvector which represents all non-key fields of the packet. If
the field has changed the corresponding bit is set and the field
value is so included in delta-body. The values of the unchanged fields
will be filled in from an old version at the receiving side.
For bool field another optimization called bool-header-folding is
applied. Instead of sending an indicator in the bitvector if the given
bool values has changed (and so using 1 byte for the real value) the
actual value of the bool is transfered in the bitvector bit of this
bool field.
Another optimization called array-diff is used to reduce the amount of
elements transfered if an array is changed. This is independent of the
delta-header bit i.e. it will only be used if the array has changed
its value and the bit indicates this. Instead of transferring the
whole array only a list of (index,new value of this index) pairs are
transferred. The index is 8bit and the end of this pair list is
denoted by an index of 255.
==Compression==
To further reduce the network traffic the (delta) packets are
compressed. To get better compression results multiple packets are
grouped together and compressed into a chunk. This chunk is then
transfered as a normal packet. A chunk packet start with the 2 byte
length field which every packet has. A chunk packet has no type. A
chunk packet is identified by having a too large length field. If the
length of the packet is over COMPRESSION_BORDER it is a chunk
packet. It will be uncompressed at the receiving side and re-feed into
the receiving queue.
If the length of the chunk packet can't be expressed in the available
space of the 16bit length field (>48kb) the chunk is sent as a jumbo
packet. The difference between a normal chunk packet and a jumbo chunk
packet is that the jumbo packet has JUMBO_SIZE in the size field and
has an additional 4 byte len field after the 2 byte len field. The
second len field contains the the size of the whole packet (2 byte
first length field + 4 byte second length field + compressed data).
Packets are grouped for the compression based on the
PACKET_PROCESSING_STARTED/PACKET_PROCESSING_FINISHED and
PACKET_FREEZE_HINT/PACKET_THAW_HINT packet pairs. If the first
(freeze) packet is encountered the packets till the second (thaw)
packet are put into a queue. This queue is then compressed and sent as
a chunk packet. If the compression would expand in size the queued
packets are sent uncompressed as "normal" packets.
The compression level can be controlled by the
FREECIV_COMPRESSION_LEVEL environment variable.
==Files==
There are four file/filesets involved in the delta protocol:
# the definition file (common/packets.def).
# the generator (common/generate_packets.py).
# the generated files (*/*_gen.[ch] or as a list client/civclient_gen.c, client/packhand_gen.h, common/packets_gen.c, common/packets_gen.h, server/hand_gen.h and server/srv_main_gen.c).
# the overview (README.delta, this file)
The definition file lists all valid packet types with their
fields. The generator takes this as input and creates the generated
files.
For adding and/or removing packets and/or fields you only have to
touch the definition file. If you however plan to change the generated
code (adding more statistics for example) you have to change the
generator.
==Changing the definition file==
Adding a packet:
# choose an unused packet number. The generator will make sure that you don't use the same number two times.
# choose a packet name. It should follow the naming style of the other packets: PACKET_<group>_<remaining>. <group> may be SERVER, CITY, UNIT, PLAYER, DIPLOMACY and so on.
# decide if this packet goes from server to client or client to server
# choose the field names and types
# choose packet and field flags
# write the entry into the corresponding section of common/packets.def
If you add a field which is a struct (say "foobar") you have to write
the following functions: dio_get_foobar, dio_put_foobar and
are_foobars_equal.
Removing a packet:
# add a mandatory capability
# remove the entry from common/packets.def
Adding a field:
: Option A:
:# add a mandatory capability
:# add a normal field line:
::: COORD x
: Option B:
:# add a non-mandatory capability (say "new_version")
:# add a normal field line containing this capability in an add-cap flag:
::: COORD x; add-cap(new_version)
Removing a field:
: Option A:
:# add a mandatory capability
:# remove the corresponding field line
: Option B:
:# add a non-mandatory capability (say "cleanup")
:# add to the corresponding field line a remove-cap flag
After changing the definition file the generator has to be run. The
common/Makefile will take care of this. You don't need to run
autoconf/automake/configure.
==Capabilities and variants==
The generator has to generate code which supports different
capabilities at runtime according to the specification given in the
definitions with add-cap and remove-cap. The generator will find the
set of used capabilities for a given packet. Lets say there are two
fields with "add-cap(cap1)" and one field with an "remove-cap(cap2)"
flag. So the set of capabilities are cap1, cap2. At runtime the
generated code may run under 4 different capabilities:
* neither cap1 nor cap2 are set
* cap1 is set but cap2 isn't
* cap1 is not set but cap2 is
* cap1 and cap2 are set
Each of these combinations is called a variant. If n is the number of
capabilities used by the packet the number of variants is 2^n.
For each of these variant a seperate send and receive function will be
generated. The variant for a packet and a connection are calculated
once and then saved in the connection struct.
Category:Technical docs