netlink: Fix up link down behaviour

Partially reverts 04f92904ea6c41517ff7154910c16ef4c3bc646b
 "userv-ipif: Always request routes from userv, regardless of link quality"

It turns out that this check is necessary to avoid bringing up a route
for a "netlink" stanza in the configuration file which is never used.

In particular, this avoids bringing up a netlink for (a) sites which
are not mentioned in the config file (b) the site on which secnet is
running.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: Fix userv invocation after pfilepath

The commit 9b8369e07aeba5ed2c69fb4a7f74d07c8cebe015
 make-secnet-sites: refactor to break out new function "pfilepath"
broke the userv service invocation, because it turned out that later
code depended on the "headerinput" variable whose assignment had been
removed and replaced by a call to pfilepath.

Make pfilepath return the lines read from the file and assign the
result to headerinput.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: Allow sites with no address

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: New -P <prefix> option

make-secnet-sites generates a config file which defines the keys
"vpn-data", "vpn", and "all-sites".

To make it possible to usefully include the output of more than one
different piece of output from make-secnet-sites, support the option
-P <prefix>, which generates a config file which defines
"<prefix>vpn-data", "<prefix>vpn", and "<prefix>all-sites".

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: If definition found in wrong place, bomb out

If it doesn't exit here, make-secnet-sites might subsequently crash
with Python stack backtrace.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: Actually include addresses in sites.conf

This was broken in

   commit c9d84e31f48a04eb0af525324707f22c738cc8b7
   Author: Ian Jackson <ijackson@chiark.greenend.org.uk>
   Date:   Sat Dec 17 21:21:47 2011 +0000

       make-secnet-sites: Allow sites with no address

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: Do newline-trimming in pline()

This will make life a little easier in a future patch.  While we're at
it, use rstrip rather than open-coding it.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: In -u mode, output file "dereferences" include directives

Whenm make-secnet-sites is writing the "output" sites file in -u
(groupfile update) mode, it includes the effective contents of files
referenced in "include" directives, rather than the "include"
directive itself.

So the "output" sites file does not any longer depend on any files
included by the header.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
make-secnet-sites: Do not permit "include" in simple sites files

Restrict the "include" directive to the "header" of -u (groupfile
update) mode.  Callers who are simply using make-secnet-sites to
transform a (possibly untrusted) sites file into a (to be trusted)
sites.conf file should not have to worry about includes.

"include" directives are already forbidden in group files.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
build system: Update build-dependency information

Update build-dependency documentation and add missing references to
flex and bison to debian/control.

Signed-off-by: Matthew Vernon <matthewv@chiark.greenend.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
userv-ipif: Always request routes from userv, regardless of link quality

Previously the userv-ipif netlink would not request (from userv) a
route for a site for which the link quality was DOWN.  The link
quality is a dynamic quantity but userv-ipif lacks any machinery for
dynamically adding routes, so this is wrong.

Instead, in userv-ipif, unconditionally add routes for all sites,
regardless of link up status.

In practice this code is run during startup and the only reason a link
might be down at that point, ie LINK_QUALITY_DOWN, is that it does not
have an address configured.  Mobile sites are often in this situation.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
SECURITY: actually reject messages with improper lengths

transform_reverse checks to see if the incoming message is a multiple
of the block cipher block size.  If the message fails this check, it
sets *errmsg but it fails to return.  Instead, it proceeds to attempt
to decrypt it.  This will involve a buffer read/write overrun.

It transform_reverse fails to check to see if the incoming message is
long enough to contain the IV, MAC and padding.  This can easily be
exploited to cause secnet to segfault.

buffer_unprepend should check that the buffer has enough data in it to
unprepend.  With the other patches, this should be irrelevant, but it
turns various other potential read overrun bugs into crashes.

site_incoming should check that the incoming message is long enough.
Again, without this, this is an buffer read overrun or possibly a
segfault.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Makefile: honour EXTRA_CFLAGS, etc.

This makes it easier to add command-line options at build-time.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
log: Eliminate potential out-of-control recursion

vMessage looks for the system log instance.  If there is one, it uses
it; otherwise it just uses stderr or stdout.

The system log instance consists, eventually, of syslog_vlog or
logfile_vlog.  Both of these functions have a fallback: if they are
properly set up they do their thing; otherwise they call vMessage.
This is wrong because in this situation they have probably been called
by vMessage so it doesn't help.

At first sight this ought to produce unbounded recursion but for
complicated reasons another bug prevents this.  Instead, messages can
just vanish.

Break out the fallback mode of [v]Message into [v]MessageFallback so
that syslog_vlog and logfile_vlog can use it directly.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
log: Print truncated messages

If a message doesn't survive the vsnprintf in vMessage untruncated,
the result may be that its newline is lost.  After that, buff will
never manage to gain a newline and all further messages will be
discarded.

Fix this by always putting "\n\0" at the end of buff.  That way a
truncated message is printed and the buffer will be cleared out for
the next call.

This does mean that sometimes a message which is just slightly shorter
than the buffer will produce a spurious empty message sent to the
syslog.  However, the whole logging approach here will inevitably
occasionally insert spurious newlines, if only to split up long
messages; long messages may also be truncated.  The bad effect of the
change in this patch is simply to make the length at which
infelicities appear very slightly shorter.  The good effect is that it
turns a very nasty failure mode into a benign one.

So I think it is reasonable to err on the side of code which clearly
cannot go wrong in a bad way.  Other approaches which make the
infelicity threshold a couple of characters longer, or which slightly
reduce the different kinds of infelicity, are much more complicated
and therefore more likely to have bugs.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
messages: add some missing newlines

Message and cfgfatal must be called with a message containing a
newline (or, a newline sent in a later call).  Fix a few call sites.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
site: transport peers: fix incorrect stride when debug output enabled

When there are multiple peer addresses, attempts to copy them from one
table of peers addresses to another with transport_peers_copy will go
wrong because the stride argument to transport_peers_debug is wrong -
we take sizeof() the pointer rather than of the array element.

This will typically cause a segfault if it happens, but the bug can
only be triggered if LOG_PEER_ADDRS debugging is enabled.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
netlink: report why a packet is bad

Replace
 <netlink>: bad IP packet from <source>
with
 <netlink>: bad IP packet from <source>: reason

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
netlink: abolish check_config and output_config

Apparently, a long time ago, MSG5 and MSG6 used to contain some
netlink configuration data, which the receiver of the MSG5 or MSG6
would check.

However, for a long time now the output_config function has been a
no-op and the check function has unconditionally eaten and discarded
anything extra in the message.

Furthermore, because the MSG6 is not retransmitted, this mechanism
couldn't be reliable without a protocol change.  So the existing
interface is defective.

So, abolish it the interface, the dummy implementation, and all the
call sites.  The check_config call sites in site.c now instead
directly discard any unexpected data at the end of MSG5 and MSG6.

This patch should cause no behavioural change in actual operation.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
site: Break out separate function for decrypting msg0

The control flow here is going to become more complicated, and this
change will make the next patches, and the resulting code, clearer.

Note that process_msg0's return value is never used; it is only
defined to return bool_t so that it can use the CHECK_AVAIL macro.
Knowing this will make it easier to see that the patch is correct.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>