Pubserver software

=General description=
==Purpose of this document==
This document describes the software, published on Gna!, that was used to drive pubserver, the public set of Freeciv servers that used to run pubserver.freeciv.org, which went down permanently in 2007. This document has not kept track of any developments since then - it does not aim to describe any other software or efforts to run public Freeciv servers.
Specifically, it does not describe publite and other initiatives to replace parts of this system with something else,
since they are not ready to take over production just yet.
Please do not use this document to describe wishes or plans regarding the pubserver software or discuss its design;
the Pubserver Internals document exists for that purpose.
See also pubserver system administration.
==Purpose of the pubserver software==
The specific purpose of pubserver is to provide a
public Freeciv service: it runs a battery of Freeciv servers that
users can connect to from all over the Internet with their Freeciv
clients to play Freeciv games.
All this is installed under the freeciv account's home
directory.
The software and website are under CVS control, with its
repository on cvs.freeciv.org; it has not been made
publically available.
==Portability==
The software is quite portable; the build system used to be cross-tested
on a Solaris host (svis02.win.tue.nl, to be exact). It may be an idea to
start crosstesting it on Cygwin, this shouldn't pose much of a problem.
One file, share/Makefile, depends on GNU make.
Some of the auxiliary tools outside the build system
(i.e. not called by civservers) are Linux specific (e.g. depend
on the Linux /proc filesystem).
==Filesystem layout==
One way to look at the software is to see where it lives on the file system.
Everything is owned by the freeciv user and can be found under ~freeciv (some things are symlinked there from somewhere else).
(Note that it would be safer to have separate users for separate parts.)
The relevant directory trees are:
;~freeciv/bin: command line utilities for maintenance and monitoring, the build system, and post-game reporting (mostly scripts written in /bin/sh and Perl)
;~freeciv/share: auxiliaries for these executables (e.g. an important Makefile)
;~freeciv/.freeciv/code: Freeciv civserver executables with supporting data, at various stages of compilation and installation (the process is controlled from share/Makefile)
;~freeciv/.freeciv/games: the working directories of all running games ~freeciv/.freeciv/*
config files, abuse reports, etcetera
;~freeciv/htdocs/games: the game reports on completed games
;~freeciv/htdocs/*: the rest of the website, with dynamic pages using PHP
This does not completely organize everything by its function; it really should.
For example, all the different kinds of command line utilities
live together in ~freeciv/bin, together with a couple of throwaway and dysfunctional scripts,
which is quite inconvenient.
Post-game data are kept partly as files on disk, partly in a MySQL database on www.freeciv.org.
=The pubserver software, organized by function=
==Overview==
In all, the pubserver software consists of over 40 scripts; they will now be described
organized by their function within the whole.
The most logical functional decomposition seems the following:
; the build system: a system of command line tools that assembles and runs a battery of Freeciv servers
; post-game reporting: a system of command line tools that archives game results and creates (not so) pretty reports for them
; cleanup: command line tools to erase working directories that are no longer needed 
; monitoring: additional command line tools for the pubserver admin to see what is going on
; website: displays the pubserver status and the post-game reports to end users
__TOC__
== The build system ==
The software to operate the Freeciv servers consists of a couple of scripts that call each other.
Here is most of the call graph, showing only calls to homemade utility scripts -
calls to external utilities (make, GET, patch, gnuplot, gifsicle, etc. etc.) are not included. 
bin/civservers
|
+-+> share/Makefile
 |
 +-+> fetch
 |
 +-+> patch
 | |
 | +-+> bin/cvsdiff2udiff
 | |
 | +-+> bin/p0p1
 |
 +-+> config
 |
 +-+> compile
 |
 +-+> diff-patch, diff-compile
 |
 +-+> install
 | 
 +-+> start, restart
 | |
 | +-+> bin/civserver-forever
 | | |
 | . +-+> bin/cs
 | . |
 | . +-+> bin/newsubdir
 | |
 | +-+> .freeciv/code/(...)/civserver
 |
 +-+> stop
 | |
 | +-+> bin/civserver-kill
 |
 +-+> wipe-fetch, wipe-patch, wipe-compile, wipe-install
 |
 +-+> fetched-in, patched-in, compiled-in, installed-in
 .
 .
 +-+> picture-civgame
 |
 +-+> -d (database update)
 | |
 | +-+> bin/serversettings
 | |
 | +-+gt; bin/doranking
 | |
 | +-+gt; /home/hirisov/script/doranking, /home/hirisov/script/tourny
 | |
 | +-+gt; bin/xmlgamelogparse
 |
 +-+> -a (analyse core dump)
 | |
 | +-+> bin/civserver-core-backtrace
 |
 +-+> -c (copy game data)
 |
 +-+> -r (generate animated replay map)
 | |
 | +-+> bin/sortsav
 | |
 | +-+> bin/ppm2gif
 | |
 | +-+> bin/fcivppsav
 | |
 | +-+> bin/fcivppint
 | |
 | +-+> bin/fcivmaprace
 | |
 | +-+> bin/fcivmapplayers
 |
 +-+> -p (generate score plots)
 |
 +-+> bin/savs2time
 |
 +-+> bin/civscore2gnuplot
 |
 +-+> bin/colorize-gnumultiplot-ps
Top-down:
=== bin/civservers ===
The purpose of this script is twofold:
* it stores the pubserver configuration details, listing which server configuration runs on which port
* it acts as the general control shell
It may not seem logical to let a script be its own configuration file,
but it keeps things in one place and provides the flexibility required.
Since civservers is its own configuration file,
it contains a minimal amount of code: all actual functionality is delegated to share/Makefile.
The script can be called without arguments to perform whatever is required to get everything
going, or with specific arguments to perform specific subfunctions.
The calling syntax is /etc/init.d idiom (civservers start, etc.) extended to support
all the various subfunctions (civservers patch, civservers install, etc.).
Subfunctions depend on each other (e.g. you can't compile without having applied the indicated
patches, you can't apply patches without having fetched the indicated code base). This is why
civservers is just a wrapper around share/Makefile and its arguments are the Makefile targets.
There are special targets of the form wipe-* to forcibly remove the results of a particular step;
e.g. wipe-patch erases the results of patch, so re-patching can be forced.
There are two ways of making this script display its actions without executing them:
* civservers -n [action] will display the make command that would be invoked, without actually doing so
* civservers <action> -n will execute the make command with the -n option, causing make to display its actions without executing them
Sorely missing is an option to execute civservers on just a specific port or range of ports;
however, the -n option can be used to get around this.
Called by cron, but often used on the command line.
It is important to be aware that this script runs automatically from
cron. Certain configurations that are easy to choose by
mistake (e.g. telling it to recompile from current CVS on every
run) will make the host spin out of resources.
=== share/Makefile ===
Automatically creates and deploys a civserver from specifications supplied as command
line options and/or environment variables.
Designed to support running different Freeciv servers, compiled from different codebases,
with possibly different patch sets, alongside each other on different ports. This feature
is needed since pubserver always needs to support at least two different release versions,
and a varying range of pubserver-specific patches; it is also used to experiment with certain
experimental server-only features, and sometimes, to merge in nonstandard rulesets (they can be repackaged as patches).
This is implemented by encoding the arguments relevant to a stage into the names of the working directories used.
This is done in a way that maximizes reuse: e.g. two versions based on the same CVS snapshot will only cause the CVS codebase
to be checked out once, even when different sets of patches are subsequently applied to it,
while a different CVS timestamp will cause a different cvs checkout into a different directory.
The disadvantage is that the directory names become very long and unreadable.
Also, the implementation is quite ugly.
The following Makefile targets represent the consecutive stages of the build process:
;fetch: fetches a source code tree from a tarball on the web or from CVS (SVN support will be added soon)
;patches: assembles the list of patches to apply (should fetch them from rt.freeciv.org; there is some code to support this, but the change was never finished)
;patch: patches the source code with the list of patches (if any) (presently found in ~freeciv/patches, should be obtained from RT)
;compile: configures and compiles the resulting source code tree
;install: installs the compiled tree into a different directory
;start: deploys an installed server on one or more TCP ports
Each stage except the last stores its results in a different
subdirectory of ~freeciv/code. The last stage calls
bin/civserver-forever to do the real work.
Other operations (i.e. symbolic Makefile targets) include:
;stop: forcibly kills a running server
;restart: stops and restarts a server, unless any user is connected to it
;diff-patched, diff-compiled: creates a patch from a (possibly edited) codebase - very useful when you are updating existing patches to work with the present codebase
;wipe-fetch, wipe-patch, wipe-compile, wipe-install: cleans out the result from the respective build stage
;fetched-in, patched-in, compiled-in, installed-in:prints the paths of the working directories used in the respective build steps so you can cd to them (they are not human readable)
Most steps can be configured in various ways with Makefile variables and environment variables.
This Makefile is not intended to be invoked directly; the organized way to invoke and configure it
it is through the bin/civservers wrapper.
Although many functions are performed by command line tools in bin/*,
quite a large amount of code has built up directly within this
Makefile, which makes it hard to read and maintain.
The individual steps (targets) should be factored out to scripts.
Called from bin/civservers. Unless you are digging all the way down to the bottom,
please don't call this Makefile directly. A useful practical hack is to generate invocations with civservers -n,
copy a result to the shell prompt, edit it manually, then invoke it.
=== bin/cvsdiff2udiff ===
Modifies patches (mainly those from CVS) to a unified diff format.
Called by civservers patch, but useful on its own.
=== bin/p0p1 ===
Looks at a Freeciv patch to determine whether to patch it with -p0 or -p1.
Called by civservers patch, but useful on its own.
=== bin/civserver-forever ===
Keeps a civserver running in an endless loop, by
restarting it when it quits. Server output is saved to
civserver.out, which is important in investigating abuse
reports.
The present version uses civserver's -quitidle
flag to make the server quits when no users are present. (Earlier
versions depended on parsing the server output to determine this
condition and sending a /quit response on input when it
was met.)
The present version also initiates post-game reporting,
which can alternatively be started independently from cron.
Called by civservers start and civservers restart,
but useful on its own.
=== bin/cs ===
Starts a civserver in a newly created subdirectory under
~freeciv/.freeciv/games. This guarantees that two
different civservers do not overwrite each other's output data,
such as savegames and game logs. The directory name is a counter;
many scripts use it to identify the game, e.g. on the website.
Called by bin/civserver-forever, but useful on its own.
=== bin/newsubdir ===
An auxiliary script to create a new subdirectory.
Called by bin/cs, but useful on its own.
=== bin/onlyone ===
Prepend this to your command line to make sure the invoked command is only executed when it isn't executing already.
Very useful on long-term or costly commands (such as picture-civgame), especially when invoked from cron.
The method to determine whether the command is already running is a trial-by-error heuristic and by no means foolproof.
Called by various other scripts, but useful on its own.
=== bin/impatiently ===
Prepend this to your command line to make sure the invoked command is terminated unless it starts to produce output within a given time interval.
Very useful on long-term commands or commands that may block (such as civserver-ping)
The method to properly administer the timeout and exact the kill is a trial-by-error heuristic and by no means foolproof.
Called by various other scripts, but useful on its own.
=== bin/civserver-kill ===
Kills the civserver process(es) running on the port(s) supplied on the command line.
Called by civservers stop,
but useful on its own.
== Post-game archiving and reporting ==
Another set of scripts has the purpose of condensing the data
dumped on disk by an ended civserver to a manageable
quantity and generating Web-accessible overviews.
=== bin/picture-civgames ===
This script can be called to do post-game processing on existing game directories.
It simply calls bin/picture-civgame on the specified game directories.
Designed to be called by cron, but useful on its own.
=== bin/picture-civgame ===
This script performs all post-game processing on a game.
It serves four purposes:
* running reporting software on the game data to display (not so) pretty statistics and graphs to end users
* condensing and archiving all relevant game data into a subdirectory on the webserver, so the original working directory can be cleaned out;
* recording game results into the game database (including a computation of ranking information)
* alerting Freeciv admins in case of problems
As you can tell from its name, the script started out doing only the first part;
it created an animated replay map and some gnuplot gaphs and saved them to a directory on the website
for viewing with htdocs/viewgame.phtml.
Later, the information saved to that directory was extended in two ways:
* game information was added that doesn't show up to end users (e.g. the civserver output, to investigate abuse, and a coredump, if the server crashed)
* intermediate information was added so the graphing can be re-run (so we can improve the graphing part and rerun it on old games)
Furthermore, game information was saved to a MySQL database; we now depend on it for authentication and ranking.
Two different ranking systems are invoked, in fact.
Finally, when the server is found to have crashed, a core dump backtrace is emailed to the Freeciv admins.
Like bin/civservers, the script can be called without arguments, in which case it will do everything, or with arguments,
to execute a particular subfunction. Care was taken to make all execution re-entrant: the script and its subfunctions can be rerun
without leading to invalid results.
The exception is the database update and ranking computation, for which I (rp) don't know how to make
them reentrant; see the descripton of bin/doranking for details.
Takes care not to execute on games that are still running: the results would generally be invalid and should not yet
be available anyway.
By default, will skip subfunctions that have been executed before;
does not employ a Makefile to implement this.
Re-execution can be forced with the -f option.
Called by bin/picture-civgames and bin/civserver-forever,
but useful on its own.
=== bin/serversettings ===
Inserts game results into the game database.
Called by bin/picture-civgame.
Ought to be reentrant.
=== bin/doranking ===
Computes player ranking updates based on the game results, and
inserts them into the game database.
Called by bin/picture-civgame.
Ought to be reentrant.
The Pubserver Internals document discusses details of the ranking system.
=== /home/hirisov/script/doranking, /home/hirisov/script/tourny ===
An improved ranking system designed and implemented by hirisov.
Called by bin/picture-civgame.
Ought to be reentrant; since ranking depends not only on the present game's results,
but also on the participants' rankings at the time the ranking is updated, this is not
so easy to achieve. Ideally, the update should be history independent, and e.g. record
a timestamp to indicate the moment of update; that way, both the update and the ranking
computation become reentrant.
=== bin/xmlgamelogparse ===
Performs ranking based on a 2.1 (XML) gamelog.
Called by bin/picture-civgame, but useful on its own.
=== bin/civserver-core-backtrace ===
Runs gdb to create a backtrace of a civserver coredump (if present).
 
Called by bin/picture-civgame, but useful on its own.
=== bin/sortsav ===
Called by bin/picture-civgame, but useful on its own.
=== bin/ppm2gif ===
Not a homegrown script, but a standard compiled executable (from Jef Poskanzer's netpbm library) to convert images from PPM to GIF format.
Called by bin/picture-civgame.
===bin/fcivppsav, bin/fcivppint, bin/fcivmaprace, bin/fcivmapplayers===
Perl scripts that extract useful information from the series of savegames left by a game.
Based on common code in share/perl.
Called by bin/picture-civgame.
There is a performance bottleneck here.
=== bin/savs2time ===
From a series of Freeciv savegames and their mtimes, computes actual time consumption and timeout information,
and saves it in Freeciv scorelog format.
Called by bin/picture-civgame, but useful on its own.
=== bin/civscore2gnuplot ===
Takes information in Freeciv scorelog format and graphs it with gnuplot,
to produce the game graphs shown in htdocs/viewgame.phtml.
Called by bin/picture-civgame, but useful on its own.
=== bin/colorize-gnumultiplot-ps ===
Recolorizes gnuplot graphs produced by bin/civsciore2gnuplot, since games may feature more players than the number of different colors gnuplot can emit.
Called by bin/picture-civgame.
==~freeciv/share==
Contains auxiliary files (mostly code) that aren't executable scripts:
;share/Makefile: the heart of the autobuilding system for civservers
;share/perl: a Perl library to parse with Freeciv game data
The Makefile is discussed elsewhere.
===share/perl===
A simple Perl library, written by Todd Goodman, is available for
parsing Freeciv savegames: Freeciv::SavFiles. It is used
in utility scripts called by picture-civgame.
==Cleanup commands==
Some are automatically invoked by cron.
===bin/cleanup-gamedata===
Removes working directories of Freeciv servers that are no longer running.
Arguments specify criteria for removal.
Called from cron, but mostly useful on the command line (cron is
careful not to remove too much).
===bin/cleanup-unused-code===
Removes the directory trees under ~freeciv/.freeciv/code that are no longer
used by the present bin/civservers configuration.
Determining this is a little tricky, which is why it is not called from cron.
===bin/civgame-clean===
Reduces the number of savegames in a server's working directory, as if civserver had been invoked with the
saveturns parameter supplied as argument.
Not used.
==Monitoring commands==
Some utilities support the pubserver admin logging on to pubserver to see what is going on:
===bin/civserver-port-pid-dir===
The primary way to survey what is running on pubserver.
For each running server, lists its pubserver port, process id, and working directory.
Within the working directory, files are to be found that show the civserver version and startup configuration.
It is important to note that the set of running servers typically does not correspond to the set configured in bin/civservers.
A different configuration will only take effect on a port after the running civserver has been terminated, which will
only happen when no more users are connected to it. Furthermore, installing new configurations may fail, e.g. due to errors in the patching or compiling stages.
===bin/civserver-connections===
Lists the running civservers with their port and the hostnames of the connected users.
===bin/civserver-ping===
Pings users connected to a given pubserver port. Useful in determining connectivity problems.
This was important in the pre-1.12.0 days, when poor connectivity from one user would routinely lock up the server for everybody else.
Not much used today.
=== bin/civserver-status ===
Another way to list the status of running civservers.
=== bin/civserver-quit ===
Issues a /quit command to a running civserver. No longer works, because civserver-forever no longer supports supplying server command line input to the civservers it starts. (This feature was rarely used, but it allows the admin some nice emergency measures, and some other scripted behavior. For example, the metaserver infostring could be automatically set every 15 minutes.)
=== bin/freecivs ===
An old client side script. Lists the Freeciv servers shown on the Freeciv metaserver and optionally
invokes Freeciv clients to connect to them.
Not used. Not updated to support the 2.x metaserver.
=== bin/free-ports-between ===
Finds ports on which no server is running.
Called by share/Makefile, but useful on its own.
===bin/proc-net-tcp2host-port===
Roughly, a non-civserver-specific version of bin/civserver-port-pid-dir.
===bin/diskspace-check===
Estimates whether enough disk space is available for running games.
Called by cron, but useful on its own.
===bin/*===
Scripts for which a description is still missing:
civcities
civparse
civpath2server
civpow
civyears2turns
cpfreeciv
doallranking
fix-civscore
forxcpu
gamelog2html
getnumbers
mkpatch
names
nrframes
report-civscore
sav2ppm
sav2score
save4patch
savplayers
savyear
tar.gz-civserver
testcolorcube
testxcpu
==The pubserver website: ~freeciv/htdocs==
The pubserver website displays information on past and present pubserver games.
It combined preprocessed information with information from the games database and
on-the-fly formatting, using PHP as the scripting language.
The main pages on the webserver:
=== htdocs/index.phtml ===
The main page; contains a table with news items that requires updating after user-visible changes.
Its output is automatically copied to index.html for performance reasons.
=== htdocs/viewgame.phtml ===
Reports on an individual game. Most reporting data, such as the animated replay map and the game statistics histograms, are pregenerated, but this script is still quite slow. Some optimizatiuon has been done, e.g. in the generation of URLs for the previous and next buttons, but more is required.
The output doesn't look too good.
Filtered havigation (e.g. a button to go to the next game matching certain criteria)
would also be good to support. 
=== htdocs/query.phtml ===
Displays a table of games from the (MySQL) database. Supports sorting and selecting by specific columns, and crosslinks with viewgame.phtml, but the interface is quite primitive; more features and columns could be added, and the interface could probably be made easier to use.
=== htdocs/rankings.phtml ===
Displays a table of players from the database, by ranking. The ranking system, designed by Paul Zastoupil and Micha Riser, has known strengths and weaknesses. See also bin/dorankings.
=== htdocs/* ===
Other webpages: a gamelog prettyprinter (contributed by Sönke Ufen), a directory of IRC users, a (cyphor) message board, and more.
=The pubserver data=
An overview of the data employed by the pubserver software.
==Data in the MySQL database==
TO DO
==Data on the file system==
FAR FROM COMPLETE.
===~freeciv/code===
pubserver.freeciv.org always runs multiple Freeciv
versions and always applies various special patches to the source
code.
In January, 2003, for example, the following versions were
running:
;1.13.0: because Linux distributions were packaging it
;1.14.0-beta2: with the "teams" patch (a popular feature)
;1.14.0: the latest stable release, against which the "teams" patch would not work
;current CVS: always available for playtesting, updated automatically
All have extra patches applied to fix bugs and add special
behaviour. For instance, abuse in the past has led to a very simple
patch that sets the minimum timeout to 40. Another patch adds a
server command to add observer players to the game.
A civserver binary requires a matching data directory for
rulesets, nations, etcetera. Therefore, every different variant is
installed (with make install into a separate directory
tree, and started from there.
These trees are under ~freeciv/.freeciv/code/installed.
The other subdirectories of ~freeciv/.freeciv/code are
working directories used during the compilation process. The trees
are subdirectories of freeciv/code/installed.
===~freeciv/games===
The base directory for running civservers; the cs
wrapper script creates a subdirectory when it starts a
civserver.
We once lost a lot of data when the number of subdirectories
grew to more than the Linux kernel could handle, but this was fixed
with a kernel upgrade. No performance issues have been noticed,
although something like an ls, due to the sorting it does,
should obviously be avoided in scripts when possible.
===~freeciv/*===
Some other directories exist under ~freeciv; the
incidents directory contains reports and logs of
individual incidents, mostly cases of obnoxious civserver user behavior.
==~freeciv/htdocs/games==
The civserver reporting scripts take the data left by an ended
game in ~freeciv/.freeciv/games and
deposit their results under this directory. Most of the results are
linked to from viewgame.phtml, the PHP
script that displays an overview for an individual games; some are
not; a few are password protected because they contain users' IP
numbers.
Category:Pubserver