chore, just update the machine once

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, kick dbn dependency from random number stuff

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, organise random number stuff into librnd and link dbn-test to that (instead of libdbn)

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, hygiene, move random specific dbn stuff to rand.c

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, remove rand specific stuff, use the rand.h API (and librand) for dbn-test

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, tool terms(1), separate different documents by \f char

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Merge branch 'tool/terms'

* tool/terms:
  minor, tool terms(1), separate different documents by \f char
major, kick --dense again

It's almost always more desirable to have sparse output.
Actually the output needs a whole protocol already:
- how many words in the corpus,
- how many documents in the output
- max count (MAX_UINT8 for now)
- etc.

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
hygiene, provide once and but_first macros

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, support multiple input documents in tf(1) tool ...

each separated by a \f on a line on its own
The output will be the corpus' ids and counts and different
documents will be separated by \f\n as well.

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
hygiene, avoid naming routines with _t suffix (reserved for types)

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, express all tf(1) operations in terms of a prep-snarf-print loop

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
chore, install tf(1) tool

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Merge branch 'tool/tf'

* tool/tf:
  chore, install tf(1) tool
  minor, express all tf(1) operations in terms of a prep-snarf-print loop
  hygiene, avoid naming routines with _t suffix (reserved for types)
  minor, support multiple input documents in tf(1) tool ...
  hygiene, provide once and but_first macros
  major, kick --dense again
Merge branch 'feat/dbn'

* feat/dbn:
  minor, remove rand specific stuff, use the rand.h API (and librand) for dbn-test
  minor, hygiene, move random specific dbn stuff to rand.c
  minor, organise random number stuff into librnd and link dbn-test to that (instead of libdbn)
  minor, kick dbn dependency from random number stuff
  chore, just update the machine once
major, remove dbn/rbm stuff

Development of the deep belief network for text corpora will
move to drbang (https://github.com/hroptatyr/drbang/).

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Merge branch 'feat/dbn'

* feat/dbn:
  major, remove dbn/rbm stuff
fix, PEBKAC, take reference text file on into the distribution

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
test, provide regression tests for the document separator feature of terms(1)

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Merge branch 'test/terms'

* test/terms:
  test, provide regression tests for the document separator feature of terms(1)
  fix, PEBKAC, take reference text file on into the distribution