[CA-141792] wrap task.destroy with approriate exception handler

  db gc thread might delete task from tasks table, we should
  ignore the Not_found exception
Database: add a 'type' to each column

The types are:
 - String: most fields
 - Set: all Set(_)
 - Pairs: all Map(_,_)

This allows a database implementation to avoid serialising sets and maps
to strings all the time.

This patch also adds 'with sexp' to the schema so it can be easily read
and written. Eventually we will be able to separate the datamodel and
the schema and have the database load the schema at runtime.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: rename MAP2 to MAP, Map2 to Make and extract a common signature

Names like "MAP2" beg the question, "what happened to MAP1?"

This patch also factorises the ROW, TABLE, TABLESET signatures into a
shared common portion, with specific extensions on top.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: replace the raw 'int64' with a 'Time.t' (= Generation.t)

This means that
- if you read the type, you realise this isn't just any old int64
- we are closer to making it fully abstract

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: every function which takes Time.t -> Time.t has the third Time.t

Some of the functions omitted the third Time.t (the deleted time), because
the object could not have been deleted yet. Elsewhere we have the convention
that a live object has deleted = 0L.

This patch makes all the Time.t -> Time.t (-> Time.t) functions consistent.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: every function which takes 3 Time.ts now takes Stat.t

The 3 Time.t's are like a file "stat":
  - a create time
  - a last-modified time
  - a deletion time (since the events are historical)

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: fold_over_recent takes a function with the same signature as fold

Previously fold and fold_over_recent used a slightly different permutation
of arguments in the function being folded over the data. The OCaml stdlib
convention is to have a function (key -> value -> 'a -> 'a) so we follow
this.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: split Table.fold_over_recent into fold_over_recent and fold_over_deleted

This allows fold_over_recent to have the same signature as the other
functions with the same name. Tracking of deleted references (without values)
is specific to the Table.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: fold_over_recent have the same signature, so put in the common definition

Now that all the common MAP instances have the same fold_over_recent signature,
we can share the definition and avoid people playing "spot the difference"
with a complicated type.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: remove the errf from fold_over_{recent,deleted} as no-one was using it

To define an error case from a fold you can either
- turn the accumulator into a result type and return `Error
- throw an exception (... and document this and guarantee to catch it)

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: add missing license headers

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: start adding ocamldoc to the interface

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-81025: Remove archived RRDs after cross-pool migration

Destroying the VM using the API and not just the database calls causes the
removal of the archived RRDs on the master.

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
Database: remove Table.find_exn

This was confusing because we already had {Row,Table,TableSet}.find
which threw exceptions. The only difference was that Table.find_exn
would throw a DBCache exception rather than Not_found.

We revert all callers to Table.find, and expect them (in the DB_ACCESS
layer) to convert the Not_found into the DB_ACCESS-specific exception
themselves.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: rename 'update_generation' to 'touch' and simplify the signature

The 'update_generation' function was defined as
- always bump the generation count (in contract to 'update')
- also transform one binding to another value

In practice all users were using it only to bump the generation count
like "touch"ing a file. Therefore we remove the transform ability and
rename the function "touch".

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: use the same 'remove' function signature everywhere

This allows us to put the 'remove' function into the MAP signature
for consistency. Now Row, Table, TableSet all have a remove which
takes a Time.t which allows tracking of delete events. We only
use these in the Table to track deleted Rows, but maybe in future
we could track deleted Tables or fields later?

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: remove unused function 'rows'

This was a trivial use of fold anyway.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: add ocamldoc for the 'fold_over_deleted' function

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Database: remove some signatures duplicated from the .mli

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Whitespace: Xapi_hooks.internal_host_dead_hook

$ camlp4o -printer o -no_comments ocaml/xapi/xapi_hooks.ml | md5sum
40447b4d2bbed13c5f1b97214766a41a  -
$ git checkout HEAD^
$ camlp4o -printer o -no_comments ocaml/xapi/xapi_hooks.ml | md5sum
40447b4d2bbed13c5f1b97214766a41a  -

Signed-off-by: John Else <john.else@citrix.com>