Do not validate pkg after a load of a manifest

pkg can be completed after loading an incomplete manifest, this is what
is done by pkg register.

This make "make install" from ports working again
Do not fully load in memory the plist to parse it
Update bundled sqlite to 2.7.16.1

While here make it use the system hearder when possible.
Readd the shell.c patch so that pkg shell alway open the local database
by default and the sqlite shell is opened with pkgng custom functions
loaded.
Do not increment lock count if we couldn't lock db

If we could not lock db it is not wise to increment lock count.
At the same time for unlocking db we decrement lock count in any
case as it is unlikely pattern that we could lock db, but could
not unlock and it is nevertheless means that db was unlocked
somehow.
Pkg delete now returns valid exit code.

When started in dry-run mode (-n flag) pkg-delete always returned
EX_SOFTWARE even if it actually succeeded. This patch fixes this
issue.
Validate package loaded from manifest.

Repository should contain only valid manifests, hence check
packages loaded from repo before adding them to db.
print ? in pkg version when comparing with remote repository and package is absent from remote
Readd possibility to pack with epoch timestamp via the UNSET_TIMESTAMP option

This is needed but not enough to enable reproducible builds
Do not set package field to "" when setting NULL

pkg_set function should not try to set fields that are to set as NULL
as empty string "". However this updated behaviour should be tested.
Treat PKG_DIGEST as ordinary field.

Manifest digest is placed directly in 'packages' table, treat it thus
as an ordinary package's field.
Rework package repo iterators.

Hack pkgdb_repo iterators to follow pkgdb iterators semantics in
particular in having it->db->sqlite field. However this whole code
should be implemented in some straightforward way.
Fix pkgdb_repo initialization.

Avoid some desctructive operation while opening pkgdb_repo. This
code was incorrectly imported from pkgdb code.
More steps to incremental update.

Now we do not touch database before comparing local and remote
repositories. Updates are now postponed by list of proposed updates.

Implemented fetch-extract function to a temporary file.

Miscellaneous fixes to incremental update logic.

TODO: unbreak comparasion in some cases.
Merge branch 'master' of git://github.com/pkgng/pkgng
Make pkgdb iterators more universal.

Pkgdb iterators may now be either cycled, which cycle over some sql
statement forever (may be used in combination with it->finished to
make several cycles), or they can be iterated only once (old behaviour
but now it is guaranteed that iterator will go through sqlite statement
only once). PKGDB_IT_FLAG_AUTO may be used to destroy iterator
automatically after the first iteration, this flag *cannot* be combined
with another flags, since caller must ensure that he never call
pkgdb_it_next one more time with destroyed iterator.
Fix flags parsing in pkg_it_next.

Remove issue introduced with the previous commit. Flags should be
parsed from iter, not from function argument.
Fix return code of pkg check.

Pkg check -s should exit with non-zero error code if some error
(checksum mismatch or missing file) is discovered.

Reported by:	citrin
Issue:	#484
Options are now self describe run pkg -vvv to output it
Enable fixed incremental updates.

Incremental updates now pass simple tests, therefore enable them
in this developemnt version to test all use cases.
Add incremental update event.

Show a short statistic after incremental update:
Incremental update completed: 0 packages updated, 1 removed and 1 added.