design: read config from cache

As jobs often ask for various parts of the configuration
in short sequence, we add a way to have those reads occur
from a cache in the job.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Support outdating a config

The config reader has a cached copy of the configuration.
Support marking it as outdated. In that way, we will later
be able to restrict reads to the case if we don't have
an up-to-date copy.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Outdate the config when waiting for locks

One of the reasons why jobs wait for locks is to avoid other
jobs modifying the same object. So, if we decide to wait for
locks, we cannot use our cached copy of the config any more.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Only read config if necessary

Make the ConfigWriter only rerequest the configuration if
it became outdated in the mean time. This will save unnecessary
traffic between jobs and WConfD.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Makefile.am: Don't use dots in -osuf

This fixes
  ./configure --enable-haskell-tests && make hs-tests
builds on GHC 7.8.

It changes `-osuf .ext` to `-osuf ext`; the old form was allowed in
GHC <= 7.6, but GHC 7.8 seems to have problems with it:
  https://ghc.haskell.org/trac/ghc/ticket/9760

The form without dot works in all GHC versions, and we don't have
dots in other -osuf arguments in the same file either, so this
was probably an accident.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Also in tests, open fake config before editing

As our tests access the actual config object, the test
setup needs to follow the same protocol as for a real
configuration. We need to open it, before modifying,
even in the test setup. Note that the test setup is outside
the code to be tested, so should not acquire any locks
(hence we open the mock config in shared mode even though
we intend to modify it).

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>
Only unlock config if we did lock it

We only acquired a config lock if it was not shared in order
to have lock-free reads. Hence, only release the config lock
if we actually acquired it.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>
Add a type-class for partial parameters

It links together partial and full parameters using functional
dependency type-class PartialParams.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Test if generated partial parameters work as expected

Namely that everything that is present in a partial value is transferred
to the result.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Simplify the construction of 'fillParams'

.. by working directly with the corresponding constructors.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add a function to convert filled parameters to/from partial

This is also useful for verifying, as it's often enough to write a test
just for the partial case.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add PartialParams instance for FilledIPolicy

.. in favor of explicit filledIPolicy.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add Monoid instances for partial parameters

.. which allow to create an empty partial parameters, as well as to
combine them together (the second "wins").

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix indentation in appConsApp in Ganeti.THH

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Don't lose stdout/stderr in AssertCommand

If a command fails/succeeds that should not do so,
show stdout and stderr. This information can be
relevant for debugging.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Makefile.am: Don't use -dynamic-too for .hpc_o files

It turns out that GHC 7.8's -dynamic-too is forbidden not only for
profiling builds object files, but also for those using HPC coverage.

This commit accordingly renames HFLAGS_NOPROF to HFLAGS_DYNAMIC
(since it now is not conditional on profiling any more),
and makes sure that it is not used for profiling or HPC.

This way we achieve that, for profiling+coverage builds
  - .dyn_o files are available for use in TH in the following 3 cases
  - .o files are for the normal binaries
  - .hpc_o files are for coverage-enabled test binaries
  - .prof_o files are for profiling binaries

We make .hpc_o files depend on the .o files because the creation of
the .o files will also create the .so files needed for TH.
This was already in place for .prof_o files.

This requires that HFLAGS now also contains `-itest/hs` since
the rule for .o files is now also responsible for building .o/.so files
in test/hs.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix python style

Keyword arguments should not be passed positionally.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Support forcing a config lock

As reads from WConfD are consistent anyway, the need for
dynamically acquiring the config lock is mostly gone;
that's why requests for shared config locks are ignored
by default. However, still support forcing shared config
locks for the cases still remaining, in particular
ClusterVerify.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Force config lock in cluster verification

...as we compare our copy of the configuration with that
on the other master candidates. So consistency only for
within the job process is not enough.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Implement GetAllDisksInfo in config

Implement 'GetAllDisksInfo' function which returns the configuration of
all disks.

Signed-off-by: Ilias Tsitsimpis <iliastsi@grnet.gr>