Make watcher identify itself on disk activation

For most jobs submitted by the watcher, it identifies
itself correctly in the reason trail. However, one
instance of an OpInstanceActivateDisk job was missed.
Fix this.

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Makefile: Fix unconditional metad files in ENABLE_METADATA

Since ENABLE_METADATA is optional, they should have their own
optional section, just like ENABLE_MOND does.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
NEWS: Mention job filters and ad-hoc filtering

These new features will be part of Ganeti 2.13.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Build confd by default

The only reason why it was optional was its dependency on regex-pcre,
which was an optional Ganeti dependency.

Job filter rules require rexex-pcre unconditionally, so now confd
requires no additional depdendencies and can be built by default.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix hlint warnings found by hlint 1.9.11

Our current hlint version cannot find them yet.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Klaus Aehlig <aehlig@google.com>

Cherry-picked from: d05f1c86fcca

Signed-off-by: Klaus Aehlig <aehlig@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>
Merge branch 'stable-2.12' into stable-2.13

* stable-2.12
  Fix hlint warnings found by hlint 1.9.11
  Make watcher identify itself on disk activation

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Fix unused import in processor mock

Signed-off-by: Aaron Karper <akarper@google.com>
Reviewed-by: Niklas Hambuechen <niklash@google.com>
Merge branch 'stable-2.13' into master

* stable-2.13
  Build confd by default
  NEWS: Mention job filters and ad-hoc filtering
  Makefile: Fix unconditional metad files in ENABLE_METADATA

* stable-2.12
  Fix hlint warnings found by hlint 1.9.11
  Make watcher identify itself on disk activation

Conflicts:
  configure.ac
  lib/cmdlib/cluster/verify.py

For configure.ac, the changed section regarding confd dependencies
was removed.

For verify.py, its contents were moved to lib/cmdlib/cluster/__init__.py
so the ENABLE_CONFD bit was removed from that one.

The cabal file and Makefile.am were adjusted to remove the confd flag.

Signed-off-by: Niklas Hambuechen <niklash@google.com>
Reviewed-by: Petr Pudlak <pudlak@google.com>
Update install docs - DRBD module parameters

In the example ensure that the DRBD options are always passed to the
module even if it's loaded before /etc/modules is processed.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix the installation instructions for the DRBD module

The options for the DRBD module should not be duplicated in the case the
procedure is run multiple times.

Signed-off-by: Petr Pudlak <pudlak@google.com>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
kvm: Minor refactor of MonitorSocket

Create socket upon connect(), keep the status of connection
(self._connected = False) in close(), make existing connect()
method protected and let connect() act as a reconnect method.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
utils: Introduce GetFreeSlot() function

Since this is a generic function that implements bitarray logic move
it from kvm to utils so that it can be easily used across all modules.

Make it raise errors.GenericError if it cannot find a free slot in
the given bitarray.

We add this function in a separate utils module (bitarrays) that can be
extended in the future. Currently only the network module and the
KVM hypervisor make use of bitarrays. This patch is a step forward
unifying these pieces of code and make both use the same bitarray
utility functions.

Add unittest for bitarrays utils module.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Refactor of add-fd and remove-fd commands

Instead of sending the qmp command along with the SCM_RIGHTS control
message we send just a blank (idea taken from qemu iotests). Then we
invoke the corresponding command (add-fd, remove-fd) with the
regular way (qmp.Execute). Since the qmp connection does not close
in between the fds will be added properly.

AddFd() wrapper will be used for disk hot-add except for the case
where we have userspace access mode. Make it raise HypervisorError
if the command fails for some reason so that any hotplug action that
takes place will abort.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Add GetFd() wrapper around getfd command

getfd qmp command will be used to pass an fd using SCM_RIGHTS
and name it properly so that NIC hot-add can take place.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Add helper methods to verify hotplug support

Introduce CheckDiskHotAddSupport() and CheckNicHotAddSupport()
helper methods that will be used to verify hotplug support.
Both check for fdsend python module and if the required qmp
commands are supported.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Helper methods for parsing query-pci output

Introduce GetPCIDevices() that gets the devices of the first PCI bus
of a running instance, HasPCIDevice() that will be used to verify
if a specific device is correctly hotplugged or not, and
GetFreePCISlot() that will be used during hot-add to find the first
available PCI slot of a running instance.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Add NIC hotplugging related methods

HotAddNic() uses netdev_add and device_add after passing the tapfd
to the running process with the GetFd() helper method.

HotDelNic() uses device_del and netdev_del qmp commands.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
qmp: Add Disk hotplugging related methods

HotAddDisk() uses blockdev-add and device_add after getting the
drive fd and passing it with the AddFd() helper method. Please note
that in case of userspace access mode this step will be omitted.

HotDelDisk() uses device_del. Please note that drive_del is
currently not supported in qmp and thus hmp should be used instead.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
kvm: New _with_qmp decorator

The decorator takes the hypervisor and the instance object as passed
to all hotplug related commands and ensures a valid QmpConnection()
object is found in hypervisor's qmp attribute.

This decorator is responsible for all actions related to the
QmpConnection (e.g. init, connect, close) and for proper exception
propagation. In order to be safe against nested invocations we close
the connection only if we create it.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>
monitor: Close socket fd if already connected

We close the socket fd only if we have an active connection.
We hide this logic in the public close() method, and all
the actions are moved to a private method.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Reviewed-by: Hrvoje Ribicic <riba@google.com>