test_integration.config: Fix crash in to_env when no replica is defined

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Do not save the input environment

Using the input environment saved in self._session_env
outside of the config loading meant that methods of
configuration other than environment variables wouldn't
be possible.

Restructure the roles/extra_roles to not depend on _session_env.

Part of the work for: https://fedorahosted.org/freeipa/ticket/3938

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Use a more declarative approach to test-wide settings

The list of options was duplicated too many times. Consolidate.

Part of the work for: https://fedorahosted.org/freeipa/ticket/3938

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Do not store the index in Domain and Host objects

The index is a detail of the environment variable method of
configuration, it should only be used there.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Load/store from/to dicts

Part of the work for: https://fedorahosted.org/freeipa/ticket/3938

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Add environment variables for JSON/YAML

Part of the work for: https://fedorahosted.org/freeipa/ticket/3938

Reviewed-By: Tomas Babej <tbabej@redhat.com>
ipa-test-config: Add --json and --yaml output options

Also update the man page.

Part of the work for: https://fedorahosted.org/freeipa/ticket/3938

Reviewed-By: Tomas Babej <tbabej@redhat.com>
test_integration.config: Convert some text values to str

When loading from file, some strings are loaded as unicode,
which would throw off assert_deepequal.

Reviewed-By: Tomas Babej <tbabej@redhat.com>
Add tests for integration test configuration

Reviewed-By: Tomas Babej <tbabej@redhat.com>
fix filtering of subdomain-based trust users

https://fedorahosted.org/freeipa/ticket/4207

Reviewed-By: Simo Sorce <ssorce@redhat.com>
ipa-kdb: do not fetch client principal if it is the same as existing entry

When client principal is the same as supplied client entry, don't fetch it
again.

Note that when client principal is not NULL, client entry might be NULL for
cross-realm case, so we need to make sure to not dereference NULL pointer here.

Also fix reverted condition for case when we didn't find the client principal
in the database, preventing a memory leak.

https://fedorahosted.org/freeipa/ticket/4223

Reviewed-By: Sumit Bose <sbose@redhat.com>
Avoid passing non-terminated string to is_master_host

When string is not terminated, queries with corrupted base may be sent
to LDAP:

... cn=ipa1.example.com<garbage>,cn=masters...

https://fedorahosted.org/freeipa/ticket/4214

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
ipa-replica-install never checks for 7389 port

When creating replica from a Dogtag 9 based IPA server, the port 7389
which is required for the installation is never checked by
ipa-replica-conncheck even though it knows that it is being installed
from the Dogtag 9 based FreeIPA. If the 7389 port would be blocked by
firewall, installation would stuck with no hint to user.

Make sure that the port configuration parsed from replica info file
is used consistently in the installers.

https://fedorahosted.org/freeipa/ticket/4240

Reviewed-By: Petr Viktorin <pviktori@redhat.com>
ipa-sam: cache gid to sid and uid to sid requests in idmap cache

Add idmap_cache calls to ipa-sam to prevent huge numbers of LDAP calls to the
directory service for gid/uid<->sid resolution.

Additionally, this patch further reduces number of queries by:
 - fast fail on uidNumber=0 which doesn't exist in FreeIPA,
 - return fallback group correctly when looking up user primary group as is
   done during init,
 - checking for group objectclass in case insensitive way

Patch by Jason Woods <devel@jasonwoods.me.uk>

Reviewed-by: Alexander Bokovoy <abokovoy@redhat.com>

https://fedorahosted.org/freeipa/ticket/4234
and
https://bugzilla.redhat.com/show_bug.cgi?id=1073829
https://bugzilla.redhat.com/show_bug.cgi?id=1074314

Reviewed-By: Sumit Bose <sbose@redhat.com>
ipaserver/dcerpc: make sure to always return unicode SID of the trust domain

Trusted domain SID could be obtained through different means. When it is
fetched from the AD DC via LDAP, it needs to be extracted from a default
context and explicitly converted to unicode.

https://fedorahosted.org/freeipa/ticket/4246

Reviewed-By: Martin Kosek <mkosek@redhat.com>
trust: do not fetch subdomains in case shared secret was used to set up the trust

Until incoming trust is validated from AD side, we cannot run any operations
against AD using the trust. Also, Samba currently does not suport verifying
trust against the other party (returns WERR_NOT_SUPPORTED).

This needs to be added to the documentation:

   When using 'ipa trust-add ad.domain --trust-secret', one has to manually
   validate incoming trust using forest trust properties in AD Domains and
   Trusts tool.

   Once incoming trust is validated at AD side, use IPA command
   'ipa trust-fetch-domains ad.domain' to retrieve topology of the AD forest.
   From this point on the trust should be usable.

https://fedorahosted.org/freeipa/ticket/4246

Reviewed-By: Martin Kosek <mkosek@redhat.com>
Prohibit deletion of active subdomain range

Changes the code in the idrange_del method to not only check for
the root domains that match the SID in the IDRange, but for the
SIDs of subdomains of trusts as well.

https://fedorahosted.org/freeipa/ticket/4247

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Fix idrange unit test failure

This is a follow up to patch for ticket 4247 - the raised
errors.DependentEntry changed, test needs to be change as well.

https://fedorahosted.org/freeipa/ticket/4247
Update Dogtag 9 database during replica installation

When Dogtag 10 based FreeIPA replica is being installed for a Dogtag 9
based master, the PKI database is not updated and miss several ACLs
which prevent some of the PKI functions, e.g. an ability to create
other clones.

Add an update file to do the database update. Content is based on
recommendation from PKI team:
   * https://bugzilla.redhat.com/show_bug.cgi?id=1075118#c9

This update file can be removed when Dogtag database upgrades are done
in PKI component. Upstream tickets:
   * https://fedorahosted.org/pki/ticket/710 (database upgrade framework)
   * https://fedorahosted.org/pki/ticket/906 (checking database version)

Also make sure that PKI service is restarted in the end of the installation
as the other services to make sure it picks changes done during LDAP
updates.

https://fedorahosted.org/freeipa/ticket/4243

Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Petr Viktorin <pviktori@redhat.com>