Package: heptapod-tests
Heptapod automated functional testsThis source tree is both a library to write functional tests involving
Heptapod and the standard set of functional tests of Heptapod itself.Caveats:Usage as a library is totally undocumented at this point.The distribution on pypi.org does not contain the tests of Heptapod
itself. In other words, it contains the library part only.This README is mostly about the tests of Heptapod itself.WARNING: to test production instances, use the dedicated mode exclusively. Other
modes assume that you are ready tothrow all data awayafter the test
run, and hence are suitable at most for preflight full testing of a fresh
production instance.InstallationClient-side installMercurialThe tests need a workinghgexecutable, available on$PATH, with the
following extensions available:evolve and topic:hg-evolve≥ 9.3.0configexpress:hg-configexpressTest harness (Selenium)tox:pip install --user toxChromeDriverFor direct installation on the system running the tests:Fedora 29 to 33:dnf install chromedriverDebian 9 to 10, Ubuntu < 20.04:apt install chromium-driverUbuntu ≥ 20.04: thechromium-driverpackage actually installs a snap,
which can be problematic in some environments. See how it is done in thedocker-insidejob of ourCI pipelines.Another option is to use a Selenium RemoteWebDriver, which proxies the
actual web browsing through a dedicated system. Selenium provides theselenium/standalone-chromeDocker image for these purposes.All further dependencies will be installed by the first run.Heptapod server requirementsThese tests can work against Heptapod servers provided either as(default) local Docker containers manageable by the system user running the
tests, orinstalled from source and being run by the same user as the tests, orcompletely remote, skipping some of the tests, orproduction, relying on users with at most ownership of a dedicated projects
group, and running a subset of suitable tests.Except in production server mode, the Gitlab root password
will be initialized by the first test to run.
The tests will fail if the Gitlab root password is already set
and does not match the expected one.Default Docker setupIn the Docker case, the expected container name is by defaultheptapod.By default, the tests expect to be running on the Docker host, and that the
container can initiate TCP connections to the host, identified as the main
IPv4 gateway of the container. Don't forget toallow incoming TCP connections
from the container in your firewall, if you have one. You can also pass a
reachable address explicitely with--heptapod-reverse-call-hostor disable
such tests by passing an empty string as address.The container HTTP and SSH ports must be forwarded by default toheptapod:81andheptapod:2022. This is usually done by making theheptapodhost name
resolve to a loopback IP address, such as 127.0.0.2, and forwarding the
container ports like this:docker run --publish 127.0.0.2:2022:22 --publish 127.0.0.2:81:22Using a dedicated host name and IP address helps preventing confusion in
the user's.ssh/known_hostsfile.Running the teststoxis the executable launching the tests. It is also responsible to
setup Python libraries used in the tests. The tests themselves are
written withpytest.Alltoxcommands have to be run from the root of this repository.It is possible to pass down options to the underlyingpytestcommand:tox -- --heptapod-url URL -k push_basicAll Heptapod specific options take the--heptapod-some-optionform. For
the full list, dotox -- --helpCommon network optionsThese are available in all cases--heptapod-url(defaulthttp://heptapod:81): HTTP URL of the tested
Heptapod instance. It must use a resolvable hostname, not an IP address.
It does not have to be resolved through DNS, an/etc/hostentry pointing
to the loopback interface is fine.--heptapod-ssh-port(default 2022): SSH port of the tested Heptapod
instance. The same host name will be used as for HTTP. If the host name
resolves to the loopback interface, it is advised to tie it to a dedicated
address, such  as127.0.0.2, to minimize risks with your SSHknown_hostsfile.--heptapod-reverse-call-host: address that the Heptapod server can use
to reach the system running theses tests (necessary to test outbound
connections, such as web hooks).--heptapod-root-password(default5iveL!fe). The password to use and maybe
set for therootuser. The default value is the same as with the GitLab
Development Kit (GDK).Running the tests concurrentlyUse the---tests-per-workeroption only.DoNOTuse the--workersoption: it would setup theHeptapodsession
fixture several times, leading to problems with GitLab user tokens and other
shared data that are session-local.Testing a Docker containerBeing the default, this is the simplest. If you followed the default namings
and the current system user can managed Docker containers,
just runningtoxwill launch the whole tests suiteSpecific options:--heptapod-container-name(defaultheptapod)Testing a local install from source.You will need to run the tests and the Heptapod server under the same user
and to pass some mandatory options:Minimal example:~/heptapod/heptapod-tests $ tox -- --heptapod-source-install\
    --heptapod--repositories-root ~/heptapod/gdk/repositories-rootTesting a remote serverMandatory reminder:Never, ever run these tests on an
Heptapod server if you're not prepared to throw all its datayou'll have to provide the--heptapod-remoteoption, and probably be explicit
about all network options:~/heptapod/heptapod-tests $ tox -- --heptapod-remote \
  --heptapod-ssh-port 22 \
  --heptapod-url https://heptapod.test \
  --heptapod-root-password SECRETThe root password option is listed because you probably don't want to have
an instance with the default root password available on the internet.Testing a production instanceNew on 2021-02-18: see !80To run the tests suitable for production instances, you will need first to
prepare:a projects group entirely dedicated to these functional testsa dedicated user that owns the dedicated group (more users will probably be
needed in the future).The production mode is activated by an explicit command-line option. Another
option is used to pass the dedicated user credentials.Example:~/heptapod/heptapod-tests $ tox -- --heptapod-prod-server \
    --heptapod-prod-group-owner-credentials ID:USERNAME:EMAIL:PASSWORD \
    --heptapod-url https://foss.heptapod.net \
    --heptapod-ssh-port 22 \
    --heptapod-ssh-user hgwhere ID is the numeric user id, and USERNAME is the user login name
(e.gtestfonct).To launch tests on an instance tied to the Clever Cloud SSO,
use additionally the--heptapod-clever-cloud-ssooption.Remarks and safety:The user password must be fully operational: the functional tests won't
take care of the finalization sequence that occurs at first login.Do not give the dedicated user any rights outside of the dedicated groups.It is advisable to block the dedicated user when not in use.Be prepared to receive email for Merge Requests on the dedicated user address.
Arguably, this is part of the testing.Docker: choosing the version to testThe versions installed in the Docker image you're using are specified by theheptapod_revisions.jsonfile.To test your local clone of heptapod/heptapod:mount your localheptapodclone in the container (assuming below it's seen
as/home/heptapod-railsfrom the container)execute the following in the container:cd /var/opt/gitlab/embedded/service/gitlab-rails
hg pull --config phases.publish=False /home/heptapod-rails
hg up -r WISHED_REVISION
gitlab-ctl restart unicornIf you want to try changes to other components (e.g.,hg-git), do something similar.
The paths can be seen in the Docker build logs, or you can read them in theinstall scriptAdding new testsBranch and topics rulesThe convention is that the tests in the default branch should pass against
the currentoctobus/heptapod:latestDocker image, soif you want to share a bug reproduction, please open a new topiceven if a bug fix in heptapod has landed, please wait for the Docker push
before publishing the corresponding teststests can be published before a Heptapod new release, but please have the
corresponding fixes landed and pushed to Docker Hub first.If there is an active stable branch (e.g.heptapod-0-6-stableor similar),
then the tests of that branch must pass against the latest release version
corresponding to that branch. The same conclusions follow.