Package: pytest-server-fixtures
Pytest Server FixturesThis library provides an extensible framework for running up real
network servers in your tests, as well as a suite of fixtures for some
well-known webservices and databases.Table of ContentsBatteries IncludedInstallationConfigurationCommon fixture propertiesMongoDBPostgresRedisRethinkDBS3 MinioApache httpdSimple HTTP ServerXvfbJenkinsServer FrameworkIntegration TestsBatteries IncludedFixtureExtra Dependency NameMongoDBmongodbPostgrespostgresRedisredisRethinkDBrethinkdbS3 Minios3Apache HttpdSimple HTTP ServerJenkinsjenkinsXvfb (X-Windows Virtual Frame Buffer)Note: v2 fixtures support launching fixtures locally, inDockercontainers or asKubernetespods (SeeConfiguration)InstallationInstallation of this package varies on which parts of it you would like
to use. It uses optional dependencies (specified in the table above) to
reduce the number of 3rd party packages required. This way if you don’t
use MongoDB, you don’t need to install PyMongo.# Install with support for just mongodbpipinstallpytest-server-fixtures[mongodb]# Install with support for mongodb and jenkinspipinstallpytest-server-fixtures[mongodb,jenkins]# Install with Docker supportpipinstallpytest-server-fixtures[docker]# Install with Kubernetes supportpipinstallpytest-server-fixtures[kubernetes]# Install with only core library and support for httpd and xvfppipinstallpytest-server-fixturesEnable the fixture explicitly in your tests or conftest.py (not required
when using setuptools entry points):pytest_plugins=['pytest_server_fixtures.httpd','pytest_server_fixtures.jenkins','pytest_server_fixtures.mongo','pytest_server_fixtures.postgres','pytest_server_fixtures.redis','pytest_server_fixtures.rethink','pytest_server_fixtures.xvfb',]ConfigurationThe fixtures are configured using the following evironment variables:SettingDescriptionDefaultSERVER_FI XTURES_HOSTNAMEHostname that servers will
listen onCurrent default
hostnameSE RVER_FIXTURES_DIS ABLE_HTTP_PROXYDisable any HTTP proxies set
up in the shell environment
when making HTTP requestsTrueSERVER_FIXTUR ES_SERVER_CLASSServer class used to run the
fixtures, choose fromthread,dockerandkubernetesthreadSERVER_FIXTURE S_K8S_NAMESPACE(Kubernetes only) Specify
the Kubernetes namespace
used to launch fixtures.None(same as
the test host)SERVER_FIXTURES _K8S_LOCAL_TEST(Kubernetes only) Set toTrueto allow
integration tests to run
(SeeIntegration
Tes
ts).FalseSERVER_FIX TURES_MONGO_BINAbsolute path to mongod
executable“” (relies on
mongod access via$PATH)SERVER_FIXTU RES_MONGO_IMAGE(Docker only) Docker image
for mongomongo:3.6SERVER_FIX TURES_PG_CONFIGPostgres pg_config
executablepg_configSERVER _FIXTURES_REDISRedis server executableredis-serverSERVER_FIXTU RES_REDIS_IMAGE(Docker only) Docker image
for redisredis:5.0.2-alpineSERVER_F IXTURES_RETHINKRethinkDB server executablerethinkdbSERVER_FIXTURE S_RETHINK_IMAGE(Docker only) Docker image
for rethinkdbrethink:2.3.6SERVER _FIXTURES_HTTPDHttpd server executableapache2SERVER_FIXTURE S_HTTPD_MODULESHttpd modules directory/usr/lib/ apache2/modulesSERVE R_FIXTURES_JAVAJava executable used for
running Jenkins serverjavaSERVER_FIXTU RES_JENKINS_WAR.warfile used to run
Jenkins/usr/share/jenk ins/jenkins.warSERVE R_FIXTURES_XVFBXvfb server executableXvfbCommon fixture propertiesAll of these fixtures follow the pattern of spinning up a server on a
unique port and then killing the server and cleaning up on fixture
teardown.All test fixtures share the following properties at runtime:PropertyDescriptionhostnameHostname that server is listening onportPort number that the server is
listening ondeadTrue/False: am I dead yet?workspace`pa th<https://path.readthedocs.io/>`__
object for the temporary directory the
server is running out ofMongoDBThemongomodule contains the following fixtures:Fixture NameDescriptionmongo_serverFunction-scoped MongoDB servermongo_server_sessSession-scoped MongoDB servermongo_server_clsClass-scoped MongoDB serverAll these fixtures have the following properties:PropertyDescriptionapipymongo.MongoClientconnected to running serverHere’s an example on how to run up one of these servers:deftest_mongo(mongo_server):db=mongo_server.api.mydbcollection=db.test_colltest_coll.insert({'foo':'bar'})asserttest_coll.find_one()['foo']=='bar'PostgresThepostgresmodule contains the following fixture:Fixture NameDescriptionpostgres_server_sessSession-scoped Postgres serverThe Postgres server fixture has the following properties:PropertyDescriptionconnect()Returns a rawpsycopg2connection
object connected to the serverconnection_configReturns a dict containing all the data
needed for another db library to
connect with.You may wish to build another fixture on top of the session-scoped
fixture; for example:defcreate_full_schema(connection):"""Create the database schema"""pass@pytest.fixture(scope='session')defdb_config_sess(postgres_server_sess:PostgresServer)->PostgresServer:"""Returns a DbConfig pointing at a fully-created db schema"""server_cfg=postgres_server_sess.connection_configcreate_full_schema(postgres_server_sess.connect())returnpostgres_server_sessRedisTheredismodule contains the following fixtures:Fixture NameDescriptionredis_serverFunction-scoped Redis serverredis_server_sessSession-scoped Redis serverAll these fixtures have the following properties:PropertyDescriptionapiredis.Redisclient connected to the running serverHere’s an example on how to run up one of these servers:deftest_redis(redis_server):redis_server.api.set('foo':'bar')assertredis_server.api.get('foo')=='bar'S3 MinioThes3module contains the following fixtures:Fixture NameDescriptions3_serverSession-scoped S3 server using the ‘minio’ tool.s3_bucketFunction-scoped S3 bucketThe S3 server has the following properties:PropertyDescriptionget_s3_client()Return a boto3Resource:
(boto3.resource('s3',...)The S3 Bucket has the following properties:PropertyDescriptionnameBucket name, a UUIDclientBoto3Resourcefrom the serverHere’s an example on how to run up one of these servers:deftest_connection(s3_bucket):bucket=s3_bucket.client.Bucket(s3_bucket.name)assertbucketisnotNoneRethinkDBTherethinkmodule contains the following fixtures:Fixture NameDescriptionrethink_serverFunction-scoped Redis serverrethink_server_sessSession-scoped Redis serverrethink_unique_dbSession-scoped unique dbrethink_module_dbModule-scoped unique dbrethink_make_tablesModule-scoped fixture to create
named tablesrethink_empty_dbFunction-scoped fixture to
empty tables created inrethink_make_tablesThe server fixtures have the following propertiesPropertyDescriptionconnrethinkdb.Connectionto thetestdatabase on the
running serverHere’s an example on how to run up one of these servers:deftest_rethink(rethink_server):conn=rethink_server.connconn.table_create('my_table').run(conn)inserted=conn.table('my_table').insert({'foo':'bar'}).run(conn)assertconn.get(inserted.generated_keys[0])['foo']=='barCreating TablesYou can create tables for every test in your module like so:FIXTURE_TABLES=['accounts','transactions']deftest_table_creation(rethink_module_db,rethink_make_tables):conn=rethink_module_dbassertconn.table_list().run(conn)==['accounts','transactions']Emptying DatabasesRehinkDb is annecdotally slower to create tables that it is to empty
them (at least at time of writing), so we have a fixture that will empty
out tables between tests for us that were created with therethink_make_tablesfixture above:FIXTURE_TABLES=['accounts','transactions']deftest_put_things_in_db(rethink_module_db,rethink_make_tables):conn=rethink_module_dbconn.table('accounts').insert({'foo':'bar'}).run(conn)conn.table('transactions').insert({'baz':'qux'}).run(conn)deftest_empty_db(rethink_empty_db):conn=rethink_empty_dbassertnotconn.table('accounts').run(conn)assertnotconn.table('transactions').run(conn)Apache httpdThehttpdmodule contains the following fixtures:Fixture NameDescriptionhttpd_serverFunction-scoped httpd server to use as a web proxyThe fixture has the following properties at runtime:PropertyDescriptiondocument_rootpath.pathto the document rootlog_dirpath.pathto the log directoryHere’s an example showing some of the features of the fixture:deftest_httpd(httpd_server):# Log files can be accessed by the log_dir propertyassert'access.log'in[i.basename()foriinhttpd_server.log_dir.files()]# Files in the document_root are accessable by HTTPhello=httpd_server.document_root/'hello.txt'hello.write_text('Hello World!')response=httpd_server.get('/hello.txt')assertresponse.status_code==200assertresponse.text=='Hello World!'Proxy RulesAn httpd server on its own isn’t super-useful, so the underlying class
for the fixture has options for configuring it as a reverse proxy.
Here’s an example where we’ve pulled in apytest-pyramidfixture and
set it up to be proxied from thehttpdserver:importpytestfrompytest_server_fixtures.httpdimportHTTPDServerpytest_plugins=['pytest_pyramid']@pytest.yield_fixture()defproxy_server(pyramid_server):# Configure the proxy rules as a dict of source -> dest URLsproxy_rules={'/downstream/':pyramid_server.url}server=HTTPDServer(proxy_rules,# You can also specify any arbitrary text you want to# put in the config fileextra_cfg='Alias /tmp /var/tmp\n',)server.start()yieldserverserver.teardown()deftest_proxy(proxy_server):# This request will be proxied to the pyramid serverresponse=proxy_server.get('/downstream/accounts')assertresponse.status_code==200Simple HTTP ServerThehttpmodule contains the following fixtures:Fixture NameDescriptionsimple_http_serverFunction-scoped instance of Python’sSimpleHTTPServerThe fixture has the following properties at runtime:PropertyDescriptiondocument_rootpath.pathto the document rootHere’s an example showing some of the features of the fixture:deftest_simple_server(simple_http_server):# Files in the document_root are accessable by HTTPhello=simple_http_server.document_root/'hello.txt'hello.write_text('Hello World!')response=simple_http_server.get('/hello.txt')assertresponse.status_code==200assertresponse.text=='Hello World!'JenkinsThejenkinsmodule contains the following fixtures:Fixture NameDescriptionjenkins_serverSession-scoped Jenkins server instanceThe fixture has the following methods and properties:PropertyDescriptionapijenkins.JenkinsAPI client
connected to the running server (see
h
ttps://python-jenkins.readthedocs.org)load_plugins()Load plugins into the server from a
directoryHere’s an example showing how to run up the server:PLUGIN_DIR='/path/to/some/plugins'deftest_jenkins(jenkins_server):jenkins_server.load_plugins(PLUGIN_DIR)assertnotjenkins_server.api.get_jobs()XvfbThexvfbmodule contains the following fixtures:Fixture NameDescriptionxvfb_serverFunction-scoped Xvfb serverxvfb_server_sessSession-scoped Xvfb serverThe fixture has the following properties:PropertyDescriptiondisplayX-windowsDISPLAYvariableHere’s an example showing how to run up the server:deftest_xvfb(xvfb_server):assertxvfb_server.displayServer FrameworkAll the included fixtures and others in this suite of plugins are built
on an extensible TCP server running framework, and as such many of them
share various properties and methods.pytest_shutil.workspace.Workspace
  |
  *--base2.TestServerV2
     |
     *--mongo.MongoTestServer
     *--redis.RedisTestServer
     *--rethink.RethinkDBServer
  *--base.TestServer
     |
     *--http.HTTPTestServer
        |
        *--http.SimpleHTTPTestServer
        *--httpd.HTTPDServer
        *--jenkins.JenkinsTestServer
        *--pytest_pyramid.PyramidTestServerClass MethodsThe best way to understand the framework is look at the code, but here’s
a quick summary on the class methods that child classes ofbase.TestServercan override.MethodDescriptionpre_setupThis should execute any setup required
before starting the serverrun_cmd(required)This should return a list of shell commands
needed to start the serverrun_stdinThe result of this is passed to the process
as stdincheck_server_up(required)This is called to see if the server is
runningpost_setupThis should execute any setup required
after starting the serverClass AttributesAt a minimum child classes must definerun_cmdandcheck_server_up. There are also some class attributes that can be
overridden to modify server behavior:AttributeDescriptionDefaultrandom_portStart the server on a
guaranteed unique random
TCP portTrueport_seedIfrandom_portis
false, port number is
semi-repeatable and based
on a hash of the class
name and this seed.65535kill_signalSignal used to kill the
serverSIGTERMkill_retry_delayNumber of seconds to wait
between kill retries.
Increase this if your
server takes a while to
die1Constructor ArgumentsThe base class constructor also accepts these arguments:ArgumentDescriptionportExplicitly set the port numberhostnameExplicitly set the hostnameenvDict of the shell environment passed to the server processcwdOverride the current working directory of the server processIntegration Tests$ vagrant up
$ vagrant ssh
...
$ . venv/bin/activate
$ cd /vagrant
$ make develop
$ cd pytest-server-fixtures

# test serverclass="thread"
$ pytest

# test serverclass="docker"
$ SERVER_FIXTURES_SERVER_CLASS=docker pytest

# test serverclass="kubernetes"
$ SERVER_FIXTURES_SERVER_CLASS=kubernetes SERVER_FIXTURES_K8S_LOCAL_TEST=True pytestChangelog1.7.0All: Support pytest >= 4.0.0All: Support Python 3.7pytest-server-fixtures: if host not defined on your machine, default
to localhostpytest-server-fixture: Pin to rethinkdb < 2.4.0 due to upstream API
changespytest-verbose-parametrize: Add support for revamped marker
infrastructurepytest-verbose-parametrize: Fix integration tests to support pytest
>= 4.1.0pytest-virtualenv: Add virtualenv as install requirement. Fixes #122pytest-webdriver: Fix RemovedInPytest4Warning using getfixturevaluecircleci: Fix checks by skipping coverall submission for developer
without push accesswheels: Generate universal wheels installable with both python 2.x
and 3.xdist: Remove support for building and distributing *.egg filesVagrantFile: Install python 3.7 and initialize python 3.7 by defaultFix DeprecationWarning warnings using “logger.warning()” function1.6.2 (2019-02-21)pytest-server-fixtures: suppress stacktrace if kill() is calledpytest-server-fixtures: fix random port logic in TestServerV21.6.1 (2019-02-12)pytest-server-fixtures: fix exception when attempting to access
hostname while server is not started1.6.0 (2019-02-12)pytest-server-fixtures: added previously removed TestServerV2.kill()
functionpytest-profiling: pin more-itertools==5.0.0 in integration tests, as
that’s a PY3 only release1.5.1 (2019-01-24)pytest-verbose-parametrize: fixed unicode parameters when using@pytest.mark.parametrize1.5.0 (2019-01-23)pytest-server-fixtures: made postgres fixtures and its tests
optional, like all other fixturespytest-server-fixtures: reverted a fix for pymongo deprecation
warning, as this will break compatibility with pymongo 3.6.0pytest-server-fixtures: dropped RHEL5 support in httpd1.4.1 (2019-01-18)pytest-server-fixtures: server fixture binary path specified in ENV
now only affect server class ‘thread’1.4.0 (2019-01-15)Fixing python 3 compatibility in Simple HTTP Server fixtureFixed broken tests in pytest-profilingPinned pytest<4.0.0 until all deprecation warnings are fixed.pytest-webdriver: replaced deprecated phantomjs with headless Google
Chrome.Add Vagrantfile to project to make test environment portable.Add .editorconfig file to project.pytest-server-fixtures: add TestServerV2 with Docker and Kubernetes
support.pytest-server-fixtures: fix for an issue where MinioServer is not
cleaned up after use.pytest-server-fixtures: fix deprecation warnings when calling
pymongo.pytest-server-fixtures: close pymongo client on MongoTestServer
teardown.pytest-server-fixtures: upgrade Mongo, Redis and RethinkDB to
TestServerV2.coveralls: fix broken coveralls1.3.1 (2018-06-28)Use pymongo list_database_names() instead of the deprecated
database_names(), added pymongo>=3.6.0 dependency1.3.0 (2017-11-17)Fixed workspace deletion when teardown is NoneFixed squash of root logger in pytest-listenerAdded S3 Minio fixture (many thanks to Gavin Bisesi)Added Postgres fixture (many thanks to Gavin Bisesi)Use requests for server fixtures http gets as it handles redirects
and proxies properly1.2.12 (2017-8-1)Fixed regression on cacheing ephemeral hostname, some clients were
relying on this. This is now optional.1.2.11 (2017-7-21)Fix for OSX binding to illegal local IP range (Thanks to Gavin
Bisesi)Setup and Py3k fixes for pytest-profiling (Thanks to xoviat)We no longer try and bind port 5000 when reserving a local IP host,
as someone could have bound it to 0.0.0.0Fix for #46 sourcing gprof2dot when the local venv has not been
activated1.2.10 (2017-2-23)Handle custom Pytest test items in pytest-webdriver1.2.9 (2017-2-23)Add username into mongo server fixture tempdir path to stop
collisions on shared multiuser filesystems1.2.8 (2017-2-21)Return function results in shutil.run.run_as_main1.2.7 (2017-2-20)More handling for older versions of path.pyAllow virtualenv argument passing in pytest-virtualenv1.2.6 (2017-2-16 )Updated devpi server server setup for devpi-server >= 2.0Improvements for random port pickingHTTPD server now binds to 0.0.0.0 by default to aid Selenium-style
testingUpdated mongodb server args for mongodb >= 3.2Corrections for mongodb fixture config and improve startup logicAdded module-scoped mongodb fixtureHandling for older versions of path.pyFix for #40 where tests that chdir break pytest-profiling1.2.5 (2016-12-09)Improvements for server runner host and port generation, now supports
random local IPsBugfix for RethinkDB fixture config1.2.4 (2016-11-14)Bugfix for pymongo extra dependencyWindows compatibility fix for pytest-virtualenv (Thanks to
Jean-Christophe Fillion-Robin for PR)Fix symlink handling for
pytest-shutil.cmdline.get_real_python_executable1.2.3 (2016-11-7)Improve resiliency of Mongo fixture startup checks1.2.2 (2016-10-27)Python 3 compatibility across most of the modulesFixed deprecated Path.py imports (Thanks to Bryan Moscon)Fixed deprecated multicall in pytest-profiling (Thanks to Paul van
der Linden for PR)Added devpi-server fixture to create an index per test functionAdded missing licence fileSplit up httpd server fixture config so child classes can override
loaded modules easierAdded ‘preserve_sys_path’ argument to TestServer base class which
exports the current python sys.path to subprocesses.Updated httpd, redis and jenkins runtime args and paths to current
Ubuntu specIgnore errors when tearing down workspaces to avoid race conditions
in ‘shutil.rmtree’ implementation1.2.1 (2016-3-1)Fixed pytest-verbose-parametrize for latest version of py.test1.2.0 (2016-2-19)New plugin: git repository fixture1.1.1 (2016-2-16)pytest-profiling improvement: escape illegal characters in .prof
files (Thanks to Aarni Koskela for the PR)1.1.0 (2016-2-15)New plugin: devpi server fixturepytest-profiling improvement: overly-long .prof files are saved as
the short hash of the test name (Thanks to Vladimir Lagunov for PR)Changed default behavior of workspace.run() to not use a subshell for
security reasonsCorrected virtualenv.run() method to handle arguments the same as the
parent method workspace.run()Removed deprecated ‘–distribute’ from virtualenv args1.0.1 (2015-12-23)Packaging bugfix1.0.0 (2015-12-21)Initial public release