Package: brightsky
Bright SkyJSON API for DWD's open weather data.The DWD (Deutscher Wetterdienst), as Germany's
meteorological service, publishes a myriad of meteorological observations and
calculations as part of theirOpen Data
program.Bright Skyis an open-source project aiming to
make some of the more popular data — in particular weather observations from
the DWD station network and weather forecasts from the MOSMIX model — available
in a free, simple JSON API.Looking for something specific?I just want to retrieve some weather dataYou can use the freepublic Bright Sky instance!I want to run my own instance of Bright SkyCheck out theinfrastructure
repo!I want to parse DWD weather files from the command line or in PythonThe parsing core for Bright Sky is maintained in a separate package nameddwdparse, which has no dependencies
outside the standard library. If you find that's not quite serving your needs,
check outwetterdienst.I want to contribute to Bright Sky's source codeRead on. :)On Bright Sky's versioningStarting from version 2.0, where we extracted the parsing core into aseparate
package, Bright Sky isno longer
intended to be used as a Python library, but only as the service available atbrightsky.dev.Consequentially, we adjust our version numbers from theperspective of that
service and its users– i.e., we will increase the major version number only
when we introduce backwards-incompatible (or otherwise very major) changes to
the actual JSON API interface, e.g. by changing URLs or parameters. This means
thatincreases of the minor version number may introduce
backwards-incompatible changes to the internals of thebrightskypackage,
including the database structure. If you usebrightskyas a Python library,
please version-pin to a minor version, e.g. by puttingbrightsky==2.0.*in
yourrequirements.txt.QuickstartRunning a full-fledged API instanceNote: These instructions are aimed at running a Bright Sky instance for
development and testing. Check out ourinfrastructure
repositoryif you want
to set up a production-level API instance.Just rundocker-compose upand you should be good to go. This will set up a
PostgreSQL database (with persistent storage in.data), run a Redis server,
and start the Bright Sky worker and webserver. The worker periodically polls
the DWD Open Data Server for updates, parses them, and stores them in the
database. The webserver will be listening to API requests on port 5000.ArchitectureBright Sky is a rather simple project consisting of four components:Thebrightskyworker, which leverages the logic contained in thebrightskyPython package to retrieve weather records from the DWD server,
parse them, and store them in a database. It will periodically poll the DWD
servers for new data.Thebrightskywebserver (API), which serves as gate to our database and
processes all queries for weather records coming from the outside world.A PostgreSQL database consisting of two relevant tables:sourcescontains information on the locations for which we hold weather
records, andweathercontains the history of actual meteorological measurements (or
forecasts) for these locations.The database structure can be set up by running themigratecommand, which
will simply apply all.sqlfiles found in themigrationsfolder.A Redis server, which is used as the backend of the worker's task queue.Most of the tasks performed by the worker and webserver can also be performed
independently. Rundocker-compose run --rm brightskyto get a list of
available commands.HackingConstantly rebuilding thebrightskycontainer while working on the code can
become cumbersome, and the default setting of parsing records dating all the
way back to 2010 will make your development database unnecessarily large. You
can set up a more lightweight development environment as follows:Create a virtual environment and install our dependencies:python -m virtualenv .venv && source .venv/bin/activate && pip install -r requirements.txt && pip install -e .Start a PostgreSQL container:docker-compose run --rm -p 5432:5432 postgresStart a Redis container:docker-compose run --rm -p 6379:6379 redisPointbrightskyto your containers, and configure a tighter date
threshold for parsing DWD data, by adding the following.envfile:BRIGHTSKY_DATABASE_URL=postgres://postgres:pgpass@localhost
BRIGHTSKY_BENCHMARK_DATABASE_URL=postgres://postgres:pgpass@localhost/benchmark
BRIGHTSKY_REDIS_URL=redis://localhost
BRIGHTSKY_MIN_DATE=2020-01-01You should now be able to directly runbrightskycommands viapython -m brightsky, and changes to the source code should be effective immediately.TestsLarge parts of our test suite run against a real Postgres database. By default,
these tests will be skipped. To enable them, make sure theBRIGHTSKY_TEST_DATABASE_URLenvironment variable is set when callingtox,
e.g. via:BRIGHTSKY_TEST_DATABASE_URL=postgres://postgres:pgpass@localhost/brightsky_test toxBeware that adding this environment variable to your.envfile will not work
as that file is not read bytox. The database will bedropped and
recreatedon every test run, so don't use your normal Bright Sky database. ;)AcknowledgementsBright Sky's development is boosted by the priceless guidance and support of
theOpen Knowledge Foundation'sPrototype
Fundprogram, and is generously funded by Germany'sFederal Ministry of Education and Research. Obvious as
it may be, it should be mentioned that none of this would be possible without
the painstaking, never-ending effort of theDeutscher
Wetterdienst.