Package: sdsstools
sdsstoolssdsstoolsprovides several common tools for logging, configuration handling, version parsing, packaging, etc. Its main purpose is to consolidate some of the utilities originally found in thepython_template, allowing them to become dependencies that can be updated.This is not intended to be a catch-all repository for astronomical tools.sdsstoolsitself aims to have minimal dependencies (i.e., mainly the Python standard library and setuptools).Using sdsstoolsTo use sdsstools simply install it withpip install sdsstoolsMost likely, you'll want to include sdsstools as a dependency for your library. To do so, either add to yoursetup.cfg[options]install_requires=sdsstools>=0.1.0(this is equivalent of passinginstall_requires=['sdsstools>=0.1.0']tosetuptools.setup), or if you are usingpoetryrunpoetry add sdsstools, which should add this line to yourpyproject.toml[tool.poetry.dependencies]sdsstools={version="^0.1.0"}Loggingsdsstools includes thesdsstools.logger.SDSSLoggerclass, which provides a wrapper around the standard Pythonloggingmodule.SDSSLogerprovides the following features:A console handler (accessible via the.shattribute) with nice colouring.Automatic capture of warnings and exceptions, which are formatted and redirected to the logger. For the console handler, this means that once the logger has been created, all warnings and exceptions are output normally but are clearer and more aesthetic.ATimedRotatingFileHandler(accessible via the.fhattribute) that rotates at midnight UT, with good formatting.A version of the logger that usesrichlog handling.To get a new logger for your application, simply dofromsdsstools.loggerimportget_loggerNAME='myrepo'log=get_logger(NAME)You can get a logger using therichRichHandlerby passinguse_rich_handler=True.The file logger is disabled by default and can be started by callinglog.start_file_logger(path). By default aTimedRotatingFileHandleris created. If you want a normalFileHandleruserotating=False. The file mode defaults tomode='a'(append).  TheTimedRotatingFileHandleroptionswhen,utc, andat_timeare available tolog.start_file_loggerfor controlling aspects of the rollover time.By default, the file logger is formatted to output a human-readblelogfile.  To output a JSON log instead, setas_json=True, when callinglog.start_file_logger.  This will create a.jsonlog file which can machine-read and more easily parsed for content.  To output both a human-readable (.log) and JSON log (.json), setwith_json=True.TheSDSSLoggerinstance also include anasyncio_exception_handlermethod that can be added to the asyncio event loop to handle exceptions; for exampleloop.set_exception_handler(log.asyncio_exception_handler).Console logging uses the standardStreamHandler. It's possible to use therichlibraryRichHandlerinstead by
passinguse_rich_handler=Truetoget_logger(). Additional keyword arguments toRichHandlercan be passed as arich_handler_kwargsdictionary. In the future therichhandler may become the default console logger.ConfigurationThesdsstools.configurationmodule contains several utilities to deal with configuration files. The most useful one isget_config, which allows to read a YAML configuration file. For examplefromsdsstools.configurationimportget_configNAME='myrepo'config=get_config(NAME,allow_user=True)get_configassumes that the file is located inetc/<NAME>.ymlrelative from the file that callsget_config, but that can be changed by passingconfig_file=<config-file-path>. Additionally, ifallow_user=Trueand a file exists in~/.config/sdss/<NAME>.yaml, this file is read and merged with the default configuration, overriding any parameter that is present in the user file. This allows to create a default configuration that lives with the library but that can be overridden by a user.In addition to the (recommended) location~/.config/sdss/<NAME>.yaml,get_configalso looks for user configuration files in~/.config/sdss/<NAME>.yml,~/.config/sdss/<NAME>/<NAME>.y(a)ml, and~/.<NAME>/<NAME>.y(a)ml.get_configreturns an instance ofConfiguration, which behaves as a dictionary but allows to dynamically reload the configuration from a new user file by callingload().sdsstools.configurationincludes two other tools,merge_config, that allows to merge dictionaries recursively, andread_yaml_fileto read a YAML file.Extending a YAML fileread_yaml_fileprovides a non-standard feature that allows you to extend one YAML file with another. To achieve this you need to add the tag!extends <base-file>at the top of the file that you want to extend. For example, if you have a filebase.yamlcat1:key1:value2cat2:key2:1that you want to use as a template forextendable.yaml#!extends base.yamlcat1:key1:value1you can useread_yaml_fileto parse the result>>>read_yaml_file('extendable.yaml'){'cat1':{'key1':'value2'},'cat2':{'key2':1}}The path to the base file must be absolute or relative to the location of the file to be extended.TheConfigurationclassBy defaultget_config()andread_yaml_file()return aConfigurationinstance. For the most part aConfigurationobject is the same as a dictionary, and it can be used as such. It has two main differences:When aConfigurationobject is initialised from a file (or a base and custom configuration files) as withget_config(), the object keeps the information about the file paths. It's then possible to callConfiguration.reload()to hot-reload the contents of the file after it has changed.It is possible to recursively get a nested configuration value, for exampleconfig['a.b'], which is equivalent todd['a']['b']but will returnNoneif a key does not exist anywhere in the chain. This behaviour can be disabled by settingconfig.strict_mode=True. Note that this syntax is not valid for assignement and attempting to dod['a.b'] = 1will raise an error.Metadatasdsscore provides tools to locate and parse metadata files (pyproject.toml,setup.cfg,setup.py).get_metadata_fileslocates the path of the metadata file relative to a givenpath.get_package_versiontries to find the version of the package by looking for a version string in the metadata file or in the egg/wheel metadata file, if the package has been installed. To use itfromsdsstools.metadataimportget_package_version__version__=get_package_version(path=__file__,package_name='sdss-camera')or'dev'This will try to find and parse the version from the metadata file (we pass__file__to indicate where to start looking); if that fails, it will try to get the version from the installed packagesdss-camera. If all fails, it will set the fallback version'dev'.Command Line Interfacesdsstoolsprovides the command line toolsdss, which is just a thin wrapper around some commonly usedInvoketasks.sdsstoolsdoes not automatically install all the dependencies for the tasks, which need to be added manually.sdssprovides the following tasksTaskOptionsDescriptioncleanRemoves files produces during build and packaging.deploy--testBuilds and deploys to PyPI (or the test server). Requirestwineandwheel.install-deps--extrasInstalls dependencies from asetup.cfgfiledocs.build--targetBuilds the Sphinx documentation. RequiresSphinx.docs.show--targetShows the documentation in the browser. RequiresSphinx.docs.clean--targetCleans the documentation build. RequiresSphinx.sdssassumes that the documentation lives indocs/sphinxrelative to the root of the repository. This can be changed by setting thesphinx.targetconfiguration in aninvoke.yamlfile, for examplesphinx:target:docsClick daemon commandThedaemonizermodule implements aClickcommand group that allows to spawn a daemon, and to stop and restart it. Internally the module usesdaemonocle(the package is not installed withsdsstoolsand needs to be pip-installed manually).A simple example of how to usedaemonizerisimporttimeimportclickfromsdsstools.daemonizerimportDaemonGroup@click.group(cls=DaemonGroup,prog='hello',pidfile='/var/tmp/hello.pid')@click.argument('NAME',type=str)@click.option('--file',type=str,default='hello.dat')defdaemon(name):withopen(file,'w')asunit:whileTrue:unit.write(f'Hi{name}!\n')unit.flush()time.sleep(1)if__name__=='__main__':daemon()This will create a new grouphellowith four subcommandsUsage: daemon [OPTIONS] NAME COMMAND [ARGS]...Options:--file--help  Show this message and exit.Commands:restart  Restart the daemon.start    Start the daemon.status   Report if the daemon is running.stop     Stop the daemon.Now we can rundaemon --file ~/hello.dat John startand a new background process will start, writing to the file every second. We can stop it withdaemon stop. In general the behaviour is identical to thedaemonocle Click implementationbut the internal are slightly different to allow the group callback to accept arguments. If the callback is a coroutine, it can be wrapped with thecli_corodecoratorimportasyncioimportsignalimportclickfromsdsstools.daemonizerimportDaemonGroup,cli_corodefshutdown(signal):ifsignal==signal.SIGTERM:cancel_something()@click.group(cls=DaemonGroup,prog='hello',pidfile='/var/tmp/hello.pid')@click.argument('NAME',type=str)@click.option('--file',type=str,default='hello.dat')@cli_coro(shutdown_func=shutdown,signals=(signal.SIGTERM,signal.SIGINT))asyncdefdaemon(name):withopen(file,'w')asunit:whileTrue:unit.write(f'Hi{name}!\n')unit.flush()awaitasyncio.sleep(1)cli_corocan accept ashutdown_funcfunction that is called when the coroutine receives a signal. The default signals handled are(SIGHUP, SIGTERM, SIGINT).Daemonizing a commandTo execute any command as a daemon you can use thedaemonizescript that is installed withsdsstools. To start the process as a daemon dodaemonize start NAME COMMANDwhenNAMEis the name associated to the daemon (so that it can be stopped later) andCOMMANDis the command to run, for example:daemonize start apoActor python ./apoActor_main.pyTo stop the daemon dodaemonize stop NAME. Seedaemonize --helpfor more options.Date functionsThe functionsdsstools.time.get_sjd()returns the integer with the SDSS-style Modified Julian Day. The function accepts an observatory ('APO'or'LCO') but otherwise will try to determine the current location from environment variables or the fully qualified domain name.Bundled packagesFor convenience,sdsstoolsbundles the following products:(Up to version 1.0.0)A copy ofreleasesthat fixes some issues with recent versions ofsemantic-version. This copy is not available insdsstools>=1.0.0.releasesis not maintained anymore, so use at your own risk.A copy oftomlto read TOML files (used by the metadata submodule).An adapted version of Astropy'scolor_print.A copy of the pydl'syanny module.You can access them directly from the top-level namespace,sdsstools.toml,sdsstools.releases. To usereleaseswith sphinx, simply add the following to yourconfig.pyextensions+=['sdsstools.releases']