Package: hao
haoconfigurations, logs and others.installpipinstallhaopreconditionThe folder contained any of the following files (searched in this very order) will be treated asproject root path.pyproject.tomlrequirements.txtsetup.pyLICENSE.idea.git.vscodeIf your project structure does NOT conform to this, it will not work as expected.featuresconfigIt will try to load YAML config file fromconffolder.                               # project root
├── conf
│   ├── config-{env}.yml        # if `export env=abc`, will raise error if not found
│   ├── config-{hostname}.yml   # try to load this file, then the default `config.yml`
│   └── config.yml              # the default config file that should always exist
├── pyproject.toml              # or requirements.txt
├── .gitIn following order:ifos.environ.get("env")isnotNone:try_to_load(f'config-{env}.yml',fallback='config.yml')# echo $envelse:try_to_load(f'config-{socket.gethostname()}.yml',fallback='config.yml')# echo hostnameSay you have the following content in your config file:# config.ymles:default:host:172.23.3.3port:9200indices:-news-papersThe get the configured values in your code:importhaoes_host=hao.config.get('es.default.host')# stres_port=hao.config.get('es.default.port')# intindices=hao.config.get('es.default.indices')# list...logsSet the logger levels to filter logse.g.# config.ymllogging:__main__:DEBUGtransformers:WARNINGlightning:INFOpytorch_lightning:INFOelasticsearch:WARNINGtests:DEBUGroot:INFO# root levelSettings for logger:# config.ymllogger:format:"%(asctime)s%(levelname)-7s%(name)s:%(lineno)-4d-%(message)s"# overwrite to change to other formathandlers:TimedRotatingFileHandler:# any Handlers in `logging` and `logging.handlers` with it's configwhen:dbackupCount:3Examplelogger:format:"%(asctime)s%(levelname)-7s%(name)s:%(lineno)-4d-%(message)s"# overwrite to change to other formathandlers:stdout:format:"%(asctime)s%(levelname)-7s%(name)s:%(lineno)-4d-%(message)s"# overwrite to change to other formatfile:format:"%(message)s"# overwrite to change to other formathandler:TimedRotatingFileHandlerargs:when:dbackupCount:3filename:test.logrolling-file:handler:TimedRotatingFileHandlerargs:when:dbackupCount:3filename:hello.loglogging:root:INFOtorch.models:INFO__main__:DEBUGaccess:level:INFOhandlers:-stdout-rolling-filetest:level:INFOhandlers:-fileDeclare and user the loggerimporthaoLOGGER=hao.logs.get_logger(__name__)LOGGER.debug('message')LOGGER.info('message')LOGGER.warnning('message')LOGGER.error('message')LOGGER.exception(err)namespacesimporthaofromhao.namespacesimportfrom_args,attr@from_argsclassProcessConf(object):file_in=attr(str,required=True,help="file path to process")file_out=attr(str,required=True,help="file path to save")tokenizer=attr(str,required=True,choice=('wordpiece','bpe'))fromargparseimportNamespacefrompytorch_lightningimportTrainer@from_args(adds=Trainer.add_argparse_args)classTrainConf(Namespace):root_path_checkpoints=attr(str,default=hao.paths.get_path('data/checkpoints/'))dataset_train=attr(str,default='train.txt')dataset_val=attr(str,default='val.txt')dataset_test=attr(str,default='test.txt')batch_size=attr(int,default=128,key='train.batch_size')# key means try to load from config.yml by the keytask=attr(str,choices=('ner','nmt'),default='ner')seed=attr(int)epochs=attr(int,default=5)Whereattris a wrapper forargpars.add_argument()Usage 1: overwrite the default value from command linepython-myour_module--task=nmtUsage 2: overwrite the default value from constructortrain_conf=TrainConf(task='nmt')Value lookup order:command lineconstructorconfig yml ifkeyspecified inattrdefaultif specified inattr