Package: pytong
Rich Tong's Not-so-Famous Python UtilitiesFor convenience, some useful utilities because importing Python modules is
pretty unfun and it is easy to just publish them to PyPi and use them
everywhere.Version 2 LoggingThe previous version 1 logging required complex keeping of log
names with a Log_root. Turns out after much reading this is much simpler
than I thought. The trick is that all that is happening is that
logging has virtual tree based on the name of the logger, so a logger namednetqr.paramsmeans that in the directory./netqr/params.pyis where the
module (that is the file) is located. This is kept in a dunder__name__So instead of having to do base names and so forth, you can just calculate log
names based on thenameand then add some more information.The way that this works is that you first create alogging.yamlthat has the
logging parameters. Then at the top of each file (aka module) you manipulate a
global variable with this to get the current active loggers:importlogginglog=logging.getLogger(__name__)This is the main thing needed becauseIn most cases though, you will actually be better off just adding differentformat
stringssince they have so much more data in them, for example, the name of the
function and then the type s for string, d for digit, etc.Note that you can also says10swhich means the first 10 characters of the
string only and it becomes a fixed width field:StringMeaning%{asctime}sLogRecord creation time%{filename}sCurrent file%{funcName}sCurrent function%{levelname}sLogging level (eg DEBUG, INFO)%{levelno}sLogging number {eg 1-100}%{lineno}dCurrent line number%{message}sMessage of log%{module}sFile without the .py%{name}sName of the logger%{pathname}sFull path of file%{process}dProcess id%{processName}sName of process%{thread}dThread id%{threadName}sThread nameSo most of the time you will be fine with this level of detail, the main thing
missing is that for class objects, you are missing some detail, but you know
the file location and the line number.The second trick is that if you want more logging detail then in a function you
can create a logger by create more dotted names, :importlogging# this logger will be used for all functions in this modulelog=logging.getLogger(__name__)classnew_class():def__init__(self):# use the class logger for all methods that appends the class type to the# current file/moduleself.log=logging.getLogger(__name__+'.'+type(self).__name__)defadd():# log with the instance name and class typeself.log("Entering a new method to add to the database")Version 2 Logging helper functionThis is much simpler than version 1 and the only helper needed is to load the
YAML file with the configuration:importloggingfrompytongimportconfig_log# this is the module wide logging by setting a global variablelog=logging.getLogger(__name__)# setLog creates a logger named <module path>/<class name>@setLogclasstest:def__init__(self):# the logger name is '__main__.test'self.log(f"In{self=}")defmain():config_log()# the logger name is '__main__' and funcname will be printedlog.debug(f"{log=}created")Version 1 Logging (deprecated)Usage is pretty simple, we use inheritance to have the right logging compared
with v2 which uses a class decorator.frompytongimportLog,BaseLog#type: ignore# then for each class you create have a log member to remember it# set a log_root which is the top and then on each Class initiation# instantiate a new class# if you set this base class you get loggingclass<YourClass>(BaseLog)# Use the decorator pattern that Keras and other use with chaining# so every class has a set_logger method that returns the class# this allows constructs like foo.set_logger.next_method...# note that we pass the name which is by default __name__# which for classes is the class namedefset_logger(self,name:str=__name__)-><YourClass>:"""Set Log.Setup the root logger and log"""self.log_root=Log(name)self.log=self.log_root.logreturnselfdef<somemethod>(self,<0therfunctions>):# a convenience as self.log is a lot of typinglog=self.loglog.debug(f'I am here with{log=}')Log by call treeThe logging module automatically creates loggers and lists logs by the call
stack. It also sends different prefixes for class calls.BuildingYou can install the needed pieces with and then upload to test.pypi.org or
pypi.orgmakepip-install# edit .envrc with the your API Token for test do not include the pypi- that is# added in the makefilemaketest-pypi# add the pypi api token without the pypi-makepypiTestingThe test scaffolding is not working yet.File LayoutWe adopt the scheme that seems a little redundent where we have a strudture
that is ./pytong/src/pytong and the tests are in ./pytong/tests.Ionelexplains why this is important. And alsoHYnekexplains why as well.The biggest reason for this is that you are forced to install code and can
check for packaging breaks. And you don't want to include test modules with
your source code.