Package: csv-logger
csv-loggerSimple class to log to csv using the logging rotating handler, output is a rolling csv logDescriptionThis library allows you to easily log information to CSV file format, in the same fashion as the logging package. This allows you to generate a rolling set of csv logs with a maximum file size and file count.Inputs:filenamemain log file name or path. if path, will create subfolders as neededdelimiterdelimiter to use in the files. Defaults to','levellogging level for logs, below which the logs will not be written to file. defaultINFOadd_level_nameslist fo strings, adds additional logging levels for custom log tagging. default:[]add_level_numsassigns specific nums toadd_level_names. default if None provided:[100,99,98,..]fmtoutput format. defaultf'%(asctime)s,%(message)s'. accepts:%(name)sName of the logger (logging channel)%(levelno)sNumeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL)%(levelname)sText logging level for the message ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL")%(pathname)sFull pathname of the source file where the logging call was issued (if available)%(filename)sFilename portion of pathname%(module)sModule (name portion of filename)%(lineno)dSource line number where the logging call was issued (if available)%(funcName)sFunction name%(created)fTime when the LogRecord was created (time.time() return value)%(asctime)sTextual time when the LogRecord was created%(msecs)dMillisecond portion of the creation time%(relativeCreated)dTime in milliseconds when the LogRecord was created, relative to the time the logging module was loaded (typically at application startup time)%(thread)dThread ID (if available)%(threadName)sThread name (if available)%(process)dProcess ID (if available)%(message)sThe result of record.getMessage(), computed just as the record is emitteddatefmtdate format for first column of logs. default%Y/%m/%d %H:%M:%Smax_sizemax size of each log file in bytes. default10MB(10,485,760)max_filesmax file count. default10headerheader to prepend to csv files. defaultNoneGetting startedInstall withpip3 install csv_loggerBasic usage example below.Since the example is set to only 1kB of data per file, the code results in 2 log files.log.csvwill always contain the most recent data, and the subsequent files (log_1.csvand so on) will have older data.#!/usr/bin/python3fromcsv_loggerimportCsvLoggerimportloggingfromtimeimportsleepfilename='logs/log.csv'delimiter=','level=logging.INFOcustom_additional_levels=['logs_a','logs_b','logs_c']fmt=f'%(asctime)s{delimiter}%(levelname)s{delimiter}%(message)s'datefmt='%Y/%m/%d%H:%M:%S'max_size=1024# 1 kilobytemax_files=4# 4 rotating filesheader=['date','level','value_1','value_2']# Creat logger with csv rotating handlercsvlogger=CsvLogger(filename=filename,delimiter=delimiter,level=level,add_level_names=custom_additional_levels,add_level_nums=None,fmt=fmt,datefmt=datefmt,max_size=max_size,max_files=max_files,header=header)# Log some recordsforiinrange(10):csvlogger.logs_a([i,i*2])sleep(0.1)# You can log list or stringcsvlogger.logs_b([1000.1,2000.2])csvlogger.critical('3000,4000')# Log some more records to trigger rolloverforiinrange(50):csvlogger.logs_c([i*2,float(i**2)])sleep(0.1)# Read and print all of the logs from file after loggingall_logs=csvlogger.get_logs(evaluate=False)forloginall_logs:print(log)log_2.csv:date,level,value_1,value_2
2022/01/31 15:49:53,logs_a,0,0
2022/01/31 15:49:53,logs_a,1,2
2022/01/31 15:49:53,logs_a,2,4
2022/01/31 15:49:53,logs_a,3,6
2022/01/31 15:49:53,logs_a,4,8
2022/01/31 15:49:53,logs_a,5,10
2022/01/31 15:49:53,logs_a,6,12
2022/01/31 15:49:53,logs_a,7,14
2022/01/31 15:49:53,logs_a,8,16
2022/01/31 15:49:54,logs_a,9,18
2022/01/31 15:49:54,logs_b,1000.1,2000.2
2022/01/31 15:49:54,CRITICAL,3000,4000
2022/01/31 15:49:54,logs_c,0,0.0
2022/01/31 15:49:54,logs_c,2,1.0
2022/01/31 15:49:54,logs_c,4,4.0
2022/01/31 15:49:54,logs_c,6,9.0
2022/01/31 15:49:54,logs_c,8,16.0
2022/01/31 15:49:54,logs_c,10,25.0
2022/01/31 15:49:54,logs_c,12,36.0
2022/01/31 15:49:54,logs_c,14,49.0
2022/01/31 15:49:54,logs_c,16,64.0
2022/01/31 15:49:55,logs_c,18,81.0
2022/01/31 15:49:55,logs_c,20,100.0
2022/01/31 15:49:55,logs_c,22,121.0
2022/01/31 15:49:55,logs_c,24,144.0
2022/01/31 15:49:55,logs_c,26,169.0
2022/01/31 15:49:55,logs_c,28,196.0
2022/01/31 15:49:55,logs_c,30,225.0
2022/01/31 15:49:55,logs_c,32,256.0log_1.csv:date,level,value_1,value_2
2022/01/31 15:49:55,logs_c,34,289.0
2022/01/31 15:49:55,logs_c,36,324.0
2022/01/31 15:49:56,logs_c,38,361.0
2022/01/31 15:49:56,logs_c,40,400.0
2022/01/31 15:49:56,logs_c,42,441.0
2022/01/31 15:49:56,logs_c,44,484.0
2022/01/31 15:49:56,logs_c,46,529.0
2022/01/31 15:49:56,logs_c,48,576.0
2022/01/31 15:49:56,logs_c,50,625.0
2022/01/31 15:49:56,logs_c,52,676.0
2022/01/31 15:49:56,logs_c,54,729.0
2022/01/31 15:49:57,logs_c,56,784.0
2022/01/31 15:49:57,logs_c,58,841.0
2022/01/31 15:49:57,logs_c,60,900.0
2022/01/31 15:49:57,logs_c,62,961.0
2022/01/31 15:49:57,logs_c,64,1024.0
2022/01/31 15:49:57,logs_c,66,1089.0
2022/01/31 15:49:57,logs_c,68,1156.0
2022/01/31 15:49:57,logs_c,70,1225.0
2022/01/31 15:49:57,logs_c,72,1296.0
2022/01/31 15:49:57,logs_c,74,1369.0
2022/01/31 15:49:58,logs_c,76,1444.0
2022/01/31 15:49:58,logs_c,78,1521.0
2022/01/31 15:49:58,logs_c,80,1600.0
2022/01/31 15:49:58,logs_c,82,1681.0
2022/01/31 15:49:58,logs_c,84,1764.0
2022/01/31 15:49:58,logs_c,86,1849.0log.csv:date,level,value_1,value_2
2022/01/31 15:49:58,logs_c,88,1936.0
2022/01/31 15:49:58,logs_c,90,2025.0
2022/01/31 15:49:58,logs_c,92,2116.0
2022/01/31 15:49:58,logs_c,94,2209.0
2022/01/31 15:49:59,logs_c,96,2304.0
2022/01/31 15:49:59,logs_c,98,2401.0AuthorJames Morris (https://james.pizza)LicenseFree software: MIT licenseCreditsPython CSV Rotating Logger gistas starting point