Package: contek-tusk
Tusk Clickhouse Metric Writercontek-tuskis a metric writer built upon Clickhouse.1. Connection Initializationimportcontek_tuskastusktusk.init(host='localhost',user='tester',password='P@assw0rd')2. Generic Metricfromcontek_tuskimportmetriccpu_usage=metric(table='system.cpu')cpu_usage.write({'region':'Tokyo','instance_id':'ABC','usage':0.43})cpu_usage.write({'region':'Seoul','instance_id':'XYZ','usage':0.22})SELECT*FROMsystem.cpuregioninstance_iddatetimeusageTokyoABC2022-03-10 06:42:42.7460.43SeoulXYZ2022-03-10 06:42:42.7460.22Note: the client will automatically use the firstDateTimecolumn to record current timestamp. Alternatively, it
can be explicitly specified:fromcontek_tuskimportmetricfromcontek_tusk.tableimportTabletable=Table(database='system',table_name='cpu',time_column='datetime')cpu_usage=metric(table)3. Summing Counterfromcontek_tusk.counterimportcountercounter=counter(table='download.quota_usage')counter.count({'resource_id':'ABC111','requester_ip':'1.2.3.4'})# default count is 1counter.count({'resource_id':'ABC122','requester_ip':'2.2.2.4'},3)counter.count({'resource_id':'ABC111','requester_ip':'1.2.3.4'},4)# the new count is 1 + 4 = 5counter.count({'resource_id':'ABC122','requester_ip':'1.2.3.4'},2)SELECT*FROMdownload.quota_usageresource_idrequester_ipdatetimecountABC1111.2.3.42022-03-10 06:42:48.4115ABC1221.2.3.42022-03-10 06:42:48.4112ABC1222.2.2.42022-03-10 06:42:48.4113Note: the client will automatically use the firstIntorUIntcolumn to record the count. Alternatively, it
can be explicitly specified:fromcontek_tusk.counterimportcountercounter=counter(table='download.quota_usage',count_column='count')4. Heartbeat Recorderfromcontek_tusk.heartbeatimportinitinit(table='maintenance.heartbeats',app_name='my_app')After initialization, a recurring task starts in the background and sends a heartbeat to the database every 30 seconds.
The heartbeat cycle can be explicitly specified:fromcontek_tusk.heartbeatimportinitinit(table='maintenance.heartbeats',app_name='my_app',heartbeat_period=15)If the application is running multiple recurring tasks. Their heartbeats can also be registered:fromcontek_tusk.heartbeatimportbeatbeat(task='scan_disk',heartbeat_period=60)SELECT*FROMmaintenance.heartbeatsapptaskdatetimesequencemy_appmain2022-03-10 06:51:01.1780my_appmain2022-03-10 06:51:31.1781my_appmain2022-03-10 06:52:01.1782my_appscan_disk2022-03-10 06:52:04.2150my_appmain2022-03-10 06:52:31.1783my_appmain2022-03-10 06:53:01.1784my_appscan_disk2022-03-10 06:53:04.21515.Loggingimportloggingfromcontek_tusk.loggingimportinitlogger=logging.getLogger(__name__)init(table='maintenance.logs',app_name='my_app')logger.info('Hello world!')logger.warning('Fire drill test.')logger.error('This is real fire.')try:raiseValueError('Guess what?')exceptValueError:logger.exception('Oh no!')logger.critical('I give up.')SELECT*FROMmaintenance.logsappdatetimelevelloggerlineerrormessagestacktracemy_app2022-03-10 08:53:03.5584example.py7Hello world!my_app2022-03-10 08:53:03.5583example.py8Fire drill test.my_app2022-03-10 08:53:03.5582example.py9This is real fire.my_app2022-03-10 08:53:03.5582example.py13ValueErrorOh no!(traceback)my_app2022-03-10 08:53:03.5581example.py14I give up.6. Full InitializationStandardimportcontek_tuskastuskimportcontek_tusk.heartbeatasheartbeatimportcontek_tusk.loggingasloggingtusk.init(host='localhost',user='tester',password='P@assw0rd')tusk.set_app_name('my_app')heartbeat.init(table='maintenance.heartbeats')logging.init(table='maintenance.logs')YAMLapp:'my_app'host:'localhost'user:'tester'password:'P@assw0rd'heartbeat:database:'maintenance'table:'heartbeats'logging:database:'maintenance'table:'logs'importcontek_tusk.yamlastusk_yamltusk_yaml.for_yaml_file('example.yml')7. Example Table Schemamaintenance.heartbeatsCREATETABLEIFNOTEXISTSmaintenance.heartbeats(`app`String,`datetime`DateTime64(3),`task`String,`sequence`UInt32,`expiry`DateTime64)ENGINE=MergeTree()ORDERBY(`app`,`datetime`,`task`,`sequence`)TTLtoDateTime(datetime)+INTERVAL28DAYDELETE;maintenance.logsCREATETABLEIFNOTEXISTSmaintenance.logs(`app`String,`datetime`DateTime64(3),`level`Int8,`logger`String,`line`UInt32,`error`String,`message`String,`stacktrace`String)ENGINE=MergeTree()ORDERBY(`category`,`host`,`app`,`datetime`,`level`,`logger`,`line`,`error`)TTLtoDateTime(datetime)+INTERVAL7DAYDELETE;8. CustomizationTo write to tables with different schemas. Specify the column names inTuskHeartbeatConfigandTuskLoggingConfig.