Package: amulog
Amulog is a tool to support system log management.
The main function is to classify log messages with automatically generated log templates (formats and variable locations),
and to store the data in a database.
This system works on python3.Source:https://github.com/amulog/amulogBug Reports:https://github.com/amulog/amulog/issuesAuthor:Satoru KobayashiLicense:BSD-3-ClauseMain featuresSupport multiple databases: sqlite and mysqlSmart log segmentation withlog2seqMultiple template generation algorithms such as: Drain, SHISO, LenMa, FT-tree, Dlog, etc.Support Online (incremental) and Offline (hindsight) useSuspend and resume the template generation processImport and Export log templates if you needEdit log templates manually if you needSearch API with datetime, hostname and log template IDsTutorialInstall$ pip install amulogGenerate configFor the first step, save following config astest.confon an empty directory.[general]
src_path = logfile.txt
src_recur = false
logging = auto.log

[database]
database = sqlite3
sqlite3_filename = log.db

[log_template]
lt_methods = drain
indata_filename = ltgen.dumpThen modifygeneral.src_pathoption to a logfile you want to load.
(If you want to use multiple files, changegeneral.src_recurinto true and specify directory name togeneral.src_path.)Generate databaseTry following command to generate database:$ python -m amulog db-make -c test.confCheck database$ python -m amulog show-db-info -c test.confshows status of the generated database.$ python -m amulog show-lt -c test.confshows all generated log templates in the given logfile.$ python -m amulog show-log -c test.conf ltid=2shows all log messages corresponding to log template ID 2.Resume generating databaseTry following command to resume generating database:$ python -m amulog db-add -c test.conf logfile2.txtExport and Import templatesFollowing command exports all log templates in the database:$ python3 -m amulog show-db-import -c test.conf > exported_tpl.txtYou can modify the exported templates manually.
Note that some special letters (\\,@,*) are escaped in the exported templates.To import the templates, save following config astest2.conf.[general]
src_path = logfile.txt
src_recur = false
logging = new_auto.log

[database]
database = sqlite3
sqlite3_filename = new_log.db

[log_template]
lt_methods = import
indata_filename = new_ltgen.dump

[log_template_import]
def_path = exported_tpl.txtThen, try generating database again:python -m amulog db-make -c test2.confFurther usagesee help with following command:python -m amulog -hReferenceThis tool is demonstrated atInternational Journal of Network ManagementandCNSM2020.If you use this code, please consider citing:@article{Kobayashi_IJNM2022,
  author = {Kobayashi, Satoru and Yamashiro, Yuya and Otomo, Kazuki and Fukuda, Kensuke},
  title = {amulog: A general log analysis framework for comparison and combination of diverse template generation methods*},
  journal = {International Journal of Network Management},
  volume = {32},
  number = {4},
  pages = {e2195},
  doi = {https://doi.org/10.1002/nem.2195},
  year = {2022}
}

@inproceedings{Kobayashi_CNSM2020,
  author = {Kobayashi, Satoru and Yamashiro, Yuya and Otomo, Kazuki and Fukuda, Kensuke},
  booktitle = {Proceedings of the 16th International Conference on Network and Service Management (CNSM'20)},
  title = {amulog: A General Log Analysis Framework for Diverse Template Generation Methods},
  pages={1-5},
  year = {2020}
}