Package: logs-analyzer
Logs-analyzerLogs-analyzer is a Python based library containing functions that can help you extract usable data from logs.StatusMaster:Develop:QuickstartSupportPython 2:2.7Python 3:3.6, 3.7Installusing pip :pip install logs-analyzerCode samplefromlogs_analyzer.libimportLogsAnalyzernginx_logsanalyzer=LogsAnalyzer('nginx')nginx_logsanalyzer.add_date_filter(minute='*',hour=22,day=4,month=5)nginx_logsanalyzer.add_filter('192.10.1.1')requests=nginx_logsanalyzer.get_requests()Non-object functionsFunction get_service_settingsGet default settings for the said service from the settings file, three type
of logs are supported right now:nginx,apache2andauth.Parametersservice_name:service name  (e.g. nginx, apache2...).ReturnReturns a dictionary containing the chosen service settings orNoneif the
service doesn't exists.Samplenginx_settings = get_service_settings('nginx')Function get_date_filterGet the date's pattern that can be used to filter data from
logs based on the parameters.Parameterssettings:the target logs settings.minute:default now, minutes or * to ignore.hour:default now, hours or * to ignore.day:default now, day of month.month:default now, month number.year:default now, year.Samplenginx_settings=get_service_settings('nginx')date_pattern=get_date_filter(nginx_settings,13,13,16,1,1989)print(date_pattern)Prints[16/Jan/1989:13:13Function filter_dataFilter received data/file content and return the results.Parameterslog_filter:string that will be used to filter datadata:data to be filtered (String) or None if the data will
be loaded from a file.filepath:filepath from where data will be loaded or None if
the data has been passed as a parameter.is_casesensitive:if the filter has to be case sensitive
(default True).is_regex:if the filter string is a regular expression
(default False).is_reverse:(boolean) invert selection.ReturnReturns filtered data (String).Samplenginx_settings=get_service_settings('nginx')date_filter=get_date_filter(nginx_settings,'*','*',27,4,2016)base_dir=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))file_name=os.path.join(base_dir,'logs-samples/nginx1.sample')data=filter_data('192.168.5',filepath=file_name)data=filter_data(date_filter,data=data)Function get_web_requestsAnalyze the web logs (Nginx & Apache2 for now) data and return list of requests
formatted as the model (pattern) defined.Parametersdata:(String) data to analyzed.pattern:(Regular expression) used to extract requests.date_pattern:(Regular expression or None) used to extract date elements
to have ISO formatted dates.date_keys:(List or None) list of extracted date elements placements.Sampleapache2_settings=get_service_settings('apache2')base_dir=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))file_name=os.path.join(base_dir,'logs-samples/apache1.sample')data=filter_data('127.0.1.1',filepath=file_name)requests=get_web_requests(data,apache2_settings['request_model'],nginx_settings['date_pattern'],nginx_settings['date_keys'])Function get_auth_requestsAnalyze the Auth logs data and return list of requests
formatted as the model (pattern) defined.Parametersdata:(String) data to analyzed.pattern:(Regular expression) used to extract requests.date_pattern:(Regular expression or None) used to extract date elements
to have ISO formatted dates.date_keys:(List or None) list of extracted date elements placements.Sampleauth_settings=get_service_settings('auth')base_dir=os.path.dirname(os.path.dirname(os.path.abspath(__file__)))date_filter=get_date_filter(auth_settings,'*',22,4,5)file_name=os.path.join(base_dir,'logs-samples/auth.sample')data=filter_data('120.25.229.167',filepath=file_name)data=filter_data(date_filter,data=data)requests=get_auth_requests(data,auth_settings['request_model'],auth_settings['date_pattern'],auth_settings['date_keys'])