pysqream-log-analyzer 0.0.3
The log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.
Requires Python: >=3.9

# Analyzer SQream Logs

The log analyzer has the ability to help us find the correct logs faster and in a more intuitive way.

* **Version:**  0.0.3

Requirements
=====================

* Python 3.9+

How to use the tool
=====================
run either:

    ./sq_log_analyzer.py --help 
    python3 sq_log_analyzer.py --help

This will show the full syntax with all options and acronyms, as well as, help details if you are not sure of the use.

Please note, using --help (or -h) will override any other arguments, which can help when in doubt


Flags
======

1.Load logs
------------

The tool needs to know which files to use, we can specify a directory or a specific file:

    --file-path (-p), requires a specific file path
    specify the full path (e.g. /media/weka_delivery/ti_folder/sqream_20230830_results.csv)
    specify just file name, the tool will search in the current directory
    Usage: --file-path=”sqream_20230830_results.csv” or --file-path=”/media/weka_delivery/ti_folder/sqream_20230830_results.csv”

    --dir-path (-P), require a specific directory path
    specify the full path (e.g. /media/weka_delivery/ti_folder/10082023)
    specify just file name, the tool will search in the current directory
    Usage: --file-path=”10082023” or --file-path=”/media/weka_delivery/ti_folder/10082023”

the tool will search in all the subdirectories for sqream_logs files

Use the file filters (--file-date-filter/-fdf or --file-month-filter/-fmf) to help the tool search only the relevant files

2.Filters
----------
Use as many filters as relevant to your search:

    --from-date/--to-date: date and time filters
    --message-filter: any message in the logs, including, queries, known phrases (e.g. query before parsing)
    --info-level: INFO, ERROR or DEBUG (most logs are info)
    --ip-address: the server ip address
    --worker-port: the worker port
    --statement-id: the statement id if you are looking for a specific statement
    --connection-id: the connection id
    --service-name: service name
    --message-type: the number of the message, like 200 for execution plan

3.Additional Parameters
-------------------------
The tool support querying the results using mysql syntax:

    use --query=”specific query” or --query-file=”file_path.sql” for query in a file

if you use column names with space in the name (like “statement id”), you must use the query file as the name must come in double quotes.

The tool support saving results to file:

    use --output-dir=”directory_name” to save the file as out.csv in the directory
    use --output-path=”file_path.csv” to save to a specific file

please note, using the same directory dir will override old files

Important
===========
* The tool currently supports only the SQreamDB logs (the one inside the cluster)

* The tool will only read files with the pattern: sqream_yyyymmdd_*.log

* When specifying a directory or a file, it will search in the current directory unless a full path is specified.

* The name of the table if you add a query is df, always

* When running a query a warning may appear (“UserWarning: the 'timedelta' type is not supported”), ignore it

* Queries may take long to run, so make sure it runs on the smallest dataset possible