semvec/pypi-packages huggingface dataset:
crate-sys-log-scraper 1.0.1
A tool to get information of the CrateDB sys.jobs_log table and present it in a meaningful way
Requires Python: >=3.6

![Code Quality](https://github.com/kamejosh/crate_sys_log_scraper/workflows/Code%20Quality/badge.svg)
![Published](https://github.com/kamejosh/crate_sys_log_scraper/workflows/Python%20Package/badge.svg)

# crate_sys_log_scraper

`crate_sys_log_scraper` (`SLS` for short) is a tool that can be used to periodically scrape the CrateDB `sys.jobs_log` table and gather statistics and insights to the statements executed in the Database.

## Why use crate_sys_log_scraper

As `sys.jobs_log` is limited to a maximum value of logs saved (and setting it to a very large number would drastically increase it's footprint) statements that have a large impact on the database can be missed. For example when using the jobs_log filter to look for queries with a high duration a query with a shorter duration but which is execute many time more often will not be in the jobs_log. With the `SLS` the `sys.jobs_log` table is scraped and the most important metrics are saved to a table for an simple overview and without creating limitless rows.

## Using crate_sys_log_scraper

Install this tool:

```bash
pip install crate_sys_log_scraper
```

Run it from the commandline with:

```bash
sls
```

This starts the `SLS` with it's default settings. The default settings can be viewed with the following command:

```bash
sls -h
```

## Settings

This chapter gives an overview over the setting of the `SLS`. Settings can either be set with environment variables or as commandline argument.

### HOST

Environment variable: `HOST`
Commandline argument: `--host`

Type: string

Value: The Hostname of the CrateDB Instance including the port

Default: `localhost:4200`

### USERNAME

Environment variable: `USERNAME`
Commandline argument: `--username`

Type: string

Value: The username of CrateDB. **Must be the superuser**

Default: `crate`

### PASSWORD

Environment variable: `PASSWORD`
Commandline argument: `--password`

Type: string

Value: The password of the CrateDB superuser.

Default: `emptystring`

### INTERVAL

Environment variable: `INTERVAL`
Commandline argument: `--interval`

Type: float

Value: The wait time between scrapes

Default: `1.0`

### STMT_TABLE

Environment variable: `STMT_TABLE`
Commandline argument: `--stmt_table`

Type: string

Value: The table name where the scrape results will be written to.

Default: `stats.statement_log`

### LAST_EXEC_TABLE

Environment variable: `LAST_EXEC_TABLE`
Commandline argument: `--last_exec_table`

Type: string

Value: The table name where the timestamp of the last scrape will be written to.

Default: `stats.last_execution`
scutlogin 0.0.1
scut login server
Requires Python: >=3.6

1. Purpose: login the Unified identity authentication system of South China University of Technology

        Warning: Do not use for illegal purposes!

2. Requirements: execjs 
 
        pip install pyexecjs

3. Call function:

    (1) Method:

        from scutlogin.login import *
        GetLoginState().login(un, pd, server_url)

    (2) Parameter description:

        un --> student number
        pd --> student password
        server_url --> service url

    (3) Rules of service url:

        https://sso.scut.edu.cn/cas/login? --> location of unified identity authentication system
        service=server_url --> the server url you need

    (4) Examples:

        IAmOK --> https://sso.scut.edu.cn/cas/login?service=https://iamok.scut.edu.cn/cas/login
        Integrated information service --> https://sso.scut.edu.cn/cas/login?service=https://my.scut.edu.cn/up/
        Student educational administration system (2018) --> https://sso.scut.edu.cn/cas/login?service=http://jw2018.jw.scut.edu.cn/sso/driotlogin
 
    (5) If login is successful:

        Directly use <r.post> or <r.get> to send request
scutLogin-flyingdream 0.0.1
scut login server
Requires Python: >=3.6

1. Purpose: login the Unified identity authentication system of South China University of Technology

        Warning: Do not use for illegal purposes!

2. Requirements: execjs 
 
        pip install pyexecjs

3. Call function:

    (1) Method:

        from scutLogin.login import *
        GetLoginState().login(un, pd, server_url)

    (2) Parameter description:

        un --> student number
        pd --> student password
        server_url --> service url

    (3) Rules of service url:

        https://sso.scut.edu.cn/cas/login? --> location of unified identity authentication system
        service=server_url --> the server url you need

    (4) Examples:

        IAmOK --> https://sso.scut.edu.cn/cas/login?service=https://iamok.scut.edu.cn/cas/login
        Integrated information service --> https://sso.scut.edu.cn/cas/login?service=https://my.scut.edu.cn/up/
        Student educational administration system (2018) --> https://sso.scut.edu.cn/cas/login?service=http://jw2018.jw.scut.edu.cn/sso/driotlogin
 
    (5) If login is successful:

        Directly use <r.post> or <r.get> to send request