Package: router-log-preprocessor
Router Log PreprocessorGarbage in, garbage out—George FuechselPreprocessors upcycle garbage input data into well-structured data to ensure reliable and accurate event handling in third-party systems such as Zabbix.
By parsing and filtering the input log data, the preprocessor helps to ensure that only high-quality data are sent for further analysis and alerting.
This helps to minimize false positives and ensure that network administrators receive reliable and actionable alerts about potential security threats or other issues.Key features:Wireless LAN Controller eventlog entries are parsed to tangible enumerationsDNSMASQ DHCPlog entries are parsed to catch which IP a given client is assigned toZabbixtemplates are included to ensure that the logs are can lead to actionable alertsExtendablepreprocessors and hooks to ensure future reliable information to network administratorsInstallation$pipinstallrouter-log-preprocessorIf needed it can also be installed from sources.
RequiresPoetry 1.3.2.$gitpullhttps://github.com/mastdi/router-log-preprocessor.git$cdrouter-log-preprocessor$poetryinstallUsageInstalling the package using pip also creates the executable script namedrouter-log-preprocessor.
On Linux systems the router log preprocessor can be run by./router-log-preprocessorThe configuration solely happens through environment variables or a.envconfiguration file located in the current working directory.
The most important variables are documented below.
A full sample can be found in.env.
The application reads, in order of the least priority to the highest file:.env,.env.dev,.env.test,.env.staging,.env.prod,meaning that values stored in.env.prodwill overwrite any values from other dovenv files.
Parameters stored in environment variables will always take priority over values loaded from a dotenv file.# Purpose: Specifies the IP address or hostname of the local interface to which the
# logging system should bind.
# Format: A string containing a valid IP address or hostname, such as "192.168.0.1" or
# "example.com".
LOG_SERVER_HOST="0.0.0.0"

# Purpose: Specifies the port number of the server to which log data should be sent.
# Format: An integer representing a valid port number, such as 514.
LOG_SERVER_PORT=8514

# Purpose: Specifies the hostname or IP address of the Zabbix server to which the
# Zabbix Sender should send monitoring data.
# Format: A string containing a valid hostname or IP address, such as "example.com" or
# "192.168.0.1".
ZABBIX_HOST="example.com"

# Purpose: Specifies the port number on which the Zabbix server is running and to
# which the Zabbix Sender should send monitoring data.
# Format: An integer representing a valid port number, such as 10051.
ZABBIX_PORT=10051As a serviceThis part will go through the steps to set up the Router Log Preprocessor as a service on Ubuntu.
The following steps will be explained in details below:Create a service userCreate a virtual environmentConfigure environment variablesSet up the logging directoryCreate the service fileStart the service and check its statusDebugging the service using journalctlPrerequisitesTo install and set up the Router Log Preprocessor as a service on Ubuntu, you will need:Python 3.8 installed or highervenvpackage (can be installed viaapt-get install python3-venvcommand)pippackage manager (should be included after activating the virtual environment)Internet connection to download the Router Log Preprocessor package from PyPI and the.envfile from the GitHub repositoryNote that some of these prerequisites may already be installed on your Ubuntu system.
You can check if Python and pip are installed by running the following commands:python3 --versionpip --versionIf both of these commands return the version number of Python and pip, respectively, you're good to go.
Otherwise, you will need to install Python and pip on your Ubuntu system before proceeding.Creating a service userFirst, we will create a service user to run the Router Log Preprocessor.
This is a good security practice as running the script as a root user is not needed.Run the following commands to create a new user calledrlp:sudo adduser rlp --disabled-password --gecos ""sudo su rlpcd ~The first command creates a new user calledrlpwith a disabled password and no additional information.
The second command switches to the new user and moves to their home directory.Creating a virtual environmentNow that we have a service user set up, we can create a virtual environment to install the Router Log Preprocessor.
The following commands create a virtual environment using Python 3.8 and install the Router Log Preprocessor package:python3 -m venv venvcd venvsource bin/activatepip install router-log-preprocessorThese commands create a new virtual environment in the venv directory, activate the environment, and install the Router Log Preprocessor package.Configuring the environment variablesWe will create a .env file in the virtual environment directory to store these variables.
You can copy the default.envfile from the Router Log Preprocessor repository and customize it according to your needs:curl -o .env https://raw.githubusercontent.com/mastdi/router-log-preprocessor/master/.envnano .envThis will download the .env file from the Router Log Preprocessor repository and open it in the Nano text editor.
Customize the file to set the environment variables you need.We are done setting up the servie user:exitSetting up the logging directoryIf you have set theLOGGING_DIRECTORYvariable in the .env file to/var/log/rlp, you need to create the directory and set the ownership to the rlp user:sudo mkdir /var/log/rlpsudo chown rlp:rlp /var/log/rlpThese commands will create the/var/log/rlpdirectory and set the ownership to therlpuser.Creating the service fileThe next step is to create a service file for the Router Log Preprocessor.
This file specifies how the service should be started and managed by the system.
Create a new file called rlp.service in the /etc/systemd/system/ directory using the following command:sudo nano /etc/systemd/system/rlp.serviceThis will open the text editor with a new file.
Copy the following text into the file:[Unit]Description=Router Log Preprocessor serviceAfter=network.target[Service]User=rlpWorkingDirectory=/home/rlp/venvEnvironment="PATH=/home/rlp/venv/bin"EnvironmentFile=/home/rlp/venv/.envExecStart=/home/rlp/venv/bin/router-log-preprocessorRestart=on-failureRestartSec=5sStartLimitInterval=60sStartLimitBurst=3[Install]WantedBy=multi-user.targetStarting the serviceThe service is now ready to be started.
The final step is to start the service, check if it is running, and ensuring that the service starts automatically on system boot.
Start the service using the following command:sudo systemctl start rlp.serviceThis starts the service based on the configuration we just provided.
Check that the service is started using the following command:sudo systemctl status rlp.serviceThis should showactive (running)in the console.
To make sure the service is started on system boot use the following command:sudo systemctl enable rlp.serviceDebugging the service creationTo debug any issues with the service, you can use thejournalctlcommand.
For example, to view the logs of therlpservice, run the following command:sudo journalctl -u rlp.service -eThis will show the logs of therlpservice and the-eflag will show the end of the logs.
You can use other flags like-fto follow the logs in real-time,-nto specify the number of lines to show, and-rto show the logs in reverse order (most recent first).If the service crashes, you can also use the--sinceand--untilflags to show the logs between a specific time range.
For example, to show the logs of the last 10 minutes, run the following command:sudo journalctl -u rlp.service --since "10 minutes ago"This will show the logs of therlpservice that were generated in the last 10 minutes.
Use this command to debug any issues with the service.