Python | System hardening and compliance reports using Lynis - GeeksforGeeks

08 Nov, 2021
Lynis is a battle-tested security tool for systems running Linux, macOS, or Unix-based operating systems. It performs an extensive health scan of your systems to support system hardening and compliance testing. The project is open-source software with the GPL license and available since 2007.
Since Lynis is flexible, it is used for several different purposes. Typical use cases for Lynis include: 
Security auditing 
Compliance testing (e.g. PCI, HIPAA, SOx) 
Penetration testing 
Vulnerability detection 
System Hardening 
System hardening refers to securing your system from potential Threats and Vulnerabilities. Lynis can be used to generate a detailed report on various Threats and Vulnerabilities in your system. The user or System Administrator can then take the necessary actions to secure the system.Lynis reports are hard to read and usually have a lot of information. Therefore, we use Bash and Python scripts to parse through the report, extract relevant information from the report such as warnings, suggestions and store them in an excel file as a report.
Prerequisites for Lynis –  
Install Lynis on your system by cloning the github repository: https://github.com/CISOfy/lynis
Install the pandas library using the command sudo pip3 install pandas.
Once you have installed Lynis on your system, navigate to the Lynis directory where you will find a bunch of files along with an executable file called Lynis.
Use the bash script (code is given below) to extract relevant information such as warning and suggestions given in the lynis report. create a file called run.sh and copy paste the bash code into that file and type: sudo ./run.sh to run the bash script.
Run the Python script (code is given below) to clean and parse the extracted data and output the relevant information as an excel file.
Below are the Bash and Python scripts –
Bash Script:  
BASH
#!/bin/bash # script to scrape/parse the report file and# extract the relevant details and run the# python script to display the details in a server. echo "running......"echo "" sudo ./lynis audit system --quick # execute warnings. sudo ./warnings.shecho "Generating warnings"echo ""echo "warnings are: "echo "" sudo cat /var/log/lynis-report.dat | grep warning | sed -e "s/warning\[\]\=//g"sudo cat /var/log/lynis-report.dat | grep warning | sed -e "s/warning\[\]\=//g" | cat > warnings.txt echo ""echo "warnings generated"echo "output file: warnings.txt" sudo chmod 755 warnings.txt #execute suggestions.  sudo ./suggestions.shecho "Generating suggestions"echo ""echo "suggestions are: "echo "" sudo cat /var/log/lynis-report.dat | grep suggestion | sed -e "s/suggestion\[\]\=//g" sudo cat /var/log/lynis-report.dat | grep suggestion | sed -e "s/suggestion\[\]\=//g" | cat > suggestions.txt echo ""echo "suggestions generated"echo "output file: suggestions.txt" sudo chmod 755 suggestions.txt  # execute packages. sudo ./packages.shecho "Generating packages"echo ""echo "packages are: "echo "" sudo cat /var/log/lynis-report.dat | grep installed_package | sed -e "s/installed_package\[\]\=//g"sudo cat /var/log/lynis-report.dat | grep installed_package | sed -e "s/installed_package\[\]\=//g" | cat > packages.txt echo ""echo "packages generated"sudo chmod 755 packages.txt  # execute shells.  sudo ./shells.shecho "Generating available shells"echo ""echo "shells are: "echo "" sudo cat /var/log/lynis-report.dat | grep available_shell | sed -e "s/available_shell\[\]\=//g"sudo cat /var/log/lynis-report.dat | grep available_shell | sed -e "s/available_shell\[\]\=//g" | cat > shells.txt echo ""echo "shells generated" echo "output file: shells.txt" sudo chmod 755 shells.txt
Python script: 
Python3
# importing librariesimport pandas as pdfrom pandas import ExcelWriterimport os # function to get the data.def get_data():         warnings = open('warnings.txt', 'r')    suggestions = open('suggestions.txt', 'r')    packages = open('packages.txt', 'r')    shells = open('shells.txt', 'r')     warn_data = warnings.readlines()    sugg_data = suggestions.readlines()    pack_data = packages.read()    shell_data = shells.readlines()     return warn_data, sugg_data, pack_data, shell_data  def clean_data():     warn, sugg, pack, shell = get_data()     warn_clean = []    for line in warn:        warn_clean.append(line.split('|'))     for i in range(len(warn_clean)):        warn_clean[i] = warn_clean[i][:2]        # print(warn_clean[i])     sugg_clean = []       for line in sugg:        sugg_clean.append(line.split('|'))     for i in range(len(sugg_clean)):        sugg_clean[i] = sugg_clean[i][:2]        # print(sugg_clean[i])     pack_clean = []    pack = pack.split('|')    pack_clean = pack    del pack_clean[0]     shell_clean = []     for i in range(len(shell)):        shell_clean.append(shell[i].rstrip('\n'))        # print(shell_clean[i])      return warn_clean, sugg_clean, pack_clean, shell_clean  def convert_to_excel():     warnings, suggestions, packages, shells = clean_data()     try:        os.mkdir('outputs')    except(Exception):        pass     os.chdir('outputs')     warn_packages = []    warn_text = []    for i in range(len(warnings)):        warn_packages.append(warnings[i][0])     for i in range(len(warnings)):        warn_text.append(warnings[i][1])         print(warn_packages, warn_text)     warn = pd.DataFrame()         warn['Packages'] = warn_packages    warn['warnings'] = warn_text     # warn.to_excel('warnings.xlsx', index = False)     writer = ExcelWriter('warnings.xlsx')         warn.to_excel(writer, 'report1', index = False)         workbook = writer.book    worksheet = writer.sheets['report1']     # Account info columns    worksheet.set_column('A:A', 15)    # State column    worksheet.set_column('B:B', 45)    # Post code    # worksheet.set_column('F:F', 10)     writer.save()     sugg_packages = []    sugg_text = []    for i in range(len(suggestions)):        sugg_packages.append(suggestions[i][0])     for i in range(len(suggestions)):        sugg_text.append(suggestions[i][1])         # print(sugg_packages, sugg_text)     sugg = pd.DataFrame()         sugg['Packages'] = sugg_packages    sugg['suggestions'] = sugg_text     writer1 = ExcelWriter('suggestions.xlsx')         sugg.to_excel(writer1, 'report2', index = False)         workbook = writer1.book    worksheet = writer1.sheets['report2']     # Account info columns    worksheet.set_column('A:A', 25)    # State column    worksheet.set_column('B:B', 120)    # Post code    # worksheet.set_column('F:F', 10)    writer1.save()     pack_data = pd.DataFrame()    pack_data['Packages'] = packages    writer1 = ExcelWriter('packages.xlsx')         pack_data.to_excel(writer1, 'report3', index = False)         workbook = writer1.book    worksheet = writer1.sheets['report3']     # Account info columns    worksheet.set_column('A:A', 75)    # State column    # Post code    # worksheet.set_column('F:F', 10)    writer1.save()     os.chdir('..')  if __name__ == '__main__':     warnings, suggestions, packages, shells = clean_data()     convert_to_excel()
Once you run the above scripts, you will find a folder called outputs in the current directory. navigate to the outputs folder where you will find excel sheets that contain warnings, suggestions, and installed packages. 
abhishek0719kadiyan
Linux-Unix
Python
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.
TCP Server-Client implementation in C
ZIP command in Linux with examples
SORT command in Linux/Unix with examples
tar command in Linux with examples
curl command in Linux with Examples
Read JSON file using Python
Adding new column to existing DataFrame in Pandas
Python map() function
How to get column names in Pandas dataframe