Package: slh-sh
tl;drslh-sh -slh.shSRL's Little Helper (SLH) is an unopinionated CLI tool in Python that improves workflow efficiency by automating repetitive chores for Systematic Literature Review projects.pipinstallslh-sh
slh-shinit
slh-sh--helpFeaturesExtractionImport CSV files of studies to SQLite database and export to the Google SHeets (exported from, e.g. Covidence, Zotero, etc).Extract PDF links from html files and download them.Rename PDF files with provided format.Extract text from PDF files (based on searched term, based on Highlights) including the citation at the end of the paragraph.Extract Citations with provided format (e.g APA) from PDF files.Extract Bibliography with provided format (e.g APA) from PDF files.Extract Keywords from PDF files.Extract distribution number of keywords in a study.Save Extracted data to local SQLite Database.SyncUpdate the data in Google Sheets on spesific Row, Column, Cell from database or complete sync to new Worksheet.Fetch and sync data from Google Sheets to new database table.ReviewOpen PDF files with default PDF viewer from CLI with its ID number -slh-sh pdf 436Highlight PDF files with defined "Theme" in Config file with specific color.Get infoCompile and format the data from database to a paper with citation.Query the database with SQL query or with provided arguments based on author, ID, theme and sub theme.ManageOne project folder for config files, database, and PDF files.Create new project folder with provided structure.Backup and restore project folder. (Under Development)RationaleThere are at least 17 steps to do a Systematic Literature Review (SLR) with a Google Sheets.slh-sh would automate all the repetitive tasks.The only manual steps (12, 16, 17) with slh-sh would be:Writing a minimal one line command, e.g.slh-sh pdf 120Reading the article and highlighting the texts based on provided Theme Color in Config File.Writing your paper.Manual SLR steps with Google Sheets without using slh-sh:Find the next study in the list on Covidence.Add new row to Google Sheet.Download the PDF file.Take the names and format the File with the naming format e.g. #100_NAME_NAME2[et_al]_YEAR.pdfPut the file name on Google Sheets.Format the authors to the naming style.Put the formatted author's name on Google Sheets.Find and copy the abstract to Google Sheets.Find and copy keywords to Google Sheets.Copy the year to Google Sheets.Find and copy APA and DOI to Google Sheets.Open, scan, and read the article; find and highlight by colors based on the themes.Find and transfer highlighted texts to Google Sheets based on the themes (colors/fields).Copy the PDF file to the Google Drive folder.Make the Filename column on Google Sheets link to Google Drive.Mark or color the Inclusion column as included, excluded, or unknown.Write your Own thought column.Take a breath.Repeat.PreperationsGoogle Project and Google Sheets APICreate a Google Project and enable Google Drive and Google Sheets API.create-projectCreate a Google Service Account and download the credentials file.create-credentialsCreate a Google Sheet and share it with the Service Account email.InstallationPython 3.8 or higher.Favorite PDF viewer. (e.gFoxit Reader)SQLite database viewer. (e.gsqlitebrowser)Install slh-sh with pip.pipinstallslh-shRunCreate a folder and copy default config file from below OR useslh-sh initto create a new project folder from default template or a questionaire.slh-sh [Command] --help- to see the help for a command.slh-sh list- to see the available commands.Folder structure-slr-project-2023-09-19(projectfolder)-config.yaml-slh.db-credentials.json-studies.csv-export.html-pdf_files(folder)-Study_1.pdf-Study_2.pdf-Study_3.pdf-...Example Config File - config.yaml## Config file for slh## project_name - Name of the project directory# gs_url - URL to Google Sheet# gd_url - URL to Google Drive# csv_export - Name of the default CSV file# html_export - Name of the default HTML file# google_credentials - Name of the default Google Credentials file# sqlite_db - Name of the default SQLite Database file# studies_pdf - Name of the default PDF folder# gs_header_row_number - Number of the header row in Google Sheet# gs_studies_sheet_name - Name of the Studies sheet in Google Sheet# gs_studies_id_column_name - Name of the ID column in Google Sheet# themes - Themes (colors) for the annotations# searches - Searches (keywords) for the studies# sources - Sources (where the study is found)## Themes, Searches, and Sources will be added to database with sync yaml command## yamllint disable rule:line-length---project_name:slr-thesis-2023-09-19gs_url:''gd_url:''csv_export:studies.csvhtml_export:export.htmlhtml_id_element:study-headerhtml_dl_class:'action-linkdownload'google_credentials:credentials.jsonsqlite_db:slh.dbpdf_path:''G:\Other computers\My MacBook Pro\Thesis\SLR_Lib'gs_header_row_number:'3'gs_studies_sheet_name:Studiesgs_studies_id_column_name:Covidencethemes:purple:hex:'#c785da'term:theme1pink:hex:'#fc5b89'term:theme2blue:hex:'#69aff1'term:theme3green:hex:'#7bc768'term:theme4yellow:hex:'#fbcd5a'term:theme5searches:search_1:term1search_2:term2sources:source_1:University Librarysource_2:Scopussource_3:Web of Sciencesource_4:source4Data Gatheringstudies.csvExported from Covidence (Export > REFERENCES > Options [Full text review] > Format [CSV] ), Zotero, etc.The Columns of the Studies table will be created based on this file.Setgs_studies_id_column_namein confit.yaml (e.g. Covidence Number), it will be used as the ID of the study in google sheets.export.html file containing PDF URLsUsing Chrome browser, open the Covidence page with the list of studies.Right click on the page and choose "Save as" and choose "Webpage, HTML Only".The HTML file will be saved in the project folder.Examples# Initialize a new project folderslh-shinit# Learn about slh-sh commandsslh-sh--help# See the help for a commandslh-sh[Command]--help# See the list of available commandsslh-shlist# Check state of project and the applicationslh-shcheck# Import studies from CSV file to databaseslh-shaddcsvstudies.csv# Download PDF filesslh-shextractdl# Generate filename based on the provided format, save in database, and rename PDF filesslh-shextractfilename--rename--db# Extract Keywords from PDF files to Studies table in the databaseslh-shextractkeywords--cov[CovidenceNumber]slh-shextractkeywords--all--db# Extract Citations from PDF files to Studies table in the databaseslh-shextractcit--db# Extract Bibliography from PDF files to Studies table in the  databaseslh-shextractbib--db# Extract Distribution of a specific research keyword to Distribution table in databaseslh-shextractdist[Term]--cov[CovidenceNumber]slh-shextractdist[Term]--all--db# Sync Distribution table from database to Google Sheets Worksheetslh-shextractdist[Term]--cov[CovidenceNumber]--wsdsheet# Sync the config file to databaseslh-shsyncconfig# Sync Studies table from database to Google Sheets Worksheetslh-shsyncupdate--allsheet--apply# Sync specific data from database to Google Sheets Worksheet's Cell or Rowslh-shsyncupdate--col"Keywords"--cov[CovidenceNumber]--apply# Sync specific Google Sheet to new database tableslh-shsyncfetch'Stage 1'Stage_1_Table# Get info about a study and its Citation and Bibliography from a database table by ID.slh-shinfo5--tableStage_1--idcolCovidence--copy# Query the database with SQL query or with provided arguments based on author, ID, theme and sub theme.slh-shqueryJohn,120,192Stage_1Derogation-c
slh-shquery"SELECT * FROM Stage_1 WHERE Author LIKE '%John%' AND ID IN (120,192) AND Theme = 'Derogation'"-s# Query ALL studies for a theme and subtheme and copy to clipboard.slh-shqueryALLStage_1Derogation-c# Open Google Drive folder in browserslh-shgd# Open Google Sheets in browserslh-shgs# Open SQLite database wit default SQLite viewerslh-shdb# Open PDF file with default PDF viewerslh-shpdf[IDe.g.CovidenceNumber]# Open DOI in browserslh-shdoi[IDe.g.CovidenceNumber]Example of SLR's workflow with slh-shTODOTips and TricksPause syncing on Google Drive while editing a PDF file.Google APP ScriptAdds Google Drive PDF links to the filename column of Google SheetsfunctionlinkToDriveFile(){varcolumnRange=SpreadsheetApp.getActiveSpreadsheet().getRange("G4:G203");varfiles=DriveApp.getFolderById('11pjqVjFXfsBVjG4CC0mzMSZ2zeLCXNrE').getFiles();console.log(fileName+".pdf")while(files.hasNext()){varfile=files.next();console.log(file.getName())for(vari=1;i<=columnRange.getNumRows();i++){varfileName=columnRange.getCell(i,1).getValue();if(file.getName()===fileName+".pdf"){console.log("added"+file.getName())console.log("added"+fileName)columnRange.getCell(i,1).setFormula(`=HYPERLINK("${file.getUrl()}", "${fileName}"`)}}}}Thanksslh-sh would not be possible without these great projects:PythonTyperPandasPymupdfGspreadSqlAlchemy+ Work of many others.LicenseThis project is licensed under the terms of the MIT license.