Package: directlfq
directLFQdirectLFQ is an open-source Python package for quantifying protein intensities based on peptide intensities or fragment-ion intensities measured with from Mass Spectrometry-based proteomics. It preserves peptide ratios, shows very accurate quantification and has a robust normalization approach. Furthermore, it allows fast processing also of very large sample cohorts, as runtime increases linearly with sample number. It is part of the AlphaPept ecosystem from theMann Labs at the Max Planck Institute of Biochemistryand theUniversity of Copenhagen.You can process DIA and DDA data analyzed byAlphaPept, MaxQuant, FragPipe, Spectronaut and DIANN as well asgeneric formats, using a Graphical User Interface (GUI) or the python package.directLFQAboutInstallationOne-click GUIPipDeveloperRunning directLFQGUICLIPython and Jupyter notebooksTroubleshootingCitationsHow to contributeLicensedirectLFQ commandsdirectLFQ outputpreparing input filesSpectronautDIA-NNMaxQuantFragPipegeneric input formatreproducing data analyses from the paperAboutGenerating protein intensities from Mass Spectrometry proteomics data comes with a variety of challenges. Differing peptides that belong to the same protein can have strongly differing intensities, for example due to differing ionization efficiencies. Missing values (i.e. peptides that have been detected in one run but not in the other) make simple summarization of peptide intensities to protein intensities problematic. Differences in sample loading can introduce systematic biases into the analysis.
With directLFQ, we provide a novel algorithm for addressing these challenges in an efficient and accurate manner. directLFQ retains peptide ratios and uses them to infer protein ratios and uses the concept ofintensity tracesfor it's main processing steps. For futher details on the algorithm, please refer to thepreprint.InstallationdirectLFQ can be installed and used on all major operating systems (Windows, macOS and Linux).
There are currently two different types of installation possible:One-click GUI installer:Choose this installation if you only want the GUI and/or keep things as simple as possible.Developer installer:Choose this installation if you are familiar with CLI tools,condaand Python. This installation allows access to all available features of directlfq and even allows to modify its source code directly. Generally, the developer version of directlfq outperforms the precompiled versions which makes this the installation of choice for high-throughput experiments.One-click GUIThe GUI of directlfq is a completely stand-alone tool that requires no knowledge of Python or CLI tools. Click on one of the links below to download the latest release for:WindowsmacOSLinuxOlder releases remain available on therelease page, but no backwards compatibility is guaranteed.PipdirectLFQ can be installed in an existing Python 3.8 environment with a singlebashcommand.pipinstalldirectlfqThis installs the core directLFQ without graphical user interface (GUI). If you want to install with additional dependencies for GUI support, you can do this with:pipinstall"directlfq[gui]"For installation with stable dependencies, use:pipinstall"directlfq[stable]"NOTE: You might need to runpip install pip==21.0before installing directlfq like this. Also note the double quotes".For those who are really adventurous, it is also possible to directly install any branch (e.g.@development) with any extras (e.g.#egg=directlfq[stable,development-stable]) from GitHub with e.g.pipinstall"git+https://github.com/MannLabs/directlfq.git@development#egg=directlfq[stable,development-stable]"Developerdirectlfq can also be installed in editable (i.e. developer) mode with a fewbashcommands. This allows to fully customize the software and even modify the source code to your specific needs. When an editable Python package is installed, its source code is stored in a transparent location of your choice. While optional, it is advised to first (create and) navigate to e.g. a general software folder:mkdir~/folder/where/to/install/softwarecd~/folder/where/to/install/softwareThe following commands assume you do not perform any additionalcdcommands anymore.Next, download the directlfq repository from GitHub either directly or with agitcommand. This creates a new directlfq subfolder in your current directory.gitclonehttps://github.com/MannLabs/directlfq.gitFor any Python package, it is highly recommended to use a separateconda virtual environment, as otherwisedependancy conflicts can occur with already existing packages.condacreate--namedirectlfqpython=3.8-y
condaactivatedirectlfqFinally, directlfq and all itsdependenciesneed to be installed. To take advantage of all features and allow development (with the-eflag), this is best done by also installing thedevelopment dependenciesinstead of only thecore dependencies:pipinstall-e"./directlfq[development,gui]"By default this installs loose dependencies (no explicit versioning), although it is also possible to use stable dependencies (e.g.pip install -e "./directlfq[stable,development-stable]").By using the editable flag-e, all modifications to thedirectlfq source code folderare directly reflected when running directlfq. Note that the directlfq folder cannot be moved and/or renamed if an editable version is installed. In case of confusion, you can always retrieve the location of any Python module with e.g. the commandimport modulefollowed bymodule.__file__.Running directLFQThere are three ways to use directlfq:GUICLIPythonNOTE: The first time you use a fresh installation of directlfq, it is often quite slow because some functions might still need compilation on your local operating system and architecture. Subsequent use should be a lot faster.GUIIf you have installed directlfq with the one-click GUI installer, you can run the GUI simply by clicking the directLFQ icon on your desktop/applications folder.If the GUI was not installed through a one-click GUI installer, it can be activate with the followingbashcommand:directlfqguiNote that this needs to be prepended with a!when you want to run this from within a Jupyter notebook. When the command is run directly from the command-line, make sure you use the right environment (activate it with e.g.conda activate directlfqor set an alias to the binary executable (can be obtained withwhere directlfqorwhich directlfq)).CLIThe CLI can be run with the following command (after activating thecondaenvironment withconda activate directlfqor if an alias was set to the directlfq executable):directlfq-hIt is possible to get help about each function and their (required) parameters by using the-hflag.Python and Jupyter notebooksdirectLFQ can be imported as a Python package into any Python script or notebook with the commandimport directlfq.
Running the standard analysis (with plots) can be done via the command:importdirectlfq.lfq_manageraslfq_managerexample_input_file_diann="/path/to/example_input_file_diann.tsv"lfq_manager.run_lfq(example_input_file_diann)Several use cases for applying directLFQ can be found as Jupyter Notebooks in the tests folder. See for example thequicktests notebook.Note that the nbdev_nbs folder contains the source code as Jupyter notebooks. These notebooks are automatically converted to Python scripts using the nbdev package and stored in the directlfq folder. The notebooks contain additional documentation and comments as well as unit tests that can be executed directly from the notebooks themselves.TroubleshootingIn case of issues, check out the following:Issues: Try a few different search terms to find out if a similar problem has been encountered beforeDiscussions: Check if your problem or feature requests has been discussed before.CitationsIn the case that directLFQ is useful to you, please consider supporting us by citing thepaperAmmar, C., Schessner, J.P., Willems, S., Michaelis, A.C., and Mann, M. (2023). Accurate label-free quantification by directLFQ to compare unlimited numbers of proteomes.  Molecular & Cellular Proteomics, 100581.How to contributeIf you like this software, you can give us astarto boost our visibility! All direct contributions are also welcome. Feel free to post a newissueor clone the repository and create apull requestwith a new branch. For an even more interactive participation, check out thediscussionsand thethe Contributors License Agreement.LicensedirectLFQ was developed by theMann Labs at the Max Planck Institute of Biochemistryand theUniversity of Copenhagenand is freely available with anApache License. External Python packages (available in therequirementsfolder) have their own licenses, which can be consulted on their respective websites.directLFQ commandsdirectLFQ is started internally via the directlfq.lfq_manager.run_lfq() command. In principle and for most use cases, you only need to provide the path to the AlphaPept/MaxQuant/DIA-NN etc. file of interest. However, there are several other options that can be used to customize the analysis. The following commands are available:input_file: The input file containing the ion intensities. Usually the output of a search engine.columns_to_add: Add the names of columns that are present in the output table and that you want to keep in the directLFQ output file. Separated by semicolons. Note that some basic additional columns such as gene names are always added to the output table by default. WARNING: Take care that columns you add are not ambigous. For example, adding the peptide sequence column will not work, because there are multiple peptide sequences per protein.selected_proteins_file: If you want to perform normalization only on a subset of proteins, you can pass a .txt file containing the protein IDs, separeted by line breaks. No header expected.mq_protein_groups_txt: In the case of using MaxQuant data, the proteinGroups.txt table is needed in order to map IDs analogous to MaxQuant. Adding this table improves protein mapping, but is not necessary.min_nonan: Min number of ion intensities necessary in order to derive a protein intensity. Increasing the number results in more reliable protein quantification at the cost of losing IDs.input_type_to_use: The type of input file to use. This is used to determine the column names of the input file. Only change this if you want to use non-default settings.maximum_number_of_quadratic_ions_to_use_per_protein: How many ions are used to create the anchor intensity trace (see paper). Increasing might marginally increase performance at the cost of runtime.number_of_quadratic_samples: How many samples are used to create the anchor intensity trace (see paper). Increasing might marginally increase performance at the cost of runtimenum_cores: The number of cores to use (default is to use multiprocessing).filename_suffix: Suffix to append to the output files.deactivate_normalization: Set to true, if no between-sample normalization should be performed before processing.filter_dict: In case you want to define specific filters in addition to the standard filters, you can add a yaml file where the filters are defined (see examplehere). In the Python API you can also directly put in the dictionary instead of the .yaml file.directLFQ outputdirectLFQ writes three output files into the directory of the input file:The main output file ends with .protein_intensities.tsv and contains the estimated LFQ protein intensities.The second output file ends with .ion_intensities.tsv and contains the aligned intensity traces of all ions. This allows to compare profiles of different ions to each other. In particular, if you run directLFQ with peptide-level quantification, you can use this file to compare the intensity traces of different peptides of the same protein.The third output file ends with .aq_reformat.tsv and contains the reformatted input data in matrix format (ions are rows, samples are columns). The values are identical to the values of the original input file, just the format is different.preparing input filesSpectronautdirectLFQ takes a Spectronaut .tsv table as input. When exporting from Spectronaut, the correct columns need to be selected. These can be obtained by downloading one of the export schemes available below. We provide one export scheme for sprecursor quantification and one export scheme for fragment ion quantification. Fragment ion quantification shows slightly more accuracy, but the files are around 10 times larger.An export scheme can then simply be loaded into Spectronaut as follows:Go to the "Report" perspective in Spectronaut, click "Import Schema" and provide the file.The data needs to be exported in the normal long format as .tsv fileDownload Spectronaut export scheme for precursor quantificationDownload Spectronaut export scheme for fragment ion quantificationDIA-NNProvide the path to the DIANN "report.tsv" output table.MaxQuantProvide the path to the MaxQuant "peptides.txt" output table or the MaxQuant evidence.txt output table. Additionally and if possible, provide the path to the corresponding "proteinGroups.txt" file.FragPipeProvide the path to the "combined_ion.tsv" output table.generic input formatIn the case that you working with a search engine that is not supported by directLFQ, you can use the generic input format. This format is a tab-separated quantity matrix file with the following columns: "protein", "ion", "run_id1", "run_id2", ..,"run_idN".
Each row contains therefore all the ion intensities that were measured for an ion in each run (see examples below). The ion identifier only needs to be unique for each ion and can be on the level you want (peptide, charged peptide, or fragment ion). After reformatting your file into this format, save the file with the ending ".aq_reformat.tsv". Then you can simply give this file as input to directLFQ and it will automatically detect the generic input format.reproducing data analyses from the paperIf you want to reproduce data analyses presented in themanuscript, you can first download the data by executingpythontests/download_testfiles.pyall_testsThis will download the underlying datasets into the appropriate location. The notebooks carrying out the analyses themselves are located in the tests directory in the respective subfolders ratio_tests, normalization_tests, runtime_tests and organellar_maps.