Question: The paper I refer to is: https://doi.org/10.1038/s41467-018-08083-z

A couple of snippets from the results & methods section:

> "We **first adopt** a highly stringent pair-wise comparison method5, in which ***a gene with a much higher RPKM (reads per kilobase million) value in one domain than all the other 8 domains (log2FC > 1, false discovery rate (FDR) < 0.05) is considered as domain specific***."

> "Reads were mapped to the Arabidopsis Information Resource TAIR10 reference genome build with TopHat2 (version 2.0.9) and BOWTIE (version 2.1.0) allowing up to two mismatches68 after filtering the low-quality reads (PHRED quality score < 20). The gene locus expression levels were calculated based on mapping outputs after removing reads mapped to ribosomal RNAs and transfer RNAs using Cuffdiff2 (version 2.1.1)69, and ***expression levels were normalized to the RPKM unit using edgeR with significant expression cutoff value set to RPKM > 1. Differential expression was assessed with edgeR and the cutoff value was >2-fold change in expression with Benjamini–Hochberg adjusted FDR < 0.01**.* We used Cuffdiff2 to quantify the abundance of annotated isoforms. For the identification of domain-specifically expressed genes, three methods were performed independently. ***Pair-wise comparison was carried out using the differential expression assessed with edgeR***."

I will address my queries as parts:

1) As far as I understand, edgeR uses TMM to normalise across libraries and fits the normalised raw counts to a NB distribution (I hope I got that part right) before testing for significance across samples and/or treatment. From the methods section in the above paper, all I gather is, the authors have somehow integrated the use of RPKM along with the edgeR package. Could someone clarify this for me?

2) And as far as I have been able to tell from the plethora of posts here and elsewhere, there doesn't seem to be any situation where FPKM/RPKM are to be used for sound qualitative/quantitative analysis of RNA seq data. Am I wrong to say this? Or are there situations where converting to FPKM units would give us some that edgeR and deseq2 don't?

3) Is this a suitable extension of what the authors of edgeR originally intended the package to be used for? I ask because I am in the middle of some analysis myself and this paper serves as one reference point for analysis.

4) Is it proper to use FPKM **after** normalising to library sizes (as done in edgeR)? I refer to What the FPKM? A review of RNA-Seq expression units (https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/)

>The first thing one should remember is that without between sample normalization (a topic for a later post), NONE of these units are comparable across experiments.

.... 4) does this imply that post between-sample normalisation, one could possibly utilise FPKM values for analysis? This seems to me what the authors might have done in the paper.

Answer: As others have said, the description is not specific enough to know for sure what they did, however there is an rpkm function in the edgeR package which returns a count table with the TMM-normalized counts further normalized by gene length, so I would guess that's what they did.

In response to your question #2, it makes sense to me why they used RPKMs here. They wanted to set an expression cut-off to filter genes with expression too low to be biologically significant. Without normalizing by gene lengths, the cut-off would be biased towards keeping longer genes that have similar transcript abundances as shorter genes which would be filtered.