Before revision:  We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure ( 3-states: Q3= 76-84, 8-states: Q8=65-73), sub-cellular localization for 10 cellular compartments ( Q10= 74) and whether a protein is membrane-bound or water-soluble (  Q2= 89).
After revision:  We validated the advantage of using the embeddings as exclusive input for several subsequent tasks. The first was a per-residue prediction of protein secondary structure ( 3-states: Q3= 76-84, 8-states: Q8=65-73), sub-cellular localization for 10 cellular compartments ( Q10= 74) and whether a protein is membrane-bound or water-soluble ( Q2= 89).