Before revision:  We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure ( 3-states: Q3= 76-84, 8-states: Q8=65-73), sub-cellular localization for 10 cellular compartments ( Q10= 74) and whether a protein is membrane-bound or water-soluble (  Q2= 89).
After revision:  We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure ( 3-states: Q3= 81\%-87\%); the second were per-protein predictions of protein sub-cellular localization for 10 cellular compartments ( Q10= 74) and whether a protein is membrane-bound or water-soluble ( Q2= 89).