moved sorting and added notnulls