fully functional version of validity metrics, etc. this commit is fully operational