Implement fully-sparse inference