add code to ignore short entries (informal data)