supervised learning might be picking up on these spurious correlations or artifacts and you know assume when you evaluate it on a new card set which is the set of answers that the model that only looks in the sentence evidence can get right drops quickly from like 16 percentage points from 80 to 88 to 72 percent and this shows up across the board there's now probably a dozen papers in this space if not more that show that kind of these analyzed systems that you know nominally we're supposed to have human level accuracy actually