Before revision:  Through a set of controlled synthetic tasks, we demonstrate models can encode these properties considerably above chance-level even when distributed  as random noise, calling into question the interpretation of absolute claims on probing tasks.
After revision:  Through a set of controlled synthetic tasks, we demonstrate models can encode these properties considerably above chance-level even when distributed in the data as random noise, calling into question the interpretation of absolute claims on probing tasks.