Learning explanations that are hard to vary

1 INTRODUCTION Consider the top of Figure 1 , which shows a view from above of the loss surface obtained as we vary a two dimensional parameter vector θ “ pθ1 , θ2q , for a fictional dataset containing two observations xA and xB . Note the two global minima on the top-right and bottom-left . Depending on the initial values of θ — marked as white circles — gradient descent converges to one of the two minima . Judging solely by the value of the loss function , which is zero in both cases , the two minima look equally good . However , looking at the loss surfaces for xA and xB separately , as shown below , a crucial difference between those two minima appears : Starting from the same initial parameter configurations and following the gradient of the loss , ∇θLpθ , xiq , the probability of finding the same minimum on the top-right in either case is zero . In contrast , the minimum in the lower-left corner has a significant overlap across the two loss surfaces , so gradient descent can converge to it even if training on xA ( or xB ) only . Note that after averaging there is no way to tell what the two loss surfaces looked like : Are we destroying information that is potentially important ? In this paper , we argue that the answer is yes . In particular , we hypothesize that if the goal is to find invariant mechanisms in the data , these can be identified by finding explanations ( e.g . model parameters ) that are hard to vary across examples . A notion of invariance implies something that stays the same , as something else changes . We assume that data comes from different environments : An invariant mechanism is shared across all , generalizes out of distribution ( o.o.d . ) , but might be hard to model ; each environment also has spurious explanations that are easy to spot ( ‘ shortcuts ’ ) , but do not generalize o.o.d . From the point of view of causal modeling , such invariant mechanisms can be interpreted as conditional distributions of the targets given causal features of the inputs ; invariance of such conditionals is expected if they represent causal mechanisms , that is — stable properties of the physical world ( see e.g . Hoover ( 1990 ) ) . Generalizing o.o.d . means therefore that the predictor should perform equally well on data coming from different settings , as long as they share the causal mechanisms . We formalize a notion of consistency , which characterizes to what extent a minimum of the loss surface appears only when data from different environments are pooled . Minima with low consistency are ‘ patchwork ’ solutions , which ( we hypothesize ) sew together different strategies and should not be expected to generalize to new environments . An intuitive description of this principle was proposed by physicist David Deutsch : “ good explanations are hard to vary ” ( Deutsch , 2011 ) . Using the notion of consistency , we define Invariant Learning Consistency ( ILC ) , a measure of the expected consistency of the solution found by a learning algorithm on a given hypothesis class . The ILC can be improved by changing the hypothesis class or the learning algorithm , and in the last part of the paper we focus on the latter . We then analyse why current practices in deep learning provide little incentive for networks to learn invariances , and show that standard training is instead set up with the explicit objective of greedily maximizing speed of learning , i.e. , progress on the training loss . When learning “ as fast as possible ” is not the main objective , we show we can trade-off some “ learning speed ” for prioritizing learning the invariances . A practical instantiation of ILC leads to o.o.d . generalization on a challenging synthetic task where several established regularizers fail to generalize ; moreover , following the memorization task from Zhang et al . ( 2017 ) , ILC prevents convergence on CIFAR-10 with random labels , as no shared mechanism is present , and similarly when a portion of training labels is incorrect . Lastly , we set up a behavioural cloning task based on the game CoinRun ( Cobbe et al. , 2019b ) , and observe better generalization on new unseen levels . 5334 Problems , Combinat ions & Games 1.1 Mate in 1 295 80Z0Z0Z0Z 7Z0Z0ZpZ0 6pZ0Z0ZpZ 5jpZ0ZPZ0 40Z0ZnZPA 3OPZ0Z0Z0 20ZKZ0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 296 80Z0Z0Z0Z 7snZ0Z0o0 60j0Z0ZPZ 5o0ZPZ0Z0 4Po0Z0M0Z 3ZKZ0Z0Z0 20ZRZ0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 297 80Z0Z0Z0Z 7Z0Zns0Z0 6BZ0ako0Z 5Z0Z0o0Z0 40O0Z0ZPZ 3Z0Z0Z0Z0 20ZKZNZ0Z 1Z0Z0Z0ZR a b c d e f g h 298 80Z0Z0Z0Z 7ZRZ0Z0Z0 60Z0ako0Z 5Z0Z0onZ0 40Z0oKZPZ 3Z0ZPZ0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 299 8rZ0l0j0s 7opZ0a0op 60Z0o0m0Z 5Z0Z0o0M0 40Z0ZPZbZ 3OQM0A0Z0 20O0Z0OPO 1Z0ZRS0J0 a b c d e f g h 300 80ZrZ0j0s 7obl0opZp 60onZ0M0Z 5Z0m0Z0A0 40Z0o0Z0Z 3ZPZPZPO0 2PZ0MQZ0O 1ZKZ0S0ZR a b c d e f g h 5334 Problems , Combinat ions & Games 1.1 Mate in 1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 a b c d e f g h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZQZ0 20J0Z0Z0Z 1Z0Z0l0Z0 a b c d e f g h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 a b c d e f g h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 a b c d e f g h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR a b c d e f g h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0ZBZ0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZkZNZ0Z 7oRZRZ0Z0 6KZ0Z0Z0Z 5Z0Z0ZnZ0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 422 80Z0Z0Z0Z 7ZNZ0Z0Z0 6RZpZ0Z0Z 5ZkZ0Z0Z0 40MpZ0Z0Z 3Z0J0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 423 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0ZNZB 40Z0Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ0Z 1Z0Z0Z0J0 a b c d e f g h 424 80Z0Z0Z0Z 7Z0o0Z0S0 60ZRZNZ0j 5Z0Z0Z0Z0 40Z0Z0Z0o 3Z0Z0Z0ZK 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 425 80Z0Z0Z0Z 7Z0Z0Z0Zp 60Z0Z0Z0L 5Z0Z0Z0Z0 40Z0ZKZko 3Z0Z0Z0Z0 20Z0Z0ZPZ 1Z0Z0ZNZ0 a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0Z 5ZkZpZ0Z0 40Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 355 8 7 0 60 0 5 0 Q 4 Z mB 3 A j 2 1 K a b c d e f g h 356 8 Z0 0Z0Z 7 Z 0Z0 60Z Z Z0Z 5Z0Z0 0Z0 40Z0o0ZNZ 3Z0ZKZ0Z0 20Z0Z0ZpL 1Z0Z0ZkZ0 a b c d e f g h 357 80Z0Z0ZRZ 7Z0Z0ZKm0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0S0 20Z0Z0Z0Z 1Z0Z0Z0Ak a b c d e f g h 358 8 ZKZ0Z 7 0Z0Z0 6 p kZ0Z 5 O0Z0 4 OQZ0Z 3 Z0Z0 2 0Z0Z 1 Z0Z0 a b c d e f g h 359 80Z0Z0Z0Z 7jPO0Z0Z0 60SnZ0Z0Z 5Z0J0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 360 80Z0Z0ArZ 7Z0Z0Z0O0 60Z0Z0JBj 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZkZN 7oRZR 6KZ0Z 5Z0Z0 n 40Z0Z 0 3Z0Z0 Z 20Z0Z 1Z0Z0 a b c d e f g h 422 80Z0Z 7ZNZ0 6RZpZ 5ZkZ0 40MpZ 3Z0J0 20Z0Z 1Z0Z0 a b c d e f g h 423 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0ZNZB 40Z0Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ0Z 1Z0Z0Z0J0 a b c d e f g h 424 8 0Z 7 o0 S 6 RZN j 5 Z0 4 0Z o 3 Z0 K 2 0Z 1 Z0 a b c d e f g h 425 8 0Z 7 Z0 p 6 0Z Z L 5 0Z0 0 0 4 0ZK ko 3Z0Z0 Z 2 0Z0 P 1Z Z0 N a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0Z 5ZkZpZ0Z0 40Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h Rc7 Qc5 5334 Problems , Combinat ions & Games 1.1 Mate in 1 295 80Z0Z0Z0Z 7Z0Z0 p 0 6pZ0 0Z 5jpZ0 P 0 40Z0 nZPA 3OPZ Z0 20ZKZ Z 1Z0Z a b c d e f g h 296 80Z0Z0Z0Z 7snZ o0 60j0 PZ 5o0ZP Z 4Po0 M 3ZKZ 20ZR 1Z0Z Z a b c d e f g h 297 80Z0Z0Z0Z 7Z0Zns 0 6BZ0ako 5Z0Z o 40O0 P 3Z0Z 20ZK N 1Z0Z R a b c d e f g h 298 80Z0Z0Z0Z 7ZRZ 60Z0ako 5 0Z0 n 4 Z0oK P 3 0ZPZ 0 20Z0 Z 1 0Z0 a b c d e f g h 299 8rZ0l0j0s 7opZ a op 6 Z0o m 5 0Z o M 4 Z0 P b 3OQM A 2 O0 OPO 1 0ZRS0J a b c d e f g h 300 80ZrZ0j0s 7obl0op p 6 onZ M 5 0m0 0A 4 Z0o0Z 3 PZP PO 2PZ0MQ O 1ZKZ S R a b c d e f g h 5334 Problems , Combinat ions & Games 1.1 Mate in 1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 a b c d e f g h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZQZ0 20J0Z0Z0Z 1Z0Z0l0Z0 a b c d e f g h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 a b c d e f g h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 a b c d e f g h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR a b c d e f g h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0ZBZ0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 355 8 0Z0Z0Z 7Z Z0Z0Z0 6 Z0Z0Z0Z 5 Z0ZQZ0 4 Z0Z0mBZ 3 0A0j0Z0 2 0Z0Z0Z 1 Z0ZKZ0 a b c d e f g h 356 8 0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0o0ZNZ 3Z0ZKZ0Z0 20Z0Z0ZpL 1Z0Z0ZkZ0 a b c d e f g h 357 80Z0Z0ZRZ 7Z0Z0ZKm0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0S0 20Z0Z0Z0Z 1Z0Z0Z0Ak a b c d e f g h 358 80Z0ZKZ0Z 7Z0Z0Z0Z0 60ZpZkZ0Z 5Z0Z0O0Z0 40Z0OQZ0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 359 80Z0Z0Z0Z 7jPO0Z0Z0 60SnZ0Z0Z 5Z0J0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 360 80Z0Z0ArZ 7Z0Z0Z0O0 60Z0Z0JBj 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h Kg2 5334Problems , Combinations & Games 1.1Matein1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 abcdefg h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z00Z0 4Z0Z 3QZ0 2J0Z 1lZ0 abcdefg h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 abcdefg h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 abcdefg h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR abcdefg h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z 3Z0Z0Z0 20Z0Z0Z 1Z0Z0ZB abcdefg h Bf8 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZk N 7oRZR 6KZ0 5 0Z n 4 Z0 3Z0Z0Z0Z0 20Z0Z0Z0Z 1 a b c d e f g h 422 8 7 N 0 6R p 5 k 4 Mp 3Z0J0Z0Z0 20Z0Z0Z0Z 1 a b c d e f g h 423 8 7 Z 6 5 N B 4 3Z0Z0o0ok 20Z0ZRZ0Z 1 J0 a b c d e f g h 424 8 0 Z 7 o Z0S0 6 ZRZN j 5 0 4 0 o 3Z0Z0Z0ZK 20Z0Z0Z0Z 1 0 a b c d e f g h 425 8 Z0Z0Z0Z 7Z0Z0Z0Zp 6 Z0Z0Z0L 5 0Z0Z0Z0 4 Z0ZKZko 3Z0Z0Z0Z0 20Z0Z0ZPZ 1 0Z0ZNZ0 a b c d e f g h 426 8 Z0Z0Z0Z 7 0ZRZ0Z0 6 o0Z0Z0Z 5 kZpZ0Z0 4 Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Com binations & Games 2.1 White to Move # 2 421 8r k N 0Z 7oR RZ Z0 6K 0 0Z 5 0 nZ0 4 Z 0Z 3 Z0 20 0Z 1 Z0 a b c d e f g h 422 8 0Z 7 NZ Z0 6RZpZ 0Z 5 kZ Z0 4 Mp 0 0Z 3 J Z0 20 0 0Z 1 0Z0 a b c d e f g h 423 80Z0Z0Z 7Z0Z0Z0 60Z0Z0Z 5Z0Z0ZN B 40Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ 1Z0Z0Z0J a b c d e f g h 424 8 Z Z Z0Z 7 o Z0S0 6 ZRZN 0j 5 Z Z Z0 4 0 0o 3Z ZK 2 Z 1 0Z a b c d e f g h 425 80 Z Z0Z 7Z p 60 L 5Z 4 Z K ko 3 0 0Z Z0 2 Z0 ZPZ 1 Z NZ0 a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0 5ZkZpZ0Z 40Z0O0 3Z0J0Z 2QZ0Z 1Z0Z0Z0Z a b c d e f g h An example . Take these two second-hand books of chess puzzles . We can learn the two independent shortcuts ( blue arrows for the left book OR handwritten solutions on the right ) , or actually learn to play chess ( the invariant mechanism ) . While both strategies solve other problems from the same books ( i.i.d . ) , only the latter generalises to new chess puzzle books ( o.o.d. ) . How to distingui h the two ? We would not have learned about the red arrows had we trained on the book on the right , and vice versa with the hand-written notes .

Summary: this work posits that invariant mechanisms exist in a dataset. a machine learning algorithm that is trained using gradient descent usually averages gradients across examples. the thesis is that by averaging gradients, information is lost. the method posits that in a gradient descent algorithm, instead of an arithmetic average, a geometric (or karcher) mean can be used to preserve information about invariant mechanisms - while ignoring confounders. there are difficulties in a straightforward application of the geometric mean, so a simple heuristic algorithm is developed, involving masking gradients depending on whether the sign of the gradient agrees across a batch of examples (or, whether some agreement threshold is reached). this algorithm is tested on a synthetic dataset, a semi-synthetic task on CIFAR-10, and coinbase, an RL algorithm.