Learning explanations that are hard to vary

1 INTRODUCTION Consider the top of Figure 1 , which shows a view from above of the loss surface obtained as we vary a two dimensional parameter vector θ “ pθ1 , θ2q , for a fictional dataset containing two observations xA and xB . Note the two global minima on the top-right and bottom-left . Depending on the initial values of θ — marked as white circles — gradient descent converges to one of the two minima . Judging solely by the value of the loss function , which is zero in both cases , the two minima look equally good . However , looking at the loss surfaces for xA and xB separately , as shown below , a crucial difference between those two minima appears : Starting from the same initial parameter configurations and following the gradient of the loss , ∇θLpθ , xiq , the probability of finding the same minimum on the top-right in either case is zero . In contrast , the minimum in the lower-left corner has a significant overlap across the two loss surfaces , so gradient descent can converge to it even if training on xA ( or xB ) only . Note that after averaging there is no way to tell what the two loss surfaces looked like : Are we destroying information that is potentially important ? In this paper , we argue that the answer is yes . In particular , we hypothesize that if the goal is to find invariant mechanisms in the data , these can be identified by finding explanations ( e.g . model parameters ) that are hard to vary across examples . A notion of invariance implies something that stays the same , as something else changes . We assume that data comes from different environments : An invariant mechanism is shared across all , generalizes out of distribution ( o.o.d . ) , but might be hard to model ; each environment also has spurious explanations that are easy to spot ( ‘ shortcuts ’ ) , but do not generalize o.o.d . From the point of view of causal modeling , such invariant mechanisms can be interpreted as conditional distributions of the targets given causal features of the inputs ; invariance of such conditionals is expected if they represent causal mechanisms , that is — stable properties of the physical world ( see e.g . Hoover ( 1990 ) ) . Generalizing o.o.d . means therefore that the predictor should perform equally well on data coming from different settings , as long as they share the causal mechanisms . We formalize a notion of consistency , which characterizes to what extent a minimum of the loss surface appears only when data from different environments are pooled . Minima with low consistency are ‘ patchwork ’ solutions , which ( we hypothesize ) sew together different strategies and should not be expected to generalize to new environments . An intuitive description of this principle was proposed by physicist David Deutsch : “ good explanations are hard to vary ” ( Deutsch , 2011 ) . Using the notion of consistency , we define Invariant Learning Consistency ( ILC ) , a measure of the expected consistency of the solution found by a learning algorithm on a given hypothesis class . The ILC can be improved by changing the hypothesis class or the learning algorithm , and in the last part of the paper we focus on the latter . We then analyse why current practices in deep learning provide little incentive for networks to learn invariances , and show that standard training is instead set up with the explicit objective of greedily maximizing speed of learning , i.e. , progress on the training loss . When learning “ as fast as possible ” is not the main objective , we show we can trade-off some “ learning speed ” for prioritizing learning the invariances . A practical instantiation of ILC leads to o.o.d . generalization on a challenging synthetic task where several established regularizers fail to generalize ; moreover , following the memorization task from Zhang et al . ( 2017 ) , ILC prevents convergence on CIFAR-10 with random labels , as no shared mechanism is present , and similarly when a portion of training labels is incorrect . Lastly , we set up a behavioural cloning task based on the game CoinRun ( Cobbe et al. , 2019b ) , and observe better generalization on new unseen levels . 5334 Problems , Combinat ions & Games 1.1 Mate in 1 295 80Z0Z0Z0Z 7Z0Z0ZpZ0 6pZ0Z0ZpZ 5jpZ0ZPZ0 40Z0ZnZPA 3OPZ0Z0Z0 20ZKZ0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 296 80Z0Z0Z0Z 7snZ0Z0o0 60j0Z0ZPZ 5o0ZPZ0Z0 4Po0Z0M0Z 3ZKZ0Z0Z0 20ZRZ0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 297 80Z0Z0Z0Z 7Z0Zns0Z0 6BZ0ako0Z 5Z0Z0o0Z0 40O0Z0ZPZ 3Z0Z0Z0Z0 20ZKZNZ0Z 1Z0Z0Z0ZR a b c d e f g h 298 80Z0Z0Z0Z 7ZRZ0Z0Z0 60Z0ako0Z 5Z0Z0onZ0 40Z0oKZPZ 3Z0ZPZ0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 299 8rZ0l0j0s 7opZ0a0op 60Z0o0m0Z 5Z0Z0o0M0 40Z0ZPZbZ 3OQM0A0Z0 20O0Z0OPO 1Z0ZRS0J0 a b c d e f g h 300 80ZrZ0j0s 7obl0opZp 60onZ0M0Z 5Z0m0Z0A0 40Z0o0Z0Z 3ZPZPZPO0 2PZ0MQZ0O 1ZKZ0S0ZR a b c d e f g h 5334 Problems , Combinat ions & Games 1.1 Mate in 1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 a b c d e f g h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZQZ0 20J0Z0Z0Z 1Z0Z0l0Z0 a b c d e f g h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 a b c d e f g h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 a b c d e f g h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR a b c d e f g h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0ZBZ0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZkZNZ0Z 7oRZRZ0Z0 6KZ0Z0Z0Z 5Z0Z0ZnZ0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 422 80Z0Z0Z0Z 7ZNZ0Z0Z0 6RZpZ0Z0Z 5ZkZ0Z0Z0 40MpZ0Z0Z 3Z0J0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 423 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0ZNZB 40Z0Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ0Z 1Z0Z0Z0J0 a b c d e f g h 424 80Z0Z0Z0Z 7Z0o0Z0S0 60ZRZNZ0j 5Z0Z0Z0Z0 40Z0Z0Z0o 3Z0Z0Z0ZK 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 425 80Z0Z0Z0Z 7Z0Z0Z0Zp 60Z0Z0Z0L 5Z0Z0Z0Z0 40Z0ZKZko 3Z0Z0Z0Z0 20Z0Z0ZPZ 1Z0Z0ZNZ0 a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0Z 5ZkZpZ0Z0 40Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 355 8 7 0 60 0 5 0 Q 4 Z mB 3 A j 2 1 K a b c d e f g h 356 8 Z0 0Z0Z 7 Z 0Z0 60Z Z Z0Z 5Z0Z0 0Z0 40Z0o0ZNZ 3Z0ZKZ0Z0 20Z0Z0ZpL 1Z0Z0ZkZ0 a b c d e f g h 357 80Z0Z0ZRZ 7Z0Z0ZKm0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0S0 20Z0Z0Z0Z 1Z0Z0Z0Ak a b c d e f g h 358 8 ZKZ0Z 7 0Z0Z0 6 p kZ0Z 5 O0Z0 4 OQZ0Z 3 Z0Z0 2 0Z0Z 1 Z0Z0 a b c d e f g h 359 80Z0Z0Z0Z 7jPO0Z0Z0 60SnZ0Z0Z 5Z0J0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 360 80Z0Z0ArZ 7Z0Z0Z0O0 60Z0Z0JBj 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZkZN 7oRZR 6KZ0Z 5Z0Z0 n 40Z0Z 0 3Z0Z0 Z 20Z0Z 1Z0Z0 a b c d e f g h 422 80Z0Z 7ZNZ0 6RZpZ 5ZkZ0 40MpZ 3Z0J0 20Z0Z 1Z0Z0 a b c d e f g h 423 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0ZNZB 40Z0Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ0Z 1Z0Z0Z0J0 a b c d e f g h 424 8 0Z 7 o0 S 6 RZN j 5 Z0 4 0Z o 3 Z0 K 2 0Z 1 Z0 a b c d e f g h 425 8 0Z 7 Z0 p 6 0Z Z L 5 0Z0 0 0 4 0ZK ko 3Z0Z0 Z 2 0Z0 P 1Z Z0 N a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0Z 5ZkZpZ0Z0 40Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h Rc7 Qc5 5334 Problems , Combinat ions & Games 1.1 Mate in 1 295 80Z0Z0Z0Z 7Z0Z0 p 0 6pZ0 0Z 5jpZ0 P 0 40Z0 nZPA 3OPZ Z0 20ZKZ Z 1Z0Z a b c d e f g h 296 80Z0Z0Z0Z 7snZ o0 60j0 PZ 5o0ZP Z 4Po0 M 3ZKZ 20ZR 1Z0Z Z a b c d e f g h 297 80Z0Z0Z0Z 7Z0Zns 0 6BZ0ako 5Z0Z o 40O0 P 3Z0Z 20ZK N 1Z0Z R a b c d e f g h 298 80Z0Z0Z0Z 7ZRZ 60Z0ako 5 0Z0 n 4 Z0oK P 3 0ZPZ 0 20Z0 Z 1 0Z0 a b c d e f g h 299 8rZ0l0j0s 7opZ a op 6 Z0o m 5 0Z o M 4 Z0 P b 3OQM A 2 O0 OPO 1 0ZRS0J a b c d e f g h 300 80ZrZ0j0s 7obl0op p 6 onZ M 5 0m0 0A 4 Z0o0Z 3 PZP PO 2PZ0MQ O 1ZKZ S R a b c d e f g h 5334 Problems , Combinat ions & Games 1.1 Mate in 1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 a b c d e f g h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZQZ0 20J0Z0Z0Z 1Z0Z0l0Z0 a b c d e f g h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 a b c d e f g h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 a b c d e f g h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR a b c d e f g h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0ZBZ0 a b c d e f g h 5334 Problems , Combinations & Games 2.1 White to Move # 2 355 8 0Z0Z0Z 7Z Z0Z0Z0 6 Z0Z0Z0Z 5 Z0ZQZ0 4 Z0Z0mBZ 3 0A0j0Z0 2 0Z0Z0Z 1 Z0ZKZ0 a b c d e f g h 356 8 0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0o0ZNZ 3Z0ZKZ0Z0 20Z0Z0ZpL 1Z0Z0ZkZ0 a b c d e f g h 357 80Z0Z0ZRZ 7Z0Z0ZKm0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0S0 20Z0Z0Z0Z 1Z0Z0Z0Ak a b c d e f g h 358 80Z0ZKZ0Z 7Z0Z0Z0Z0 60ZpZkZ0Z 5Z0Z0O0Z0 40Z0OQZ0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 359 80Z0Z0Z0Z 7jPO0Z0Z0 60SnZ0Z0Z 5Z0J0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 360 80Z0Z0ArZ 7Z0Z0Z0O0 60Z0Z0JBj 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0Z0Z0 20Z0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h Kg2 5334Problems , Combinations & Games 1.1Matein1 301 8rZbZkZ0s 7ZpopZ0Zp 6pZ0ZpL0Z 5ZNO0Z0Z0 40OKZ0Z0Z 3O0Z0OnZP 20Z0Z0O0Z 1Z0Z0Z0Z0 abcdefg h 302 80Z0ZkZ0Z 7Z0Z0OpZ0 60Z0O0Z0Z 5Z0Z00Z0 4Z0Z 3QZ0 2J0Z 1lZ0 abcdefg h 303 80Z0ZkZ0Z 7Z0Z0Z0Z0 60Z0Z0ONZ 5Z0ZQZ0Z0 40Z0Z0Z0Z 3Z0Z0ZpZ0 20Z0ZnZ0J 1Z0Z0ZqZ0 abcdefg h 304 80Z0Z0s0Z 7j0Z0Z0Z0 6No0Z0Z0Z 5Z0Z0Z0Zp 40O0ZbZpO 3Z0Z0OrZ0 2RZ0Z0OKA 1Z0Z0Z0Z0 abcdefg h 305 80Z0Z0Z0Z 7Z0Z0Z0Z0 60Z0Z0Z0Z 5Z0Z0Z0Z0 40Z0Z0Z0Z 3Z0Z0ZNZ0 20orZ0ZPO 1s0j0J0ZR abcdefg h 306 8ra0Z0Z0Z 7j0o0Z0ZR 6PZPZ0Z0Z 5OpJ0Z0Z0 40Z0Z0Z 3Z0Z0Z0 20Z0Z0Z 1Z0Z0ZB abcdefg h Bf8 5334 Problems , Combinations & Games 2.1 White to Move # 2 421 8rZk N 7oRZR 6KZ0 5 0Z n 4 Z0 3Z0Z0Z0Z0 20Z0Z0Z0Z 1 a b c d e f g h 422 8 7 N 0 6R p 5 k 4 Mp 3Z0J0Z0Z0 20Z0Z0Z0Z 1 a b c d e f g h 423 8 7 Z 6 5 N B 4 3Z0Z0o0ok 20Z0ZRZ0Z 1 J0 a b c d e f g h 424 8 0 Z 7 o Z0S0 6 ZRZN j 5 0 4 0 o 3Z0Z0Z0ZK 20Z0Z0Z0Z 1 0 a b c d e f g h 425 8 Z0Z0Z0Z 7Z0Z0Z0Zp 6 Z0Z0Z0L 5 0Z0Z0Z0 4 Z0ZKZko 3Z0Z0Z0Z0 20Z0Z0ZPZ 1 0Z0ZNZ0 a b c d e f g h 426 8 Z0Z0Z0Z 7 0ZRZ0Z0 6 o0Z0Z0Z 5 kZpZ0Z0 4 Z0O0Z0Z 3Z0J0Z0Z0 2QZ0Z0Z0Z 1Z0Z0Z0Z0 a b c d e f g h 5334 Problems , Com binations & Games 2.1 White to Move # 2 421 8r k N 0Z 7oR RZ Z0 6K 0 0Z 5 0 nZ0 4 Z 0Z 3 Z0 20 0Z 1 Z0 a b c d e f g h 422 8 0Z 7 NZ Z0 6RZpZ 0Z 5 kZ Z0 4 Mp 0 0Z 3 J Z0 20 0 0Z 1 0Z0 a b c d e f g h 423 80Z0Z0Z 7Z0Z0Z0 60Z0Z0Z 5Z0Z0ZN B 40Z0Z0Z 3Z0Z0o0ok 20Z0ZRZ 1Z0Z0Z0J a b c d e f g h 424 8 Z Z Z0Z 7 o Z0S0 6 ZRZN 0j 5 Z Z Z0 4 0 0o 3Z ZK 2 Z 1 0Z a b c d e f g h 425 80 Z Z0Z 7Z p 60 L 5Z 4 Z K ko 3 0 0Z Z0 2 Z0 ZPZ 1 Z NZ0 a b c d e f g h 426 80Z0Z0Z0Z 7Z0ZRZ0Z0 60o0Z0Z0 5ZkZpZ0Z 40Z0O0 3Z0J0Z 2QZ0Z 1Z0Z0Z0Z a b c d e f g h An example . Take these two second-hand books of chess puzzles . We can learn the two independent shortcuts ( blue arrows for the left book OR handwritten solutions on the right ) , or actually learn to play chess ( the invariant mechanism ) . While both strategies solve other problems from the same books ( i.i.d . ) , only the latter generalises to new chess puzzle books ( o.o.d. ) . How to distingui h the two ? We would not have learned about the red arrows had we trained on the book on the right , and vice versa with the hand-written notes .

Summary: The authors introduce and formalize the concept of Invariant Learning Consistency (ICL), which is motivated by the idea that "good explanations are hard to vary" in the context of deep learning. Instead of using the arithmetic mean to pool gradients (logical OR), the authors propose to use the element-wise geometric mean of gradients with a logical AND masking. Experimental results on both synthetic and real-world data sets are reported under the setup of supervised learning and reinforcement learning.