Self-Supervision is All You Need for Solving Rubik's Cube

1 INTRODUCTION . Combinatorial search is a challenging task both in theory and in practice . Representative problems include Traveling Salesman Problem , in which a salesman tries to travel through multiple cities in the shortest path possible . Despite how easy it sounds , the problem is labeled NP-hard due to its combinatorial complexity ( Papadimitriou & Steiglitz , 1998 ) ; as the number of cities increases , the number of possible combinations skyrockets . In the real world , algorithms for such problems are applied in various ways to optimize logistics , supply chain , resource planning , etc . In the past , researchers have proposed several shortcut algorithms to tackle the combinatorial complexity of such problems . Recently , in particular , reinforcement learning has enabled automatic search of algorithms to solve combinatorial problems near-optimally ( Mazyavkina et al. , 2021 ) ; by rewarding efficient moves , DNNs can learn to find increasingly more optimal solutions . Rubik ’ s Cube is also one such problem ( Demaine et al. , 2018 ) , and there have been not only logical search methods ( Korf , 1997 ; Rokicki et al. , 2014 ) but also heuristic ( Korf , 1982 ; El-Sourani et al. , 2010 ; Arfaee et al. , 2011 ) and reinforcement learning approaches ( McAleer et al. , 2018 ; Agostinelli et al. , 2019 ; Corli et al. , 2021 ) that can solve any scrambled state of the puzzle . Nevertheless , combinatorial search is still no easy task ; you need detailed knowledge of the target problem , coding explicit logic , or configuring a reinforcement learning system . Reinforcement learning is especially notable for the need to adjust architecture , loss function , and hyperparameters several times to stabilize the learning process often involving non-differentiable operations . To overcome this difficulty , we developed a novel method to train DNNs in a self-supervised manner for solving combinatorial problems with a predefined goal . The method teaches a DNN probability distributions of optimal moves associated with problem states , with an assumption that , for a goal-predefined problem , random moves stemming from the goal state are probabilistically biased towards optimality . In our experiment , by training a DNN to solve Rubik ’ s Cube , we evaluate our method against an optimal solver and DeepCubeA ( Agostinelli et al. , 2019 ) . Contribution . We show that self-supervised learning can be all you need for solving some goalpredefined combinatorial problems near-optimally , without requiring specialized knowledge or reinforcement learning . 2 PROPOSED METHOD . In this section , we first provide an overview of the proposed method and then discuss its constraints . Below , we consider a combinatorial problem as a pathfinding task on an implicit graph . Each node in the graph represents a unique state and has edges as moves to its adjacent nodes with an equal probability . 2.1 OVERVIEW . The fundamental idea of the proposed method is as follows : 1 ) apply a sequence of random moves to the target problem , and 2 ) train a DNN to predict the last random move based on the problem state . By learning associations between problem states and probability distributions of the last moves , the DNN can infer step-by-step moves as a reverse path to the initial goal state . After training , the multinomial probability distribution of moves at inference step i should approximate P ( Xi = j : j ∈ M ) = ∑ k∈ M P ( Xi+1 = k ) · P ( Xi = j | Xi+1 = k ) ( 1 ) where Xi is a choice of move at step i , and j and k belong to the set of all available moves M at steps i and i+1 . Here , the choice at step i does not actually depend on the future step i+1 . Rather , it should approximate how similar states depended on past steps during training , which in turn can be viewed as future steps at inference . In this sense , the probability distribution of the move Xi implicitly depends on paths to the current state or similar that appeared during training . Figure 1 shows a miniature example . 2.2 CONDITION AND ASSUMPTIONS . There is one necessary condition for the proposed method to work , one assumption for a DNN to approximate towards an optimal solver , and another assumption for effective training and inference . First , to be solved with the proposed method , a problem must meet the following condition . Condition 1 . For the problem of interest , its goal state is predefined and readily available . For a DNN to learn probability distributions of moves connecting to the goal state , any path consisting of random moves must be derived from the goal state . Therefore , Condition 1 is necessary for the proposed method . Subsequently , the following two assumptions are desirable for a self-supervised DNN to approximate an optimal solver . Assumption 1 . The more optimal a move is , the more frequently it occurs . Although Assumption 1 is not necessary for applying the proposed method , the more true Assumption 1 is for the problem , the more optimal the DNN will be . Here , we formulate the statistical condition for Assumption 1 to be true on a problem when all moves at every node have an equal likelihood . Let pn , a be the probability of taking a particular move a as the first step to a specific target node in a path of n moves or more , pn , a = kmax∑ k=n Ck|n Ak ( 2 ) where k is the move count to the target state , kmax is the maximum number of k , A is the total number of possible moves at all nodes , and Ck|n is the number of k-move paths ( k ≥ n ) . Likewise , let b be an alternative to a that requires n + 1 moves or more , Assumption 1 is true if and only if pn , a ≥ pn+1 , b . Thus , provided pn , a = Cn|n/An + ∑kmax n+1 ( Ck|n/A k ) , the following must be satisfied for any n for Assumption 1 to be true : Cn|n An + kmax∑ k=n+1 Ck|n Ak ≥ kmax∑ k=n+1 Ck|n+1 Ak ( 3 ) At this point , both Ck|n and Ck|n+1 are unknown on an implicit graph , and the ∑ terms do not necessarily equal when the paths start with different moves . However , the only potential difference is the first move in two paths of different optimalities , and just taking a sub-optimal b move should not largely increase the total number of same-distance paths . Therefore , we assume that this holds true to varying degrees depending on the problem being addressed . Assumption 2 . The maximum number of moves required to solve any state—so-called God ’ s Number—or at least its upper bound should be known for a target problem . In the proposed method , let NG be God ’ s number of a given problem , the DNN receives problem states shuffled for 1 ∼ NG moves . If you estimate the number smaller than it actually is , then the DNN would not well adapt to problem states of higher complexities during inference . On the contrary , if you overestimate NG , you may unnecessarily apply additional moves to states of the highest complexity , resulting in more non-optimal moves in training scrambles . One way to address this limitation would be to keep increasing the maximum number of shuffles , as long as the DNN can make predictions beyond chance for the maximum complexity . If Condition 1 and Assumption 1 are met , random moves should have expected probability distributions peaking around optimal moves . It is also notable that the degree to which a problem satisfies Assumption 1 affects the optimality of the trained DNN : the more possible paths violating Assumption 1 , the less optimal the DNN will be . Although the example in Figure 1 has a very restricted setting for explanatory simplicity , this observation could apply to similar problems of higher dimensions and complexities under these constraints . Accordingly , we hypothesize DNNs to learn to solve goal-predefined combinatorial problems near-optimally . Last moveScramble 3 EXPERIMENT . 3.1 RUBIK ’ S CUBE . Rubik ’ s Cube is a cubic puzzle with 6 faces each having 9 color stickers . The goal of the puzzle is to move the stickers by rotating pieces on 6 faces , so that each face has stickers of only one color . Including the goal state , the puzzle can have approximately 4.3× 1019 different states . In our experiment , the puzzle is represented by its sticker locations and their colors . By assigning indices to all sticker locations and colors , one-hot encoding can represent the puzzle in 324 dimensions ( 54 sticker locations×6 colors ) . For rotating Rubik ’ s Cube , we employ the quarter-turn metric ( a 90° turn of a face counts as one move whereas a 180° turn counts as two ) , meaning 12 possible moves for every given state . In this configuration , solving Rubik ’ s Cube can be described as an implicit graph search problem in the 12-D action space . 3.2 LEARNING OBJECTIVE . Given the state of a Rubik ’ s Cube scrambled for random 1 ∼ 26 moves1 , a DNN predicts the last move of a scramble ( see Figure 2 ) . Here , we set the maximum scramble length to 26 because the number is known as sufficient for optimally solving any state of Rubik ’ s Cube ( Kunkle & Cooperman , 2007 ) . Since the inverse of the last scramble move is expectedly close to optimal under Assumption 1 , we hypothesize that the DNN can learn to solve the puzzle near-optimally through this learning objective . 3.3 MODEL . We employ the same architecture as DeepCube and DeepCubeA ( McAleer et al. , 2018 ; Agostinelli et al. , 2019 ) . The model first has two fully-connected ( FC ) layers , followed by four residual blocks ( He et al. , 2016 ) each containing two FC layers . Finally , for the prediction output , the model has a 12-D FC layer with softmax activation . The DNN is trained for 1 , 000 , 000 steps with a fixed batch size of 5 , 200 ( 200 states per scramble length × 26 scramble moves ) . We use Adam optimizer ( Kingma & Ba , 2014 ) with the initial learning rate of 10−4 for updating model weights . 1We exclude obviously redundant moves like R ( 90° clockwise turn on the right face ) following R ’ ( counterclockwise ) . 3.4 INFERENCE . For solving Rubik ’ s Cube with the trained model , we adopt a simple beam search . Starting from depth 1 with a scrambled state , at every depth i , the DNN predicts the next moves for every candidate state . Let k be the width of the beam search , we pass at most k candidate paths and corresponding states to the next depth i+ 1 , sorted by their expected probabilities . Also , for every potential move , if a move a is predicted to be worse than chance ( i.e. , p ( Xi = a ) < 1/12 ) , we exclude the path from the set of candidates before sorting . The search continues until any of the candidate states is solved , at which point the search depth matches the solution length .

Summary: The paper presents a self-supervised method for learning to solve Rubik's cube instances. The method relies on randomly perturbing a solved Rubik's cube and training a neural network to predict the last move that led to the perturbed state. The experiments show that a self-supervised Transformer combined with breadth-first-search can successfully solve randomly scrambled Rubik's cubes.