A method for controlling a robot comprising: Receiving an indication of a target configuration to be reached from an initial configuration of the robot; Determining a coarse-scale value map, which assigns values to each state of a coarse representation of an environment in which the robot is located, by a value iteration planner by iteratively refining estimates of values of the states of the coarse representation, wherein transition probabilities are determined using a transition probability model mapping coarse-scale states and coarse-scale actions to transition probabilities for coarse-scale states; and For each coarse-scale state of a sequence of coarse-scale states of the robot in the coarse representation of the environment, wherein the sequence starts from an initial coarse-scale state determined from the initial configuration of the robot and ends when the robot reaches the target configuration or a maximum number of fine-scale states of a fine representation of the environment has been reached, Determining a fine-scale sub-goal from the coarse-scale value map by selecting a coarse-scale action based on the value map, wherein the coarse-scale action is greedily selected by a fine-scale control policy; Performing, by an actuator of the robot, fine-scale control actions according to the fine-scale control policy trained via reinforcement learning to reach the determined fine-scale sub-goal and obtaining sensor data to determine fine-scale states reached by the robot in the fine representation of the environment as a result of performing the fine-scale control actions for each fine-scale state of a resulting sequence of fine-scale states of the robot in the fine representation of the environment, wherein the sequence starts from a current fine-scale state of the robot in the fine representation of the environment which the robot has within the coarse-scale state and ends with a last fine-scale state when the robot reaches the determined fine-scale sub-goal, the robot transitions to a different coarse-scale state, or a maximum sequence length of the sequence of fine-scale states has been reached; Determining the next coarse-scale state of the sequence of coarse-scale states from the last fine-scale state of the sequence of fine-scale states. Receiving an indication of a target configuration to be reached from an initial configuration of the robot; Determining a coarse-scale value map, which assigns values to each state of a coarse representation of an environment in which the robot is located, by a value iteration planner by iteratively refining estimates of values of the states of the coarse representation, wherein transition probabilities are determined using a transition probability model mapping coarse-scale states and coarse-scale actions to transition probabilities for coarse-scale states; and For each coarse-scale state of a sequence of coarse-scale states of the robot in the coarse representation of the environment, wherein the sequence starts from an initial coarse-scale state determined from the initial configuration of the robot and ends when the robot reaches the target configuration or a maximum number of fine-scale states of a fine representation of the environment has been reached, Determining a fine-scale sub-goal from the coarse-scale value map by selecting a coarse-scale action based on the value map, wherein the coarse-scale action is greedily selected by a fine-scale control policy; Performing, by an actuator of the robot, fine-scale control actions according to the fine-scale control policy trained via reinforcement learning to reach the determined fine-scale sub-goal and obtaining sensor data to determine fine-scale states reached by the robot in the fine representation of the environment as a result of performing the fine-scale control actions for each fine-scale state of a resulting sequence of fine-scale states of the robot in the fine representation of the environment, wherein the sequence starts from a current fine-scale state of the robot in the fine representation of the environment which the robot has within the coarse-scale state and ends with a last fine-scale state when the robot reaches the determined fine-scale sub-goal, the robot transitions to a different coarse-scale state, or a maximum sequence length of the sequence of fine-scale states has been reached; Determining the next coarse-scale state of the sequence of coarse-scale states from the last fine-scale state of the sequence of fine-scale states.