Original: The expected regret by law of total probability, since regret is at most NH ,
Revised: The expected regret by law of total probability, since regret is at most NH ,

Original: [Equation]
Revised: [Equation]

Revised: P

Revised: [Equation]

Revised: [Equation]

Original: Since H is lower-order, we have the same rate.
Revised: Since H is lower-order, we have the same rate.

Original: H P ROOF OF M AIN T HEOREMS
Revised: I P ROOF OF M AIN T HEOREMS

Original: First we prove Theorem 4.1 and Theorem 4.2.
Revised: First we prove Theorem 4.1 and Theorem 4.2.

Original: Theorem 4.1 (Regret under generative source access)
Revised: Theorem 4.1 (Regret under generative source access)

Original: Suppose Assumptions 3.1-3.4 and 4.1, and suppose the input policies π k are λ min -exploratory.
Revised: Suppose Assumptions 3.1-3.4 and 4.1, and suppose the input policies π k are λ min -exploratory.

Original: Then, for any δ ∈ (0 , 1) , w.p. 1 − δ , R EP T RANSFER when deployed in the target task has regret at most (cid:101) O (cid:16) ¯ αH 2 d 1 .
Revised: Then, for any δ ∈ (0 , 1) , w.p. 1 − δ , R EP T RANSFER when deployed in the target task has regret at most (cid:101) O (cid:16) ¯ αH 2 d 1 .

Original: 5 (cid:112) T log(1 /δ )
Revised: 5 (cid:112) T log(1 /δ )

Original: (cid:17) , with at most Kn generative accesses per source task, with n = O (cid:16) λ − 1min
Revised: (cid:17) , with at most Kn generative accesses per source task, with n = O (cid:16) λ − 1 min

Original: Aα 3max KT (cid:16) log | Φ | δ + K log | Υ | (cid:17)(cid:17) .
Revised: Aα 3max KT (cid:16) log | Φ | δ + K log | Υ | (cid:17)(cid:17) .

Original: Theorem 4.2 (Regret for R EP T RANSFER with R EWARD F REE subroutine)
Revised: Theorem 4.2 (Regret for R EP T RANSFER with R EWARD F REE subroutine)

Original: Suppose Assumptions 3.1-3.4 and 4.1 hold, and let δ ∈ (0 , 1) .
Revised: Suppose Assumptions 3.1-3.4 and 4.1 hold, and let δ ∈ (0 , 1) .

Original: Let { π k } K − 1 k =

Revised: Let { π k } K − 1 k =

Original: be exploratory policies learned from running R EWARD F REE on each source task with N L SVI -U CB and N R EWARD F REE set as in
Revised: be exploratory policies learned from running R EWARD F REE on each source task with N L SVI -U CB and N R EWARD F REE set as in

Original: Lemma 4.2.
Revised: Lemma 4.2.

Original: Then, w.p. 1 − δ , running R EP T RANSFER with { π k } K − 1 k =1 has regret in the target task of (cid:101) O (cid:16) ¯ αH 2 d 1 .
Revised: Then, w.p. 1 − δ , running R EP T RANSFER with { π k } K − 1 k =1 has regret in the target task of (cid:101) O (cid:16) ¯ αH 2 d 1 .

Original: 5 (cid:112) T log(1 /δ )
Revised: 5 (cid:112) T log(1 /δ )

Original: (cid:17) , with at most (cid:101) O (cid:0) A 4 α 3max d 5 H 7 K 2 Tψ − 2 (log( | Φ | /δ )
Revised: (cid:17) , with at most (cid:101) O (cid:0) A 4 α 3max d 5 H 7 K 2 Tψ − 2 (log( | Φ | /δ )

Original: + K log | Υ | ) (cid:1) generative accesses per source task.
Revised: + K log | Υ | ) (cid:1) generative accesses per source task.

Original: Proof of Theorem 4.1 and Theorem 4.2.
Revised: Proof of Theorem 4.1 and Theorem 4.2.

Original: For the regret bound, set M V = H and M µ = ¯ α and apply
Revised: For the regret bound, set M V = H and M µ = ¯ α and apply

Original: Theorem G.2.
Revised: Theorem H.2.

Original: This choice of M µ is valid by the argument following Assumption G.1.
Revised: This choice of M µ is valid by the argument following Assumption H.1.

Original: This gives us a regret bound of
Revised: This gives us a regret bound of

Original: [Equation]
Revised: [Equation]

Original: where ε ms can be made smaller than 1 / √ T , in which the second term dominates.
Revised: where ε ms can be made smaller than 1 / √ T , in which the second term dominates.

Original: Now, we calculate the pre-training phase sample complexity in a source task.
Revised: Now, we calculate the pre-training phase sample complexity in a source task.

Original: First, let’s calculate the reward-free model learning sample complexity, i.e. this is the number of samples required for learning (cid:98) P k .
Revised: First, let’s calculate the reward-free model learning sample complexity, i.e. this is the number of samples required for learning (cid:98) P k .

Original: Recall that we need this to be sufficiently large such that ε TV = 1 /N
Revised: Recall that we need this to be sufficiently large such that ε TV = 1 /N

Original: L SVI -U CB .
Revised: L SVI -U CB .

Original: As required by Lemma 4.2, we need,
Revised: As required by Lemma 4.2, we need,

Original: [Equation]
Revised: [Equation]

Original: Second, we calculate the cross-sampling sample complexity.
Revised: Second, we calculate the cross-sampling sample complexity.

Original: Recall that n is the number of samples in each pairwise dataset.
Revised: Recall that n is the number of samples in each pairwise dataset.

Original: In order to reduce ϵ ms to 1 / √ T , by Lemma 4.1, we need
Revised: In order to reduce ϵ ms to 1 / √ T , by Lemma 4.1, we need

Original: [Equation]
Revised: [Equation]

Original: which implies that we need
Revised: which implies that we need

Original: [Equation]
Revised: [Equation]

Original: Incorporating the coverage result from Lemma 4.2 gives,
Revised: Incorporating the coverage result from Lemma 4.2 gives,

Original: [Equation]
Revised: [Equation]

Original: Since each task is in at most K − 2 pairwise datasets, each of size n , the total pre-training sample complexity per task is at most,
Revised: Since each task is in at most K − 2 pairwise datasets, each of size n , the total pre-training sample complexity per task is at most,

Original: [Equation]
Revised: [Equation]

Original: Now we prove Theorem 5.2, restated below.
Revised: Now we prove Theorem 5.2, restated below.

Original: Theorem 5.2 (Regret with online access)
Revised: Theorem 5.2 (Regret with online access)

Original: Suppose Assumptions 3.1-3.4,5.1,5.2 hold.
Revised: Suppose Assumptions 3.1-3.4,5.1,5.2 hold.

Original: For any δ ∈ (0 , 1) , w.p. 1 − δ , Algorithm 5 with appropriately set parameters achieves a regret in the targettask at most (cid:101) O (cid:16) ¯ αd 1 .
Revised: W.p. 1 − δ , Algorithm 5 with appropriate parameters achieves a regret in the target (cid:101) O (cid:16) ¯ αd 1 .

Original: 5 H 2 (cid:112) T log(1 /δ )
Revised: 5 H 2 (cid:112) T log(1 /δ )

Original: (cid:17) , with at most poly (cid:0) A, α max , d, H, K, T, ψ − 1 , ψ − 1 raw , log( | Φ || Υ | /δ ) (cid:1) online queries in the source tasks.
Revised: (cid:17) , with at most poly (cid:0) A, α max , d, H, K, T, ψ − 1 , ψ − 1 raw , log( | Φ || Υ | /δ ) (cid:1) online queries in the source tasks.

Original: Proof of Theorem 5.2.
Revised: Proof of Theorem 5.2.

Original: We follows the same format as the proof of Theorem 4.1.
Revised: We follows the same format as the proof of Theorem 4.1.

Original: The regret bound is identical.
Revised: The regret bound is identical.

Original: Now let’s compute the pre-training sample complexity.
Revised: Now let’s compute the pre-training sample complexity.

Original: The regret bound requires us to set ε ms ≤ 1 / √ T .
Revised: The regret bound requires us to set ε ms ≤ 1 / √ T .

Original: Here, our ε ms comes from Lemma F.1, so
Revised: Here, our ε ms comes from Lemma G.1, so

Original: [Equation]
Revised: [Equation]

Original: which implies we need
Revised: which implies we need

Original: [Equation]
Revised: [Equation]

Original: Plugging in the coverage of Lemma 4.2,
Revised: Plugging in the coverage of Lemma 4.2,

Original: [Equation]
Revised: [Equation]

Original: Here, we only collect one dataset, so the total pre-training sample complexity is
Revised: Here, we only collect one dataset, so the total pre-training sample complexity is

Original: NR EWARD F REE + n =
Revised: N

Original: [Equation]
Revised: [Equation]

Original: I E XPERIMENT D ETAILS I. C ONSTRUCTION OF C OMBLOCK
Revised: J E XPERIMENT D ETAILS J. C ONSTRUCTION OF C OMBLOCK

Original: In this section we first introduce the vanilla Combination lock (Comblock) environment that is widely used as the benchmark for algorithms for Block MDPs.
Revised: In this section we first introduce the vanilla Combination lock (Comblock) environment that is widely used as the benchmark for algorithms for Block MDPs.

Original: We provide a visualization of the comblock environment in Fig. 1(a).
Revised: We provide a visualization of the comblock environment in Fig. 1(a).

Original: Concretely, the environment has a horizon H , and 3 latent states z i ; h , i ∈ { 0 , 1 , 2 } for each timestep h and 10 actions.
Revised: Concretely, the environment has a horizon H , and 3 latent states z i ; h , i ∈ { 0 , 1 , 2 } for each timestep h and 10 actions.

Original: Among the three latent states, we denote z 0 , zas the good states which leads to the final reward and z 2 as the bad states.
Revised: Among the three latent states, we denote z 0 , zas the good states which leads to the final reward and z 2 as the bad states.

Original: At the beginning of the task, the environment will uniformly and independent sample 1 out of the 10 actions for each good state z 0; h and z 1; h for each timestep h , and we denote these actions a 0; h , a 1; h as the optimal actions (corresponding to each latent state).
Revised: At the beginning of the task, the environment will uniformly and independent sample 1 out of the 10 actions for each good state z 0; h and z 1; h for each timestep h , and we denote these actions a 0; h , a 1; h as the optimal actions (corresponding to each latent state).

Original: These optimal actions, along with the task itself, determines the dynamics of the environment.
Revised: These optimal actions, along with the task itself, determines the dynamics of the environment.

Original: At each good latent state s 0; h or s 1; h , if the agent takes the correct action, the environment transits to the either good state at the next timestep (i.e., s 0; h +1 , s 1; h +1 ) with equal probability.
Revised: At each good latent state s 0; h or s 1; h , if the agent takes the correct action, the environment transits to the either good state at the next timestep (i.e., s 0; h +1 , s 1; h +1 ) with equal probability.

Original: Otherwise, if the agent takes any 9 of the bad actions, the environment will transition to the bad state s 2; h +1 deterministicly, and the bad states transit to only bad states at the next timestep deterministicly.
Revised: Otherwise, if the agent takes any 9 of the bad actions, the environment will transition to the bad state s 2; h +1 deterministicly, and the bad states transit to only bad states at the next timestep deterministicly.

Original: There are two situations where the agent receives a reward: one is uponing arriving the good states at the last timestep, the agent receives a reward of 1.
Revised: There are two situations where the agent receives a reward: one is uponing arriving the good states at the last timestep, the agent receives a reward of 1.

Original: The other is upon the first ever transition into the bad state, the agent receives an “anti-shaped” reward of 0.1 with probability 0.5.
Revised: The other is upon the first ever transition into the bad state, the agent receives an “anti-shaped” reward of 0.1 with probability 0.5.

Original: Such design makes greedy algorithms without strategic exploration such as policy optimization methods easily fail.
Revised: Such design makes greedy algorithms without strategic exploration such as policy optimization methods easily fail.

Original: For the initial state distribution, the environment starts in s 0;0 or s 1;0 with equal probability.
Revised: For the initial state distribution, the environment starts in s 0;0 or s 1;0 with equal probability.

Original: The dimension of the observation is 2 ⌈ log( H + |S| +1) ⌉ .
Revised: The dimension of the observation is 2 ⌈ log( H + |S| +1) ⌉ .

Original: For the emission distribution, given a latent state s i ; h , the observation is generated by first concatenate the one hot vectors of the state and the horizon, adding i.i.d.
Revised: For the emission distribution, given a latent state s i ; h , the observation is generated by first concatenate the one hot vectors of the state and the horizon, adding i.i.d.

Original: N (0,0.1) noise at each entry, appending 0 at the end if necessary.
Revised: N (0,0.1) noise at each entry, appending 0 at the end if necessary.

Original: Then finally we apply a linear transfermation on the observation with a Hadamard matrix.
Revised: Then finally we apply a linear transfermation on the observation with a Hadamard matrix.

Original: Note that without a good feature or strategic exploration, it takes 10 H actions to reach the final goal with random actions.
Revised: Note that without a good feature or strategic exploration, it takes 10 H actions to reach the final goal with random actions.

Original: I. C ONSTRUCTION OF TRANSFER SETUP IN S ECTION .6.
Revised: J. C ONSTRUCTION OF TRANSFER SETUP IN S ECTION .

Original: In this section we introduce the detailed construction of the experiment in Section.
Revised: In this section we introduce the detailed construction of the experiment in Section.

Original: For the source environment, we simply generate 5 random vanilla comblock environment described in Section.
Revised: For the source environment, we simply generate 5 random vanilla comblock environment described in Section.

Original: I.1.
Revised: J.1.

Original: Note that in this way we ensure that the emission distribution shares across the sources, but the latent dynamics are different because the optimal actions are independently randomly selected.
Revised: Note that in this way we ensure that the emission distribution shares across the sources, but the latent dynamics are different because the optimal actions are independently randomly selected.

Original: For the target environment, for each timestep h , we randomly acquire the optimal actions at h from one of the sources and set it to be the optimal action of the target environment at timestep h , if the selected optimal actions are different for the two good states.
Revised: For the target environment, for each timestep h , we randomly acquire the optimal actions at h from one of the sources and set it to be the optimal action of the target environment at timestep h , if the selected optimal actions are different for the two good states.

Original: Otherwise we keep sampling until they are different.
Revised: Otherwise we keep sampling until they are different.

Original: Note that under such construction, since we fix the emission distribution, Assumption 3. is satisfied if we set α = 1 for the source environment where we select the optimal action and α =for the other sources, at each timestep.
Revised: Note that under such construction, since we fix the emission distribution, Assumption 3. is satisfied if we set α = 1 for the source environment where we select the optimal action and α =for the other sources, at each timestep.

Original: To see how Assumption 5.1 is satisfied, recall that comblock environment naturally satisfies Assumption 3.3, and identical emission implies that the conditional ratio of all observations between source and target is1.
Revised: To see how Assumption 5.1 is satisfied, recall that comblock environment naturally satisfies Assumption 3.3, and identical emission implies that the conditional ratio of all observations between source and target is

Original: I. C ONSTRUCTION OF TRANSFER SETUP IN S ECTION .6.
Revised: . J. C ONSTRUCTION OF TRANSFER SETUP IN S ECTION .

Original: Now we introduce the construction of the Comblock with Partitioned Observation (ComblockPO) environment.
Revised: Now we introduce the construction of the Comblock with Partitioned Observation (ComblockPO) environment.

Original: Comparing with the vanilla comblock environment, the major difference is in the observation space.
Revised: Comparing with the vanilla comblock environment, the major difference is in the observation space.

Original: In this setting, the size of the observation depends on the number of source environments K .
Revised: In this setting, the size of the observation depends on the number of source environments K .

Original: Let the size of the original observation space be O = |O| , the size of the observation for comblock-PO is KO .
Revised: Let the size of the original observation space be O = |O| , the size of the observation for comblock-PO is KO .

Original: For the k -th source environment, where k ∈ [ K ] , the environment first generates the O -dimensional observation vector as in the original comblock, and then embed it to the ( k − 1) O -th to kO -th entries of the KO -dimensional observation vector, where it is 0 everywhere else.
Revised: For the k -th source environment, where k ∈ [ K ] , the environment first generates the O -dimensional observation vector as in the original comblock, and then embed it to the ( k − 1) O -th to kO -th entries of the KO -dimensional observation vector, where it is 0 everywhere else.

Original: Thus we can see that the observation space for each source environment is disjoint (and thus the name partitioned observations).
Revised: Thus we can see that the observation space for each source environment is disjoint (and thus the name partitioned observations).

Original: For the target enviornment, since the latent dynamcis are the same, we only need to design the emission distribution: for each latent state s i ; h , we assign the emission distribution uniformly at random from one of the sources.
Revised: For the target enviornment, since the latent dynamcis are the same, we only need to design the emission distribution: for each latent state s i ; h , we assign the emission distribution uniformly at random from one of the sources.

Original: I. I MPLEMENTATION DETAILS
Revised: J. I MPLEMENTATION DETAILS

Original: Our implementation builds on B RIEE (Zhang et al., 2022).
Revised: Our implementation builds on B RIEE (Zhang et al., 2022).

Original: In the Multi-task R EP L EARN stage, we requires our learned feature to predict the Bellman backup of all the sources simultaneously.
Revised: In the Multi-task R EP L EARN stage, we requires our learned feature to predict the Bellman backup of all the sources simultaneously.

Original: Therefore, in each iteration we have k discriminators and k sets of linear weights (instead of 1 in
Revised: Therefore, in each iteration we have k discriminators and k sets of linear weights (instead of 1 in

Original: B RIEE ), where k is the number of source environments.
Revised: B RIEE ), where k is the number of source environments.

Original: For the deployment stage we implement
Revised: For the deployment stage we implement

Original: LSVI following Algorithm 6.
Revised: LSVI following Algorithm 6.

Original: To create the training dataset for Multi-task R EP L EARN , for each ( i, j ) environment pairs where i ̸
Revised: To create the training dataset for Multi-task R EP L EARN , for each ( i, j ) environment pairs where i ̸

Original: = j , we collect 500 samples for each timestep h .
Revised: = j , we collect 500 samples for each timestep h .

Original: For each ( i, i ) environment pairs, we collect 500 × ( k − 1)
Revised: For each ( i, i ) environment pairs, we collect 500 × ( k − 1)

Original: × k samples for each timestep h , where k denotes the number of sources.
Revised: × k samples for each timestep h , where k denotes the number of sources.

Original: Thus we ensure that the number of samples from cross transition of different environments is the same as the number of samples from cross transition of the same environment.
Revised: Thus we ensure that the number of samples from cross transition of different environments is the same as the number of samples from cross transition of the same environment.

Original: For the online setting, we simply sample 1000 × ( k − 1) × k samples for each ( i, i ) cross transition to ensure that the total number of samples is the same for G-R EP T RANSFER and O-R EP T RANSFER .
Revised: For the online setting, we simply sample 1000 × ( k − 1) × k samples for each ( i, i ) cross transition to ensure that the total number of samples is the same for G-R EP T RANSFER and O-R EP T RANSFER .

Original: To sample the initial state action pair (i.e., (˜ s, ˜ a ) pair as in (1)), for 90% of the samples, we follow the final policy from each source environment trained using B RIEE .
Revised: To sample the initial state action pair (i.e., (˜ s, ˜ a ) pair as in (1)), for 90% of the samples, we follow the final policy from each source environment trained using B RIEE .

Original: For the remaining 10% , we follow the same policy to state ˜ s , and then take a uniform random policy to get ˜ a .
Revised: For the remaining 10% , we follow the same policy to state ˜ s , and then take a uniform random policy to get ˜ a .

Original: With this sampling scheme we ensure that Assumption 3.3 is satisfied.
Revised: With this sampling scheme we ensure that Assumption 3.3 is satisfied.

Original: In the setting of Section.
Revised: In the setting of Section.

Original: 6.2, we follow a more simple procedure to ensure that the samples are more balanced among the three states: we skip the first sampling step from environment
Revised: 6, we follow a more simple procedure to ensure that the samples are more balanced among the three states: we skip the first sampling step from environment

Original: i (i.e., sample s given (˜ s, ˜ a ) ), and simply reset environment i to s , where s is one of the three states with equal probability, and generate the observation accordingly.
Revised: i (i.e., sample s given (˜ s, ˜ a ) ), and simply reset environment i to s , where s is one of the three states with equal probability, and generate the observation accordingly.

Original: Note that such visitation distribution is also possible in the online setting with a more nuanced sampling procedure, and in the experiment we use the same sampling procedure for both G-R EP T RANSFER and O-R EP T RANSFER for a fair comparison.
Revised: Note that such visitation distribution is also possible in the online setting with a more nuanced sampling procedure, and in the experiment we use the same sampling procedure for both G-R EP T RANSFER and O-R EP T RANSFER for a fair comparison.

Original: I. H YPERPARAMETERS
Revised: J. H YPERPARAMETERS

Original: In this section, we record the hyperparameters we try and the final hyperparameter we use for each baselines.
Revised: In this section, we record the hyperparameters we try and the final hyperparameter we use for each baselines.

Original: The hyperparameters for R EP T RANSFER in Section.
Revised: The hyperparameters for R EP T RANSFER in Section.

Original: 6.1 is in Table.
Revised: 6 is in Table.

Original: The hyperparameters for R EP T RANSFER in Section.
Revised: The hyperparameters for R EP T RANSFER in Section.

Original: 6.2 is in Table.
Revised: 6 is in Table.

Original: The hyperparameters for B RIEE is in Table.
Revised: The hyperparameters for B RIEE is in Table.

Original: We use the same set of hyperparameters for G-R EP T RANSFER and O-R EP T RANSFER . 
Revised: We use the same set of hyperparameters for G-R EP T RANSFER and O-R EP T RANSFER . [Table] [Table] [Table]

Revised: [Table]

Revised: {1e

Original: [Table] 
Revised: [Table] [Table]

Revised: Discriminator f hidden layer size {256}

Revised: RepLearn

Revised: [Table]

Revised: T {30}

Revised: [Table]

Revised: number

Revised: [Table]

Revised: gradient steps {64}

Revised: [Table]

Revised: number

Revised: [Table]

Revised: steps {64}

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: RepLearn

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: steps {

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: steps {

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: Decoder

Revised: [Table]

Revised: batch size {

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Revised: coefficient λ {0.01}

Revised: [Table]

Revised: [Table]

Revised: temperature {1}

Revised: [Table]

Revised: softmax temperature {0.1,1}

Revised: [Table]

Revised: β {

Original: [Table]
Revised: [Table]

Original: I. V ISUALIZATIONS
Revised: J. V ISUALIZATIONS

Original: In this section we provide a comprehensive visualization of the decoders for all baselines in the target environment.
Revised: In this section we provide a comprehensive visualization of the decoders for all baselines in the target environment.

Original: We observe that the behaviors of all baselines are similar across the 5 random seeds.
Revised: We observe that the behaviors of all baselines are similar across the 5 random seeds.

Original: Thus to avoid redundancy
Revised: Thus to avoid redundancy, we only show the visualization from 1 random seed.

Original: [Table]
Revised: We provide an example in Fig. 3 on how to interpret the visualization: let the emission function of the target environment be o , and let the decoder that we are evaluating be ϕ , and to generate the blue block in Fig. 3, we sample 30 observations { s n } 30 n =1 from the target environment at z 1 , 13 , the latent state 1 (the title of the subplot) from timestep 13 (the x-axis). Concretely, { s n } 30 n =1 ∼ o ( · | z 1 , 13 ) .

Original: we only show the visualization from  1 random seed.
Revised: The blue block denotes the three-dimensional decoded latent states ˆ z from these 30 observations: ˆ z = 1 30 (cid:80) 30 n =1 ϕ ( s n ) .

Original: I.6. V ISUALIZATIONS FROM S ECTION .6.
Revised: [Table] J.6. V ISUALIZATIONS FROM S ECTION .

Original: We record the visualization of the 5 sources from Fig.
Revised: We record the visualization of the 5 sources from Fig.

Original: 3 to Fig.
Revised: 3 to Fig.

Original: 7; O-R EP T RANSFER in Fig.8;
Revised: 7; O-R EP T RANSFER in Fig.8;

Original: G-R EP T RANSFER in Fig.
Revised: G-R EP T RANSFER in Fig.

Original: 9; running B RIEE on target in Fig.
Revised: 9; running B RIEE on target in Fig.

Original: I.6. V ISUALIZATIONS FROM S ECTION . 6.
Revised: [Table] [Figure] [Table] [Figure]

Original: We record the visualization of the 2 sources from Fig.
Revised: We record the visualization of the 2 sources from Fig.

Original: 11 and Fig.
Revised: 11 and Fig.

Original: 12; O-R EP T RANSFER in Fig.13;
Revised: 12; O-R EP T RANSFER in Fig.13;

Original: G-R EP T RANSFER in Fig.
Revised: G-R EP T RANSFER in Fig.

Original: 14; running B RIEE on target in Fig.
Revised: 14; running B RIEE on target in Fig.

Original: Note that the features collapse at some timesteps in Fig.
Revised: Note that the features collapse at some timesteps in Fig.

Original: 14 and Fig. 15, but this is acceptable because the optimal actions at those timesteps 
Revised: 14 and Fig. 15, but this is acceptable because the optimal actions at those timesteps are the same for the collapsed states. [Figure] [Table] [Figure]

Revised: [Figure]

Original: [Table]
Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Figure] [Table]
Revised: [Table]

Original: [Figure][Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Figure] [Table]
Revised: [Table]

Original: [Figure][Figure] [Table] [Figure]
Revised: [Figure]