Title: Enhancing Robot Program Synthesis Through Environmental Context
Venue: NeurIPS 2023
Accepted: yes
Keywords: program synthesis

partial envrionment

robotic programming

domain-specific language
Abstract: Program synthesis aims to automatically generate an executable program that conforms to the given specification. Recent advancements have demonstrated that deep neural methodologies and large-scale pretrained language models are highly proficient in capturing program semantics.
For robot programming, prior works have facilitated program synthesis by incorporating global environments. However, the assumption of acquiring a comprehensive understanding of the entire environment is often excessively challenging to achieve.
In this work, we present a framework that learns to synthesize a program by rectifying potentially erroneous code segments, with the aid of partially observed environments. To tackle the issue of inadequate attention to partial observations, we propose to first learn an environment embedding space that can implicitly evaluate the impacts of each program token based on the precondition. Furthermore, by employing a graph structure, the model can aggregate both environmental and syntactic information flow and furnish smooth program rectification guidance.
Extensive experimental evaluations and ablation studies on the partially observed VizDoom domain authenticate that our method offers superior generalization capability across various tasks and greater robustness when encountering noises.

Enhancing Robot Program Synthesis Through Environmental Context

Abstract

Program synthesis aims to automatically generate an executable program that conforms to the given specification. Recent advancements have demonstrated that deep neural methodologies and large-scale pretrained language models are highly proficient in capturing program semantics. For robot programming, prior works have facilitated program synthesis by incorporating global environments. However, the assumption of acquiring a comprehensive understanding of the entire environment is often excessively challenging to achieve. In this work, we present a framework that learns to synthesize a program by rectifying potentially erroneous code segments, with the aid of partially observed environments. To tackle the issue of inadequate attention to partial observations, we propose to first learn an environment embedding space that can implicitly evaluate the impacts of each program token based on the precondition. Furthermore, by employing a graph structure, the model can aggregate both environmental and syntactic information flow and furnish smooth program rectification guidance. Extensive experimental evaluations and ablation studies on the partially observed VizDoom domain authenticate that our method offers superior generalization capability across various tasks and greater robustness when encountering noises.

1 Introduction

Program synthesis endeavors to produce an executable program that fulfills the prescribed specifications. Given that formulating a definitive and lucid specification can often prove more arduous than physically authoring the program, a widely-used method is to furnish input/output (I/O) examples as a close approximation of the requisite specification [16, 17, 27, 34], also known as Programming By Example (PBE). It allows a more straightforward presentation of the desired program functionality. The utilization of PBE enables machines to acquire the capacity to implicitly capture a set of regulations and generalize them to analogous scenarios. Over the past few years, PBE has demonstrated its remarkable potential to elevate automation and ease human labor in numerous domains, including but not limited to array transformation [3, 46, 56], graphic design [16], and comprehension of human demonstrations [21, 42].

In the realm of robot programs, a crucial factor is the interaction between the robot and its environment. In practice, a robot can solely rely on its sensors to perceive the environment by gathering diverse types of data, which forms the basis for every decision. Consider a robot designed for food delivery,

which is tasked with transporting food to a specified destination. The robot is faced with a decision between two paths: one that is obstructed by barriers and leads to a dead-end, and another that is more circuitous and distant. If the robot is furnished with comprehensive knowledge of the environment, it may be possible to create a simple program consisting of only a few actions. However, if the robot can only obtain partial observations, such as RGB image data from its current perspective, a more intricate program with sophisticated control flow may be necessary to navigate out of the dead-end and reach the intended destination. Consequently, the latter program can generalize better in accomplishing more complex tasks. The variation in the level of observation can significantly impact the ultimate outcome. Therefore, the challenge in robot program synthesis arises from the fact that robots are limited to partial observations of their surroundings, making it difficult to assess the global impact of the generated program tokens toward the desired output. This is due to the fact that a comprehensive understanding of the environment is not achievable.

At present, there exists a diverse range of deep learning techniques aimed at solving PBE problems using neural network models [5, 8, 32]. Empirical studies [3, 13] have shown that recurrent neural network (RNN) architectures can learn strategies that generalize across problems and remain robust to moderate levels of noise. However, these methods have limitations in their ability to effectively utilize environmental observations due to their reliance on the RNN as the backbone. Additionally, recent studies have introduced reinforcement learning [7, 28], debugger method [19] and latent generative methods [10, 48] into the field. Furthermore, other studies have revealed that incorporating intermediate states produced by executing partially generated programs can improve the capability to address program synthesis tasks, especially for sequential programming tasks [9, 43, 47, 56]. These techniques have primarily been evaluated on the Karel [12, 18, 44] dataset, in which the model possesses complete awareness of the entire environment. To clarify, even if a particular location in the environment is inaccessible or unknown to the robot, the model can still generate a program with knowledge of that spot. Nonetheless, this assumption of acquiring a global view of the environment is arduous to achieve in reality, and their aptitude to generate accurate programs with only partial observations remains to be explored.

To tackle this issue, we introduce our novel E nvironmental-context V alidated l A tent P rogram S ynthesis framework (EVAPS). Drawing inspiration from the successful previous work SED [19], as well as the current trend of large language models [1, 22, 23], we believe that the trail-eval-repair loop provides valuable guidance for the evolution of programs, leading to better generalization ability. In order to leverage partial environmental observations, EV APS initially obtains candidate programs by employing the same neural program synthesizer component used in SED. Subsequently, by executing the candidate program, EVAPS acquires the partial environment before and after each action ( i.e., the environmental context). By concurrently modeling both the environmental and syntax contexts, EVAPS iteratively alters the programs that do not produce the correct output in an effort to resolve semantic conflicts across program tokens, ultimately correcting erroneous program fragments.

The framework was assessed in the partially observed VizDoom domain [24, 55], where a robot operates in a 3D world and interacts with its environment, including objects and adversaries. The results of the experiment demonstrate that the proposed approach is superior in modeling program semantic subtleties and resolving potential errors, thereby enhancing its generalization ability. Additionally, we authenticate the efficacy of leveraging environmental contexts and aligning them with code syntax by carrying out an ablation study. Further experiments show that our method is more capable of solving complex tasks compared to prior works. Furthermore, we validate that our method remains robust when encountering various levels of noise.

2 Problem Formulation

In a typical PBE task, we are given a set of N training samples, represented as { { IO } K , P } N . Each sample consists of K I/O pairs that act as specifications, and a gold program P that correctly maps the input state I to the output state O , such that P ( I k i ) → O k i , where ∀ i ∈ [1 , N ] , ∀ k ∈ [1 , K ] . Previous studies have investigated techniques to synthesize a candidate program ˆ P solely based on I/O. Our objective is to train a neural model, denoted as G , which takes a candidate program ˆ P , and the corresponding environmental contexts O obtained by executing the program ˆ P as an input. The model generates a refined program by incorporating environmental contexts: G ( ˆ P , O ) →P ∗ , such that the mapping of program P ∗ ( I ) approaches more closely to P ( I ) given input I . To assess the

ifelse

inTarget

attack

else

moveLeft

def

run

Candidate Program Graph

Vizdoom

Interpreter

Program Segments

𝒪𝑝𝑟𝑒

1

𝒪𝑝𝑜𝑠𝑡

1

𝒪𝑝𝑟𝑒

2

𝒪𝑝𝑜𝑠𝑡

2

Convolution

𝑆1

𝑆2

𝒉1

1

𝒉2

1

𝒉3

1

𝒉4

1

𝒉1

2

𝒉2

2

Graph Attention

𝒪1

′

MLP

+ Softmax

𝒪2

′

𝒉1

′1

𝒉2

′1

𝒉3

′1

𝒉4

′1

𝒉1

′2

𝒉2

′2

(b)

(a)

LSTM

𝜖 1

′1

𝜖 2

′1

𝜖 3

′1

𝜖 4

′1

𝜖 1

′2

𝜖 2

′2

Decoding

Refined Program Tokens

𝒉1

1

𝒉2

1

𝒉3

1

𝒉4

1

𝒉2

′1

Program Graph Attention

Candidate Program

def run():

ifelse inTarget demon:

attack()

else:

moveLeft()

generalization ability of the model G , we include another set of held-out examples that are not part of the training samples, denoted as { IO } K test , enabling us to measure the equivalence between P ∗ and P using IO k ∈ { IO } K ∪ { IO } K test .

3 Methodology

3.1 Leveraging Partial Observations

In the VizDoom domain, the robot perceives its surroundings O by collecting an RGB image O R ∈ R n H × n W × n C and a depth buffer O D ∈ R n H × n W × 1 from its visual perspective, denoted as O = ⟨O R , O D ⟩ . This forms its partial observation of the world. By executing a program, we can obtain a sequence of observation changes O 0 ac 1 = ⇒O 1 ac 2 = ⇒ ... ac ne = ⇒O n e , where ac i refers to the i -th executed action, n e denotes the number of total execution steps, and O 0 denotes the initial observed state. As depicted in Figure 1(a), the hidden environment representations can be learned by applying convolutional network layers, and the embedded vector at the l -th layer can be denoted as:

where K represents the convolutional kernel, and g [ l ] signifies the activation function at layer l .

To effectively utilize insights gleaned from partial observations, it is essential that we obtain the antecedent states O pre (also known as the precondition) and the potential consequent effects O post corresponding to each token of program ˆ P , so that the model can implicitly evaluate whether this token contributes towards completing the task in the current context, and determine whether substitution with another token would heighten the likelihood of task fulfillment.

Given that only those tokens that denote robot actions ( e.g., moveForward ) have the potential to alter the environment, whereas other tokens function as constituents of a particular statement ( e.g., if and while ), a straightforward token-by-token modification strategy appears unappealing. Since non-action tokens cannot exist independently and can only render a code block semantically complete when they co-occur with action tokens, we deem it more fitting to process them as a unified segment.

We define a segment as a triplet comprising of S = ⟨ cond, token, body ⟩ . Here, cond refers to the condition of the segment, and body denotes the associated actions or subsegments. In the case of an action token, the segment form deteriorates into S = ⟨ null, action, null ⟩ . By recursively parsing the abstract syntax tree of program ˆ P , we can eventually yield a collection consisting of T program segments, denoted as S T . And by executing program segment S t ∈ S T using the VizDoom interpreter, we can acquire the post-segment effect with Execute ( S t , O pre ) → O post , thereby obtaining the environmental context 〈 O t pre , O t post 〉 of segment S t , presented in Figure 1. The interconnection between the observable world state change and the code segment promotes greater semantic versatility, which, in turn, enables the model to calculate the probability of substituting each token based on the partial environment, resulting in the following formulation:

where L t denotes the number of tokens in segment S t , ϵ s refers to the s -th token of segment S t , and ϵ ′ s represents the refined program token that substitutes ϵ s . To optimize the model parameters, the objective is to minimize the cross-entropy loss, given N training examples, where S i T represents the program segment set of the i -th example, which contains T i segments:

3.2 Code Symbol Alignment

Now that the environmental contexts have been obtained, they are currently paired separately with their own code segments, and the correlations among environmental contexts are not fully exploited. Relying solely on partial observation is insufficient for the model to comprehend the semantic and syntactic long-range connections of program tokens, making it arduous for the model to consolidate information that traverses through subsequent segments during the program repair process. Furthermore, with regard to program syntax, the inherent nature of a program necessitates the model to jointly reason over code symbols, such as types. For instance, the code while(turnLeft) is syntactically incorrect because turnLeft is an action rather than a perception, which is prohibited to follow while . Therefore, it is desirable to align the code symbols with the partial observations to facilitate a more comprehensive information flow.

To accomplish this, we first construct a program graph G ⊆ V × E (presented in Figure 1) to represent each candidate program ˆ P , where V denotes the set of program token nodes and E denotes the set of undirected edges. Subsequently, we employ graph attention [6, 50] for each token to capture a contextualized representation. Similar to the definition of a segment, each node within the program graph possesses a leftward pointer that directs towards nodes signifying conditions, and a rightward pointer that indicates a subsequent token node. The node's value encompasses the token index, and token type. Meanwhile, the relationship between tokens and their corresponding segments is stored. In this way, we not only align the code symbols and environmental contexts but also aggregate further semantic and syntactical representations. Additionally, it enables the model to effectively capture the nested structure of a program, thereby bringing more expressiveness.

To elaborate, we begin by obtaining the node representations h i ∈ R d for each node V i ∈ G through embedding. Subsequently, a scoring function α : R d × R d → R is employed to compute the attention score of its neighboring node V j . This score reflects the significance of a neighbor node. The attention scores are then normalized across all neighbors j ∈ N i using softmax, which is defined as:

where A and W are weight matrices that can be learned, and ∥ represents vector concatenation. As a result, node V i acquires a new representation by calculating the weighted mean of the modified features of its neighboring nodes, as illustrated in Figure 1(b):

Next, we can obtain the corresponding environmental embedding of the token in V i through O ′ i = Conv ( O i , K ) (Eq. (1)). To align the code symbol representations and environmental contexts, we update the representation of node V i by simply concatenating these embeddings x i = [ h ′ i ∥O ′ i ] . This leads to a modification to Eq. (2):

The key connection between the proposed approach and observable environments resides in establishing a connection between the program execution context and the partially observable environment. EVAPS takes the execution context of segment S and the corresponding partially observable environment as a unit for model training. This enables the combination of program syntax and the corresponding partially observable environment to predict a token, thereby enhancing the accuracy of program synthesis. As a result, the model can adeptly resolve the underlying conflict within a program by concurrently capturing well-integrated and highly pertinent semantic and contextual patterns, culminating in a greater practicality of programs.

4 Experiment

4.1 Experimental Setup

VizDoom Domain. The programs in this study are structured using Domain-Specific Languages (DSL) in the VizDoom environment [24]. This environment provides a 3D world that closely resembles real-world scenarios, allowing the robot to efficiently perceive, interpret, and learn the 3D realm. In recent times, there has been a burgeoning interest in VizDoom exploring the robot's capabilities to make tactical and strategic determinations [2, 11, 14, 25, 41, 47, 53]. The VizDoom experimental environment consists of 7 action primitives, 6 perception primitives, and the state representation dimension is 120 × 160 × 3 . The average number of steps required to complete

Program 𝑚≔ def run() : 𝑠 Statement 𝑠 ≔ while( 𝑏 ) : ( 𝑠 ) | 𝑠 1 ; 𝑠2 | 𝑎 | repeat( 𝑟 ) : ( 𝑠 ) | | if( 𝑏 ) : ( 𝑠 ) | ifelse( 𝑏 ) : ( 𝑠1 ) else : ( 𝑠2 ) Repetition 𝑟 ≔ Integer number of repetitions Condition 𝑏≔ perception 𝛾 | not 𝑏 Action 𝑎≔ moveForward | moveBackward | moveLeft | moveRight | turnLeft | turnRight | attack Perception 𝛾≔ isThere 𝜀 | inTarget 𝜀 Monster 𝜀 ≔ demon | hellKnight | revenant

tasks is 4 . 6 . We contend that the VizDoom environment is more suitable for robot program synthesis research compared to Karel, due to its limited partially observed nature [55], which renders the decision and execution process more realistic and practical for the robot. The DSL comprises actions, perceptions, and control flows, sufficient to encapsulate the fundamental concepts of robot programming. The detailed description is presented in Figure 2.

Dataset. To generate a dataset for learning environmental-context embeddings, we adopt the same approach as previous studies [14, 47], randomly generating 100 , 000 distinct samples. The dataset is then partitioned into a training set with 70 , 000 samples, a validation set with 20 , 000 samples, and a testing set with 10 , 000 samples. We adhere to the program synthesis conventions [7, 19], where each sample includes a correct program, 5 input-output pairs that act as specifications, and an extra input-output example as the test sample, which is solely employed for evaluation purposes and not for training. During the generation process, we also conduct checks to ensure that all execution branches in the program are covered at least once, thereby representing all aspects of the program's behavior and providing sufficiently challenging tasks.

Evaluation Metric. To comprehensively assess the capability of synthesizing programs based on provided inputs and outputs, we evaluate all models using three metrics: (1) Exact Match . It considers a generated program to be an exact match if every token in it is identical to the gold program. This is expressed mathematically as: ∑ N i =1 Φ[ TKN t ( P ∗ i ) = TKN t ( P i )] , ∀ t ∈ [1 , L ] where TKN t ( P i ) refers to the t -th token of program P i , Φ[ · ] yields 1 if the condition is satisfied and 0 otherwise, and L denotes the length of the program. (2) Semantic Match . It takes into account that a synthesized program may be semantically equivalent but not syntactically identical to the gold program. This metric is expressed as: ∑ N i =1 Φ[ P ∗ i ( I k i ) = P i ( I k i ) = O k i ] , ∀ I k i , O k i ∈ { IO } K i . (3) Generalization Match . To genuinely evaluate the generalization ability of a model, it is crucial to assess its ability to generalize on unseen samples in addition to semantic consistency. To achieve this, we include additional examples and evaluate the model's performance as: ∑ N i =1 Φ[ P ∗ i ( I k i ) = P i ( I k i ) = O k i ] , ∀ I k i , O k i ∈ { IO } K i ∪{ IO } K test i . Kindly note that our primary concern regarding the model's performance is the metric of generalization match.

4.2 Results

Overall Synthesis Performance. Table 1 presents the primary outcomes of our EV APS in comparison to prior works [7, 10, 43, 49] for VizDoom program synthesis. Notably, we have implemented the beam search technique for Inferred Trace, Latent Execution, and Transformer to enable the generation of multiple candidate programs while preserving their original architecture. Additionally, we have incorporated a convolutional neural network on top of the Transformer to facilitate the learning of specification embedding, given that the specification is in the form of grids rather than sequences, and the Transformer serves as the decoder. The supplementary material contains the detailed parameters of all the compared methods.

We can observe that EVAPS consistently outperforms other methods across three metrics, with an approximate improvement of +9 . 6% compared to the runner-up. This indicates its superior capability to resolve potential semantic errors by incorporating environmental contexts, thereby enhancing its generalization ability while preserving syntax modeling ability. Moreover, EVAPS surpasses SED primarily due to the fact that SED only enhances program tokens based on execution outcomes and

specifications, whereas EVAPS places greater emphasis on the semantic locality, supplemented by utilizing partial observations. While the attention mechanism of Transformer facilitates its ability to learn syntax, its generalization ability is relatively poor. Although LGRL can achieve comparable results in terms of generalization, it faces challenges in producing program sequences that are identical to the gold programs due to its tendency to generate semantically equivalent programs in a more complex environment. Inferring traces can provide information gain, but the partially observed environment restricts the model from capturing subtle semantic features. This limitation also applies to the latent executor, as the global status becomes agnostic to the model, thereby limiting its ability to extrapolate the execution states and learn the latent effects of each token.

Ablation Study. To thoroughly verify the efficacy of the partial observation leveraging module and the code symbol alignment module of EVAPS, we perform an ablation study to authenticate their capabilities and scrutinize their impacts as Table 2 presents. We consider the following ablations:

Naïve: a variation of the program synthesis baseline LGRL [7] that utilizes greedy decoding ( i.e., implements a beam search with beam size B = 1 ). It employs a standard encoder-decoder architecture that acquires the ability to directly synthesize a program from scratch.

EVAPS+O: an ablation of EVAPS incorporating partial environmental observations to modify tokens and resolve semantic conflicts in synthesized programs.

EVAPS+S: an ablation of EVAPS aggregating long-range semantic and syntax connections of neighboring code symbols.

EVAPS (EVAPS+O+S): incorporates both modules, allowing for a more comprehensive information flow and leveraging environmental context to facilitate program synthesis.

Based on the ablation outcomes presented in Table 2, we can deduce that utilizing only partial environmental observations or code symbol alignment can enhance the joint semantic and syntax modeling ability. However, the information gain is limited if only one of these techniques is employed. This is because the former concentrates primarily on the environmental context of a particular token, disregarding the implicit connections between different code segments in semantic spaces. On the other hand, the latter focuses mainly on information aggregation rather than environmental contexts. Consequently, combining these techniques can endow the model with supplementary information and further boost the program synthesis process, resulting in an approximate enhancement of +16% in terms of generalization.

Task Complexity. Considering the varying complexities of different robot tasks, generating corresponding codes to accomplish them may pose varying levels of difficulty. Hence, we endeavor to delve deeper into the efficacy of each method in handling diverse levels of complexity. The issue of program aliasing, wherein two programs are semantically equivalent but not identical (e.g., while(r=3):turnLeft and turnRight ), renders the evaluation of complexity using program

20

30

40

50

60

70

80

Generalization (%)

Steps = 2

Steps = 3

Steps = 4

1

5

10

15

20

Top-K

20

30

40

50

60

70

80

Generalization (%)

Steps = 5

1

5

10

15

20

Top-K

Steps = 6

1

5

10

15

20

Top-K

Steps = 7+

LGRL

SED

Inferred Trace

Latent Execution

Transformer

EVAPS (Ours)

length or control flow presence imperfect. Hence, we propose partitioning the tasks into different complexity levels based on the minimum number of steps required to complete the task , which we denote as steps.

To be specific, we have divided the testing set into six distinct categories based on their level of complexity, with each category requiring 2, 3, 4, 5, 6, or 7+ steps to complete, respectively. The 7+ category is reserved for tasks that demand no fewer than seven steps to accomplish. We then proceed to train and evaluate the model's performance in each category separately, and the results of this analysis are presented in Figure 3. Firstly, it is evident that as the complexity of the task increases, all methods experience a certain degree of performance degradation. This is reasonable since the embedding space becomes more intricate to learn as the task becomes more complex. Furthermore, we can infer that the performance gap between EVAPS and other methods widens as the complexity increases. This suggests that EV APS is better equipped to adapt to more intricate tasks, owing to its ability to break down complex objectives into simpler behaviors by leveraging partial environmental contexts. On the other hand, the modeling ability of Latent Execution becomes relatively inferior, particularly for Step=7+, which indirectly implies that it is challenging for a model to deduce the impact of complex behaviors in a single attempt.

Interestingly, for most techniques, the ability to generalize experiences a surge at approximately the Top-5 candidate programs. However, the rate of improvement tends to decelerate and the curve becomes flat. We surmise that this phenomenon occurs because certain semantic representations are conspicuous and uncomplicated to acquire, whereas others are more nuanced. Even with more candidate programs, the semantic features corresponding to these programs are somewhat overlapped, thereby reaching the limit for capturing semantic subtleties and hindering further generalization.

Handling Noisy Observations. For robot programming, real-world observations are collected through sensors in real-time, making it inevitable for occasional noise to occur. Consequently, we tend to evaluate the ability of our method to handle such noise. Given that the VizDoom dataset lacks noisy examples, we artificially introduce noise into the dataset by invalidating a certain percentage of observation grids based on uniform random probability [15].

The results regarding different levels of noise (with a maximum noise of 20%) are illustrated in Figure 4. It can be inferred that the presence of noisy observations inevitably undermines the capacity of EVAPS to learn program semantics and generalize. Nevertheless, EVAPS still manages to exhibit a decent modeling ability. If we consider all Top-K potential candidates, the maximum performance drop obtained is merely about 7.5%. Meanwhile, with fewer candidate programs, the performance gap widens, but it is bound to be 12%. This outcome aligns with our expectations, thus we can conclude that EV APS is capable of recognizing significant semantic patterns and maintaining robustness when encountering noises.

1

5

10

15

20

Top-K

40

50

60

70

Generalization (%)

0%

5%

10%

15%

20%

Dicussion. The partial observation that the proposed approach relies on is often available in practice, thus it is theoretically feasible to apply this approach to the real world. Certainly, we anticipate some practical difficulties during the transition from the simulation environment to the real world. For instance, it can be labor-intensive to collect enough environmental data from the real world for the purpose of training. Further investigation into the efficient acquisition of environmental data while sustaining its quality is yet to be explored in future works.

5 Related Work

Programming By Example. In the realm of program synthesis, the synthesis of programs from examples has long been a preoccupation of researchers. In recent years, deep learning methods have made many breakthroughs [37, 13, 36], and many researchers have proposed different methods for Improve the accuracy of synthesis. Symbolic search is regarded as an efficient technique for addressing program synthesis challenges [40, 38, 51, 26]. Wang et al. have devised a language for abstract queries, which facilitates the generation of SQL [51]. Moreover, Spoc has recognized the significance of pseudocode and explores its potential for aiding code search, focusing on alternative translations of pseudocode [26]. While symbolic search can yield promising outcomes within specific training domains, it lacks the ability to expand, and it is relatively hard to generate code snippets beyond the training datasets.

To tackle the issue of code fragment homogeneity, two approaches have gained prominence: the utilization of hidden variables and the employment of reward mechanisms. In order to effectively utilize the information contained in latent variables, some researchers consider using latent generative models [3, 10, 48]. Aim at obtaining the intermediate execution results of partial programs in C language, Chen et al. approximate by learning the latent representation of partially generated programs to facilitate program token search [10]. Additionally, reinforcement learning has been integrated into synthetic neural network training in numerous studies [7, 16, 9, 48, 19]. Bunel et al. combined program synthesis with reinforcement learning [7], and avoided the problem of Program Aliasing by defining proper rewards; Trivedi et al. considered a two-stage learning scheme on this basis [48]. First, a program embedding space is learned in an unsupervised manner, and then the most suitable program is searched in the embedding space with a given reward.

In our approach, we also use a reward mechanism for generating candidate programs. However, we extend this by adopting the concept from SED [19] to train an evaluator that assesses the candidate programs for further repair. While both EVAPS and SED incorporate environmental observation to train the neural program debugger, there are notable distinctions between the two: i) The execution feedback of SED relies on a global perspective, which is only available in some special cases. Conversely, EVAPS embraces partial observation, which is more achievable in real-world scenarios. ii) SED treats the program as a whole for training, while EV APS pays more attention to the execution context. iii) EV APS establishes a connection between the program execution context and the partial observable environment. Distinguishing itself from SED, EVAPS places greater emphasis on the partially observable environment in which the robot operates and the local context of the program.

We believe that this approach enables the network to concentrate more precisely on the actual scope of the impact of the program.

Program Repair. Program repair has also been a widely researched topic. Most researchers pay attention to the grammatical features [30, 29, 45] and semantic information [52, 54] of programs. Rolim and Hua [39, 20] employ abstract syntax trees to derive effective outputs, while Tfix [4] treats programs as text and trains a general Transformer model for repairs. While these approaches have achieved breakthroughs in different problem domains, they often overlook the semantic aspects of programs. Wang et al. address this by incorporating the learning of semantic embeddings from program execution trajectories [52], while Ye et al. enhance neural networks using a loss function based on program compilation and execution information [54]. Our work takes both semantic and grammatical information into account. Our evaluation network, which is based on the execution information of the program in the environment and the program's grammar domain-specific language, can effectively accomplish program repairs.

Regarding traditional program repair approaches, we are aware of a branch of rich works in the program repair area, which can be classified into three categories: i) Search-based. This type of approach considers program repair as a search problem, exploring the space of all possible program candidates to identify one that satisfies the given weak specification, i.e., test cases. One of the prior works in this area is GenProg [31]. ii) Semantic-based. This type of approach extracts semantic information from the program under repair (typically represented as path constraints) and then generates patches by solving those constraints. A well-known work is SemFix [35]. iii) Learningbased. This type of approach leverages a number of patches generated by developers to learn a model that repairs programs. An earlier work is Prophet [33]. Those approaches depend on high-quality test suites to validate patch candidates, which are unavailable in the setting our approach is targeted at, i.e., VizDoom.

6 Conclusion

This paper introduces EVAPS, a novel approach to augment the applicability of neural program synthesis by integrating partial environmental observations. By utilizing both the environmental context leveraging module and the code symbol alignment module, the ability to rectify semantically erroneous program segments and generalize across various tasks is substantially enhanced. Furthermore, we verify that EVAPS exhibits greater resilience when confronted with noise and can adeptly capture subtle features even under more intricate tasks. The proposed approach presents a promising direction for robot program synthesis by incorporating partial environmental observations. The ability of the framework to rectify semantically erroneous program segments, generalize across tasks, and maintain robustness in the presence of noise showcases its potential for practical applications in robot program synthesis.

Acknowledgments and Disclosure of Funding

This work was supported by Shanghai Municipal Science and Technology Major Project (No.2021SHZDZX0103).

References

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 , 2021.

Maria Bakhanova and Ilya Makarov. Deep reinforcement learning in vizdoom via dqn and actor-critic agents. In Proceedings of IWANN , 2021.

Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. Deepcoder: Learning to write programs. In Proceedings of ICLR , 2017.

Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin Vechev. Tfix: Learning to fix coding errors with a text-to-text transformer. In Proceedings of ICML , 2021.

Matko Bošnjak, Tim Rocktäschel, Jason Naradowsky, and Sebastian Riedel. Programming with a differentiable forth interpreter. In Proceedings of ICML , 2017.

Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks? In Proceedings of ICLR , 2022.

Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging grammar and reinforcement learning for neural program synthesis. In Proceedings of ICLR , 2018.

Rudy R Bunel, Alban Desmaison, Pawan K Mudigonda, Pushmeet Kohli, and Philip Torr. Adaptive neural compilation. In Proceedings of NeurIPS , 2016.

Xinyun Chen, Chang Liu, and Dawn Song. Execution-guided neural program synthesis. In Proceedings of ICLR , 2019.

Xinyun Chen, Dawn Song, and Yuandong Tian. Latent execution for neural program synthesis beyond domain-specific languages. In Proceedings of NeurIPS , 2021.

Raphaël Dang-Nhu. Plans: Neuro-symbolic program learning from videos. In Proceedings of NeurIPS , 2020.

Jacob Devlin, Rudy R Bunel, Rishabh Singh, Matthew Hausknecht, and Pushmeet Kohli. Neural program meta-induction. In Proceedings of NeurIPS , 2017.

Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. In Proceedings of ICML , 2017.

Xuguang Duan, Qi Wu, Chuang Gan, Yiwei Zhang, Wenbing Huang, Anton Van Den Hengel, and Wenwu Zhu. Watch, reason and code: Learning to represent videos using program. In Proceedings of ACM MM , 2019.

Majed El Helou and Sabine Süsstrunk. Blind universal bayesian image denoising with gaussian noise level learning. IEEE Transactions on Image Processing , 2020.

Kevin Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. Write, execute, assess: Program synthesis with a repl. In Proceedings of NeurIPS , 2019.

Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Josh Tenenbaum. Learning to infer graphics programs from hand-drawn images. In Proceedings of NeurIPS , 2018.

Abhay Garg, Anand Sriraman, Kunal Pagarey, and Shirish Karande. Are transformers all that karel needs? In Proceedings of NeurIPS Workshop , 2021.

Kavi Gupta, Peter Ebert Christensen, Xinyun Chen, and Dawn Song. Synthesize, execute and debug: Learning to repair for neural program synthesis. In Proceedings of NeurIPS , 2020.

Jinru Hua, Mengshi Zhang, Kaiyuan Wang, and Sarfraz Khurshid. Towards practical program repair with on-demand candidate generation. In Proceedings of ICSE , 2018.

Justin Huang, Dieter Fox, and Maya Cakmak. Synthesizing robot manipulation programs from a single observed human demonstration. In Proceedings of IROS , 2019.

Jeevana Priya Inala, Chenglong Wang, Mei Yang, Andres Codas, Mark Encarnación, Shuvendu Lahiri, Madanlal Musuvathi, and Jianfeng Gao. Fault-aware neural code rankers. In Proceedings of NeurIPS , 2022.

Naman Jain, Skanda Vaidyanath, Arun Iyer, Nagarajan Natarajan, Suresh Parthasarathy, Sriram Rajamani, and Rahul Sharma. Jigsaw: Large language models meet program synthesis. In Proceedings of ICSE , 2022.

Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Ja´ skowski. Vizdoom: A doom-based ai research platform for visual reinforcement learning. In Proceedings of CIG , 2016.

Adil Khan, Muhammad Naeem, Muhammad Zubair Asghar, Aziz Ud Din, and Atif Khan. Playing first-person shooter games with machine learning techniques and methods using the vizdoom game-ai research platform. Entertainment Computing , 2020.

Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy S Liang. Spoc: Search-based pseudocode to code. In Proceedings of NeurIPS , 2019.

Miguel Lázaro-Gredilla, Dianhuan Lin, J Swaroop Guntupalli, and Dileep George. Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs. Science Robotics , 2019.

Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Proceedings of NeurIPS , 2022.

Xuan-Bach D Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. S3: syntaxand semantic-guided repair synthesis via programming by examples. In Proceedings of ESEC , 2017.

Claire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, and Westley Weimer. A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. In Proceedings of ICSE , 2012.

Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. Genprog: A generic method for automatic software repair. Ieee transactions on software engineering , 2011.

Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomáš Koˇ cisk` y, Andrew Senior, Fumin Wang, and Phil Blunsom. Latent predictor networks for code generation. In Proceedings of ACL , 2016.

Fan Long and Martin Rinard. Automatic patch generation by learning correct code. In Proceedings of POPL , 2016.

Zohar Manna and Richard J Waldinger. Toward automatic program synthesis. Communications of the ACM , 1971.

Hoang Duong Thien Nguyen, Dawei Qi, Abhik Roychoudhury, and Satish Chandra. Semfix: Program repair via semantic analysis. In Proceedings of ICSE , 2013.

Maxwell Nye, Luke Hewitt, Joshua Tenenbaum, and Armando Solar-Lezama. Learning to infer program sketches. In Proceedings of ICML , 2019.

Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. Neuro-symbolic program synthesis. In Proceedings of ICLR , 2017.

Oleksandr Polozov and Sumit Gulwani. Flashmeta: A framework for inductive program synthesis. In Proceedings of OOPSLA , 2015.

Reudismam Rolim, Gustavo Soares, Rohit Gheyi, Titus Barik, and Loris D'Antoni. Learning quick fixes from code repositories. arXiv preprint arXiv:1803.03806 , 2018.

Eric Schkufza, Rahul Sharma, and Alex Aiken. Stochastic superoptimization. ACM SIGARCH Computer Architecture News , 2013.

Kun Shao, Dongbin Zhao, Nannan Li, and Yuanheng Zhu. Learning battles in vizdoom via deep reinforcement learning. In CIG , 2018.

Lin Shao, Toki Migimatsu, Qiang Zhang, Karen Yang, and Jeannette Bohg. Concept2robot: Learning manipulation concepts from instructions and human demonstrations. The International Journal of Robotics Research , 2021.

Eui Chul Shin, Illia Polosukhin, and Dawn Song. Improving neural program synthesis with inferred execution traces. In Proceedings of NeurIPS , 2018.

Richard Shin, Neel Kant, Kavi Gupta, Chris Bender, Brandon Trabucco, Rishabh Singh, and Dawn Song. Synthetic datasets for neural program synthesis. In Proceedings of ICLR , 2019.

Richard Shin, Illia Polosukhin, and Dawn Song. Towards specification-directed program repair. In Proceedings of ICLR Workshop , 2018.

Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. Learning to combine per-example solutions for neural program synthesis. In Proceedings of NeurIPS , 2021.

Shao-Hua Sun, Hyeonwoo Noh, Sriram Somasundaram, and Joseph Lim. Neural program synthesis from diverse demonstration videos. In Proceedings of ICML , 2018.

Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J Lim. Learning to synthesize programs as interpretable and generalizable policies. In Proceedings of NeurIPS , 2021.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of NeurIPS , 2017.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In Proceedings of ICLR , 2018.

Chenglong Wang, Alvin Cheung, and Rastislav Bodik. Synthesizing highly expressive sql queries from input-output examples. In Proceedings of PLDI , 2017.

Ke Wang, Rishabh Singh, and Zhendong Su. Dynamic neural program embeddings for program repair. In Proceedings of ICLR , 2018.

Marek Wydmuch, Michał Kempka, and Wojciech Ja´ skowski. Vizdoom competitions: Playing doom from pixels. IEEE Transactions on Games , 2019.

He Ye, Matias Martinez, and Martin Monperrus. Neural program repair with execution-based backpropagation. In Proceedings of ICSE , 2022.

Haiyan Yin, Jianda Chen, Sinno Jialin Pan, and Sebastian Tschiatschek. Sequential generative exploration model for partially observable reinforcement learning. In Proceedings of AAAI , 2021.

Amit Zohar and Lior Wolf. Automatic program synthesis of long programs with a learned garbage collector. In Proceedings of NeurIPS , 2018.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the Environmental-context Validated lAtent Program Synthesis framework (EVAPS), which builds upon the SED approach by utilizing a trail-eval-repair loop to enhance program evolution and improve generalization capabilities. EVAPS leverages partial environmental observations by initially obtaining candidate programs from existing synthesizers. By executing these candidate programs and capturing the environmental context before and after each action, EVAPS models both the environmental and syntax contexts concurrently. Through iterative modifications to address semantic conflicts across program tokens, EVAPS aims to correct erroneous program fragments that do not produce the desired output.\n\nThe proposed framework was evaluated in the partially observed Vizdoom domain, involving a robot navigating a 3D world and interacting with various elements, including objects and adversaries. The experiment results demonstrate the superiority of the approach over the baselines in accurately modeling program semantic subtleties and effectively resolving potential errors.\n\nSTRENGTHS:\n+ EVAPS synthesizes robot programs operating in partially observed environments (surrounding RGBD image inputs). \n+ EVAPS uses a trail-eval-repair loop to iteratively improve program generalizability by rectifying potentially erroneous code segments.\n+ EVAPS is capable of handling noisy observations.\n\nA key novelty of EVAPS, when compared to previous program synthesis approaches in the Karel domain, is its ability to handle partial environmental observations. This capability is crucial for synthesizing robot programs, as relying on global environment information is often impractical in real-world robot navigation tasks. EVAPS consists of two essential modules: the partial observation leveraging module, which emphasizes the environmental context of specific program tokens, and the code symbol alignment module, which focuses on modeling the implicit connections between different code segments in semantic spaces. By integrating both syntax and semantic information, these modules facilitate program repair and enhance the effectiveness of EVAPS in generating accurate and meaningful robot programs.\n\nWEAKNESSES:\nThe title of the paper may appear to overstate the actual contribution made by the research.  The choice of the Vizdoom environment as a benchmark for evaluating Robotic Program Synthesis in this paper raises some concerns. The evaluation of EVAPS in the navigation-focused Vizdoom environment does appear to have limitations. While efficient navigation is undoubtedly an important robotic task, it may not fully represent the diverse range of tasks in robotics, particularly those involving object manipulation in fully observable environments. It would be valuable to explore the applicability of EVAPS in synthesizing robot programs for other more realistic environments like Miniworld and AI2-Thor, which involve long-horizon vision-based tasks. Assessing the performance of EVAPS in such contexts would provide a more comprehensive evaluation of its capabilities. Given the current state of the paper, it remains uncertain whether EVAPS is adequately equipped to handle the challenges posed by complex robotic navigation tasks. To determine the true extent of EVAPS' applicability across a broader range of robotic tasks and environments, additional research and experimentation are necessary.\n\nNevertheless, I do believe this paper contributes to the program synthesis community. The comparative evaluation of EVAPS against the baselines, when provided with partial observations, showcases its superior performance, thereby highlighting the significance of the observation leveraging module and the code symbol alignment module. However, there remains a lingering uncertainty regarding the practical implications of this demonstrated value in the context of robot control.\n\nQUESTIONS:\nEVAPS seems incremental to SED. To gain a comprehensive understanding of EVAPS' contribution, it would be beneficial to apply the framework to the Karel domain, where the baselines were evaluated. This approach would effectively demonstrate the effectiveness of the observation leveraging module and the code symbol alignment module. Is there any evidence or indication from the authors regarding the performance of EVAPS specifically in the Karel domain?\n\nLIMITATIONS:\nThe paper presents an approach for robot program synthesis. However, the evaluation falls short in providing a convincing argument on how EVAPS can be effectively applied to learn robot-control policies, as the experiments are confined to a relatively simple video game environment. Further investigation and experimentation in more complex and realistic robotic scenarios would be necessary to demonstrate the practical utility and effectiveness of EVAPS in learning robot-control policies.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper claims that global observation in robot program synthesis is not achievable, so it proposes to use partial observation. And it learns an observation embedding module and a semantic-grammatical alignment module to repair the candidate programs that can increase the accuracy and generalization of robot program synthesis.\n\nSTRENGTHS:\n1. The problem this paper focused on is important to the application of PS as the perfect and complete observable environment is often not available in the real world.\n\n2. The experiment setting and analysis are clear and comprehensive. The paper analyses the effect of multiple aspects factors in the in-perfect observation of the proposed method.\n\nWEAKNESSES:\n[W1]. The key weakness is the choice of baseline methods. As a program repair method, it’s better to compare with methods that have a fair setting rather than program generation methods. There are other program repair methods that utilize trajectory to enhance performance. Moreover, the baseline methods are too old (most before 2020).\n\n[1] Execution-guided neural program synthesis\n\n[2] Write, execute, assess: Program synthesis with a REPL\n\n[2] Improving neural program synthesis with inferred execution traces\n\n\n[W2]. Needs more explanation about the difference between global environment information and partially observable environments. \n\n[W2.1] As using pre-observation and post-observation which are related to specific program segments to enhance the information contained in the embedding is a general trick. The PS methods which take global environment information as input can also learn such representation to enhance accuracy and generalization. What is the key connection between the proposed method and partially observable environments? \n\n[W2.2] Needs some insightful evaluation in the results section to demonstrate the relation between the main motivation (partial observable environment) and the proposed architecture.\n\n[W3]. There are some typos, like line 203, citation 41 is not the baseline\n\nQUESTIONS:\nPlease see weaknesses.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper proposes EVAPS to enhance robotic program synthesis by integrating parietal environmental observations. Specifically, EVAPS utilizes both the environmental context leveraging module and the code symbol alignment module to improve its ability to rectify semantically erroneous program segments and generalize across various tasks. Comprehensive experiments on the partially observed Vizdoom benchmark demonstrate its superior performance over other baselines across various tasks. Overall, this is a well-written paper and its extensive experiments and ablation studies verify the effectiveness of the proposed method. I would be leaning to accept this paper.

STRENGTHS:
* The proposed method is technically sound and well motivated.
* This paper is well written and well structured. It provides clear formulations and an overview figure to well explain the method.
* The experimental evaluation is rather comprehensive and provides convincing results to demonstrate the effectiveness of the proposed method.

WEAKNESSES:
I did not identify any weaknesses for this paper as I am not familiar with the task of robotic program synthesis.

QUESTIONS:
Can you shed some light on the limitations of the proposed method?

LIMITATIONS:
This paper just discussed the limitations very lightly. I would like to see more discussion on its limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Environmental-context Validated lAtent Program Synthesis framework (EVAPS). a program synthesis model that generates executable program for robotic programming, evaluated in the vizdoom environment. It initially obtains candidate programs using other available synthesizers, then performs program repair by executing the candidate program and collect resulted partial environmental observations. It outperfoms a range of prior works for viszoom program synthesis, and demonstrated robustness again observation noises and task complexity.\n\nSTRENGTHS:\n- using the aid of partially observed environments for program synthesis and repair is novel and reasonable\n- the proposed framework is sound, and the design choices for utilizing observations, and using a graph structure to aggregate environmental and syntactic information flow is convincing\n- extensive experiments\n\nWEAKNESSES:\n- the writing of the paper can be improved, by reducing repetitive and inconsistent adjectives\n- do the baselines use program repair? Does any of them rely on environmental observations?\n- the assumption of executing candidate programs makes more sense when a privileged simulation environment is available, and when real-time control is not needed. Can the author provide further justification regarding this? How would you expect to transfer to real world?\n- the author claim that EVAPS shows is more robust agains noise in the conclusion, but there's no comparison with baseline  on this matter in the experiment section.\n\nQUESTIONS:\nsee weeknesses.\n\nLIMITATIONS:\nThe major limitation i see in this framework is the assumption of aqcuiring partial observations and executing program candidates in the environment. These assumption is not valid in real-world real-time control. More justifications on this would be very helpful.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an approach for program synthesis in robotic domains where the environment context can provide valuable insight for what the correct program should be. The approach takes candidate programs, executes them to get a trace of observations, and then passes the program and trace through a combination of local and global feature extraction with neural networks. The output is a refined program. The approach is trained and tested on randomly generated Vizdoom programs and is substantially better than the alternatives. They also have an ablation study for the contribution of the two types of feature extraction and a study of the approach's noise tolerance.\n\nSTRENGTHS:\nThe results are good. The description of the approach is very clear and easy to understand. Some details are missing but overall it's clear. Vizdoom seems like a popular domain and their approach performs much better than the tested alternatives. \n\nThe introduction does a great job of describing the intuition behind incorporating observation feedback into the program proposal.\n\nI don't have a lot to say about the paper. It is mostly a standard type of paper of applying a new, well-thought out approach to an existing domain. The approach seems sound and the results are good. The main insight is incorporating the environmental feedback to improve the program, which is similar to execution-guided synthesis.\n\nWEAKNESSES:\nThe paper doesn't have any glaring weaknesses. As usual, it could be strengthened by applying the approach to another benchmark. \n\nMy main concern is that the gist of the idea seems very similar to work on execution-guided synthesis. But, the approach is shown to be much better than SED on the vizdoom benchmark, so there must be something more here.\n\n# Addressing rebuttal\nI have read the authors rebuttal. In particular, they satisfactorily address my main concern about similarity to other work, which I have no further concerns about.\n\nQUESTIONS:\nCan SOTA approaches on Karel be applied to Vizdoom? If so, how do they compare? It seems like the intuitive reason this approach works is similar to the idea behind execution-guided synthesis work applied to Karel, so that could be a good comparison to include.\n\nWhere do the program candidates come from before refinement? I don't recall the paper explaining this. This is a big confusion of mine.\n\nI would prefer to have the description of the Vizdoom domain in the evaluation section rather than the preliminary. A figure showing an example task would be helpful too.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}