Original: G RAPHIX : A P RE - TRAINED G RAPH E DIT M ODEL FOR A UTOMATED P ROGRAM R EPAIR
Revised: G RAPHIX : A P RE - TRAINED G RAPH E DIT M ODEL FOR A UTOMATED P ROGRAM R EPAIR

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present G RAPHIX , a pre-trained graph edit model for automatically detecting and ﬁxing bugs and code quality issues in Java programs.
Revised: We present G RAPHIX , a pre-trained graph edit model for automatically detecting and ﬁxing bugs and code quality issues in Java programs.

Original: Unlike sequence-tosequence models, G RAPHIX leverages the abstract syntax structure of code and represents the code using a multi-head graph encoder.
Revised: Unlike sequence-tosequence models, G RAPHIX leverages the abstract syntax structure of code and represents the code using a multi-head graph encoder.

Original: Along with an autoregressive tree decoder, the model learns to perform graph edit actions for automated program repair.
Revised: Along with an autoregressive tree decoder, the model learns to perform graph edit actions for automated program repair.

Original: We devise a novel pre-training strategy for G RAPHIX , namely deleted sub-tree reconstruction , to enrich the model with implicit knowledge of program structures from unlabeled source code.
Revised: We devise a novel pre-training strategy for G RAPHIX , namely deleted sub-tree reconstruction , to enrich the model with implicit knowledge of program structures from unlabeled source code.

Original: The pre-training objective is made consistent with the bug ﬁxing task to facilitate the downstream learning.
Revised: The pre-training objective is made consistent with the bug ﬁxing task to facilitate the downstream learning.

Original: We evaluate
Revised: We evaluate

Original: G RAPHIX on the Patches in The Wild Java benchmark, using both abstract and concrete code.
Revised: G RAPHIX on the Patches in The Wild Java benchmark, using both abstract and concrete code.

Original: Experimental results show that G RAPHIX signiﬁcantly outperforms a wide range of baselines including CodeBERT and BART and is as competitive as state-of-the-art pre-trained Transformer models despite using fewer parameters.
Revised: Experimental results show that G RAPHIX signiﬁcantly outperforms a wide range of baselines including CodeBERT and BART and is as competitive as state-of-the-art pre-trained Transformer models despite using fewer parameters.

Original: Further analysis demonstrates strong inductive biases of G RAPHIX in learning meaningful structural and semantic code patterns, both in abstract and concrete source code.
Revised: Further analysis demonstrates strong inductive biases of G RAPHIX in learning meaningful structural and semantic code patterns, both in abstract and concrete source code.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Detecting bugs and code quality issues in programs and ﬁxing them is an important task in software development.
Revised: Detecting bugs and code quality issues in programs and ﬁxing them is an important task in software development.

Original: Oftentimes, this two-step process is manual and labor-intensive.
Revised: Oftentimes, this two-step process is manual and labor-intensive.

Original: In order to ease the burden on developers and reduce the development cost, several automated tools are developed and integrated with development environments (IDEs) and/or code review systems.
Revised: In order to ease the burden on developers and reduce the development cost, several automated tools are developed and integrated with development environments (IDEs) and/or code review systems.

Original: There exist many program analysis techniques (Sadowski et al., 2015) that use hand-written rules to provide linter-style recommendations while others (Paletov et al., 2018) are able to detect speciﬁc bug types (e.g., crypto
Revised: There exist many program analysis techniques (Sadowski et al., 2015) that use hand-written rules to provide linter-style recommendations while others (Paletov et al., 2018) are able to detect speciﬁc bug types (e.g., crypto

Original: API uses) but leave them to the developer to ﬁx.
Revised: API uses) but leave them to the developer to ﬁx.

Original: In the recent years, advances in deep learning have appealed researchers to move to data-driven approaches (Vasic et al., 2018) that learn to detect and ﬁx bugs from data.
Revised: In the recent years, advances in deep learning have appealed researchers to move to data-driven approaches (Vasic et al., 2018) that learn to detect and ﬁx bugs from data.

Original: Henceforth, we refer to the problem of automatically detecting and ﬁxing bugs as automated program repair (Goues et al., 2019).
Revised: Henceforth, we refer to the problem of automatically detecting and ﬁxing bugs as automated program repair (Goues et al., 2019).

Original: One main challenge with using machine learning for automated program repair is the lack of large human-labeled datasets.
Revised: One main challenge with using machine learning for automated program repair is the lack of large human-labeled datasets.

Original: Several works (Tufano et al., 2019b;a) propose heuristics to extract bug-ﬁx data from code changes.
Revised: Several works (Tufano et al., 2019b;a) propose heuristics to extract bug-ﬁx data from code changes.

Original: Code changes are represented as (code before, code after) pairs mostly at the method level, and typically pertain to feature enhancements, bug ﬁxes and code refactoring.
Revised: Code changes are represented as (code before, code after) pairs mostly at the method level, and typically pertain to feature enhancements, bug ﬁxes and code refactoring.

Original: Code changes relating to bug ﬁxes are identiﬁed through the presence of keywords such as “bug” in commit messages, through the size of changes or both.
Revised: Code changes relating to bug ﬁxes are identiﬁed through the presence of keywords such as “bug” in commit messages, through the size of changes or both.

Original: An advantage of using code change data is that there are plenty of code changes available in source code repositories.
Revised: An advantage of using code change data is that there are plenty of code changes available in source code repositories.

Original: Based on code change data, the learning of bug ﬁxes has been taken in three main directions.
Revised: Based on code change data, the learning of bug ﬁxes has been taken in three main directions.

Original: The ﬁrst line of work regards the bug-ﬁx learning problem as sequence-based code generation and harnesses the power of sequence-to-sequence models with RNN (Hata et al., 2018;
Revised: The ﬁrst line of work regards the bug-ﬁx learning problem as sequence-based code generation and harnesses the power of sequence-to-sequence models with RNN (Hata et al., 2018;

Original: Chen et al., 2019; Tufano et al., 2019b;a) and more recently with pre-trained Transformer models (Feng et al., 2020; Guo et al., 2021;
Revised: Chen et al., 2019; Tufano et al., 2019b;a) and more recently with pre-trained Transformer models (Feng et al., 2020; Guo et al., 2021;

Original: Ahmad et al., 2021; Wang et al., 2021; Berabi et al., 2021).
Revised: Ahmad et al., 2021; Wang et al., 2021; Berabi et al., 2021).

Original: Secondly, instead of generating the ﬁxed code from scratch, Zhao et al.
Revised: Secondly, instead of generating the ﬁxed code from scratch, Zhao et al.

Original: ; Chakraborty et al.
Revised: ; Chakraborty et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: Panthaplackel et al.
Revised: Panthaplackel et al.

Original: (2021) extend sequence models and propose to generate edits on the buggy code.
Revised: (2021) extend sequence models and propose to generate edits on the buggy code.

Original: Lastly,
Revised: Lastly,

Original: Yin et al.
Revised: Yin et al.

Original: ; Tarlow et al.
Revised: ; Tarlow et al.

Original: ; Dinella et al.
Revised: ; Dinella et al.

Original: ; Yao et al.
Revised: ; Yao et al.

Original: (2021) deviate from the token-based representation of code and leverage the hierarchical structure in the abstract syntax tree
Revised: (2021) deviate from the token-based representation of code and leverage the hierarchical structure in the abstract syntax tree

Original: (AST).
Revised: (AST).

Original: The high level idea is to learn a graph edit model to iteratively transform the AST of the buggy code into the AST of the ﬁxed code.
Revised: The high level idea is to learn a graph edit model to iteratively transform the AST of the buggy code into the AST of the ﬁxed code.

Original: The learning is supervised by tree differencing edits via a cross-entropy loss.
Revised: The learning is supervised by tree differencing edits via a cross-entropy loss.

Original: Despite the beneﬁts of more precise bug localization, shorter tree edits and natural graph representations of programs, existing graph-based models have been less competitive than Transformer-based models thus far.
Revised: Despite the beneﬁts of more precise bug localization, shorter tree edits and natural graph representations of programs, existing graph-based models have been less competitive than Transformer-based models thus far.

Original: In this paper, we pursue the direction of representing explicit program structures with graphs and make contributions toward source code understanding using machine learning.
Revised: In this paper, we pursue the direction of representing explicit program structures with graphs and make contributions toward source code understanding using machine learning.

Original: Speciﬁcally, we present
Revised: Speciﬁcally, we present

Original: G RAPHIX , a pre-trained graph edit model for automated program repair.
Revised: G RAPHIX , a pre-trained graph edit model for automated program repair.

Original: Inspired by H OPPITY (Dinella et al., 2020), we design G RAPHIX as a sequential decision process with a multi-head graph encoder and an autoregressive tree decoder.
Revised: Inspired by H OPPITY (Dinella et al., 2020), we design G RAPHIX as a sequential decision process with a multi-head graph encoder and an autoregressive tree decoder.

Original: Conditioned on the graph state and the edit history at each step, the decoder iteratively makes primitive actions (e.g., adding a node) on the graph corresponding to the input AST.
Revised: Conditioned on the graph state and the edit history at each step, the decoder iteratively makes primitive actions (e.g., adding a node) on the graph corresponding to the input AST.

Original: Unlike H OPPITY (Dinella et al., 2020), we enhance the encoder with multiple graph heads to capture diverse aspects of hierarchical code structures.
Revised: Unlike H OPPITY (Dinella et al., 2020), we enhance the encoder with multiple graph heads to capture diverse aspects of hierarchical code structures.

Original: In addition, we guide the decoder with an underlying Abstract Syntax Description Language (ASDL).
Revised: In addition, we guide the decoder with an underlying Abstract Syntax Description Language (ASDL).

Original: Owing to the syntax language, the graph structure and semantic edits, G RAPHIX exhibits strong inductive biases in learning generic ﬁxing and refactoring patterns from code changes.
Revised: Owing to the syntax language, the graph structure and semantic edits, G RAPHIX exhibits strong inductive biases in learning generic ﬁxing and refactoring patterns from code changes.

Original: Moreover, we devise a novel pre-training strategy, namely deleted sub-tree reconstruction , that enables G RAPHIX to learn implicit program structures from unlabeled data.
Revised: Moreover, we devise a novel pre-training strategy, namely deleted sub-tree reconstruction , that enables G RAPHIX to learn implicit program structures from unlabeled data.

Original: Here, the model is pre-trained to reconstruct a randomly deleted sub-tree given other code context, using the same cross-entropy loss as the program repair task.
Revised: Here, the model is pre-trained to reconstruct a randomly deleted sub-tree given other code context, using the same cross-entropy loss as the program repair task.

Original: Our pre-training task generalizes the masked language model and denoising objectives (Devlin et al., 2019; Lewis et al., 2020) in sequence models to the AST representation of source code.
Revised: Our pre-training task generalizes the masked language model and denoising objectives (Devlin et al., 2019; Lewis et al., 2020) in sequence models to the AST representation of source code.

Original: Our idea shares the conceptual principle with generative models of code (Li et al., 2018; Brockschmidt et al., 2018) for expression generation and the structural language model (Alon et al., 2020) for any-code completion.
Revised: Our idea shares the conceptual principle with generative models of code (Li et al., 2018; Brockschmidt et al., 2018) for expression generation and the structural language model (Alon et al., 2020) for any-code completion.

Original: We evaluate G
Revised: We evaluate G

Original: RAPHIX on both concrete and abstract versions of the
Revised: RAPHIX on both concrete and abstract versions of the

Original: Patches in The Wild
Revised: Patches in The Wild

Original: Java benchmark
Revised: Java benchmark

Original: (Tufano et al.,
Revised: (Tufano et al.,

Original: 2019b
Revised: 2019b

Original: We show that
Revised: We show that

Original: G
Revised: G

Original: RAPHIX signiﬁcantly outperforms edit-based and Transformer models, and the performance is as competitive as recent state-of-the-art pre-trained baselines despite using fewer parameters.
Revised: RAPHIX signiﬁcantly outperforms edit-based and Transformer models, and the performance is as competitive as recent state-of-the-art pre-trained baselines despite using fewer parameters.

Original: Our in-depth analysis of the generated patterns demonstrates the ability of G RAPHIX to learn a wide range of meaningful and generic bug ﬁxing and code refactoring patterns.
Revised: Our in-depth analysis of the generated patterns demonstrates the ability of G RAPHIX to learn a wide range of meaningful and generic bug ﬁxing and code refactoring patterns.

Original: Compared to H OPPITY , which merely learns short edits such as modifying access keywords ( private to public ), adding break , or removing redundant type arguments, etc., G RAPHIX is able to handle longer, more complicated structural and semantic edits (up tograph edits).
Revised: Compared to H OPPITY , which merely learns short edits such as modifying access keywords ( private to public ), adding break , or removing redundant type arguments, etc., G RAPHIX is able to handle longer, more complicated structural and semantic edits (up tograph edits).

Original: We showcase a variety of bug-ﬁx examples such as ﬁxing off-by-one errors, possible null pointer exceptions
Revised: We showcase a variety of bug-ﬁx examples such as ﬁxing off-by-one errors, possible null pointer exceptions

Original: , etc.
Revised: , etc.

Original: in our analysis.
Revised: in our analysis.

Original: In summary, we make the following main contributions in this work:
Revised: In summary, we make the following main contributions in this work:

Original: G RAPHIX is a medium-scale graph edit model that can be pre-trained with the deleted sub-tree reconstruction objective for automated program repair.
Revised: G RAPHIX is a medium-scale graph edit model that can be pre-trained with the deleted sub-tree reconstruction objective for automated program repair.

Original: G RAPHIX signiﬁcantly outperforms the edit-based models and is able to achieve high topexact match accuracy on the Patches in the Wild benchmark even without pre-training.
Revised: G RAPHIX signiﬁcantly outperforms the edit-based models and is able to achieve high topexact match accuracy on the Patches in the Wild benchmark even without pre-training.

Original: Using an explicit graph representation on the abstract syntax tree, our model is as competitive as large-scale Transformer models.
Revised: Using an explicit graph representation on the abstract syntax tree, our model is as competitive as large-scale Transformer models.

Original: Unlike previous work, which only focused on the abstract version of the benchmark, we evaluate G RAPHIX on both abstract and concrete code and show the effectiveness of our model in both settings.
Revised: Unlike previous work, which only focused on the abstract version of the benchmark, we evaluate G RAPHIX on both abstract and concrete code and show the effectiveness of our model in both settings.

Original: This also suggests that the code abstraction may not be necessary.
Revised: This also suggests that the code abstraction may not be necessary.

Original: Our in-depth analysis of the generated ﬁxes demonstrates that G RAPHIX exhibits strong inductive biases in learning to both structural and semantic code patterns, leading many meaningful bug ﬁxes.
Revised: Our in-depth analysis of the generated ﬁxes demonstrates that G RAPHIX exhibits strong inductive biases in learning to both structural and semantic code patterns, leading many meaningful bug ﬁxes.

Original: Despite our focus on Java, our model and the pre-training strategy can be extended further to work with multiple languages for universal source code understanding using a language-agnostic abstract language and parser such as tree-sitter 1 .
Revised: Despite our focus on Java, our model and the pre-training strategy can be extended further to work with multiple languages for universal source code understanding using a language-agnostic abstract language and parser such as tree-sitter 1 .

Original: In addition, the syntax tree structure is highly amenable to program dependencies.
Revised: In addition, the syntax tree structure is highly amenable to program dependencies.

Original: We hope our work will inspire the community to advance the area further by incorporating more program dependencies, designing efﬁcient graph learning algorithms with large model architectures and datasets.
Revised: We hope our work will inspire the community to advance the area further by incorporating more program dependencies, designing efﬁcient graph learning algorithms with large model architectures and datasets.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Deep learning for program repair Advances in machine learning in the past decade have drastically changed the landscape of research for automated program repair.
Revised: Deep learning for program repair Advances in machine learning in the past decade have drastically changed the landscape of research for automated program repair.

Original: Inspired by Neural Machine Translation, Hata et al.
Revised: Inspired by Neural Machine Translation, Hata et al.

Original: ; Chen et al.
Revised: ; Chen et al.

Original: ; Tufano et al.
Revised: ; Tufano et al.

Original: (2019b;a) regard code as a sequence of tokens and use sequence-to-sequence models to “translate” the buggy code into the ﬁxed code.
Revised: (2019b;a) regard code as a sequence of tokens and use sequence-to-sequence models to “translate” the buggy code into the ﬁxed code.

Original: More recently, pre-trained Transformer-based models such as CodeBERT (Feng et al., 2020),
Revised: More recently, pre-trained Transformer-based models such as CodeBERT (Feng et al., 2020),

Original: GraphCodeBERT (Guo et al., 2021), PL-BART (Ahmad et al., 2021) and most recently CodeT5 (Wang et al., 2021) achieve state-of-the-art results on “Patches in the Wild” Java benchmark (Tufano et al., 2019b) for program repair.
Revised: GraphCodeBERT (Guo et al., 2021), PL-BART (Ahmad et al., 2021) and most recently CodeT5 (Wang et al., 2021) achieve state-of-the-art results on “Patches in the Wild” Java benchmark (Tufano et al., 2019b) for program repair.

Original: A series of edit-based approaches model the code edits made on the buggy code as opposed to the “translation”-based counterparts.
Revised: A series of edit-based approaches model the code edits made on the buggy code as opposed to the “translation”-based counterparts.

Original: Zhao et al.
Revised: Zhao et al.

Original: ; Chakraborty et al.
Revised: ; Chakraborty et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: Panthaplackel et al.
Revised: Panthaplackel et al.

Original: (2021) extend sequence-to-sequence RNN models to predict the sequence of edits given the buggy code.
Revised: (2021) extend sequence-to-sequence RNN models to predict the sequence of edits given the buggy code.

Original: Unlike these works, we build G RAPHIX on the inherent abstract syntax structure of code instead of its token-based representation.
Revised: Unlike these works, we build G RAPHIX on the inherent abstract syntax structure of code instead of its token-based representation.

Original: Graph neural networks GNNs such as graph convolutional networks (Kipf & Welling, 2016), graph attention networks (Veliˇckovi´c et al., 2018) provide a set of powerful tools for learning graph-structured data.
Revised: Graph neural networks GNNs such as graph convolutional networks (Kipf & Welling, 2016), graph attention networks (Veliˇckovi´c et al., 2018) provide a set of powerful tools for learning graph-structured data.

Original: Source code with rich graph structures in abstract syntax trees and program dependency graphs naturally ﬁts to the GNN toolbox.
Revised: Source code with rich graph structures in abstract syntax trees and program dependency graphs naturally ﬁts to the GNN toolbox.

Original: Yin et al.
Revised: Yin et al.

Original: ; Allamanis et al.
Revised: ; Allamanis et al.

Original: Hellendoorn et al.
Revised: Hellendoorn et al.

Original: ; Wang et al.
Revised: ; Wang et al.

Original: (2020) augment the AST with additional data ﬂow edges between variables and use GNNs for tasks such as variable name prediction.
Revised: (2020) augment the AST with additional data ﬂow edges between variables and use GNNs for tasks such as variable name prediction.

Original: Yin et al.
Revised: Yin et al.

Original: ; Tarlow et al.
Revised: ; Tarlow et al.

Original: ; Dinella et al.
Revised: ; Dinella et al.

Original: ; Yasunaga & Liang (2020)
Revised: ; Yasunaga & Liang (2020)

Original: Yao et al.
Revised: Yao et al.

Original: ) use graph models based on ASTs for program repair and editing tasks in C, C and Javascript, separately.
Revised: ) use graph models based on ASTs for program repair and editing tasks in C, C and Javascript, separately.

Original: Particularly
Revised: Particularly

Original: Brockschmidt et al.
Revised: Brockschmidt et al.

Original: ; Dinella et al
Revised: ; Dinella et al

Original: Yao et al
Revised: Yao et al

Original: 2021) use graph representations with treebased decoders for AST generation by editing the partial AST one node at a time.
Revised: 2021) use graph representations with treebased decoders for AST generation by editing the partial AST one node at a time.

Original: Brockschmidt et al
Revised: Brockschmidt et al

Original: Yao et al.
Revised: Yao et al.

Original: (2021) further equip the decoder with an Abstract Syntax Description Language (ASDL) to ensure the syntactic correctness of the generated code.
Revised: (2021) further equip the decoder with an Abstract Syntax Description Language (ASDL) to ensure the syntactic correctness of the generated code.

Original: Our work advances H OPPITY (Dinella et al., 2020) and (Yao et al., 2021) with multi-head graph encoding and pre-training and demonstrates the beneﬁts of program structures on learning bug-ﬁx patterns in Java code.
Revised: Our work advances H OPPITY (Dinella et al., 2020) and (Yao et al., 2021) with multi-head graph encoding and pre-training and demonstrates the beneﬁts of program structures on learning bug-ﬁx patterns in Java code.

Original: Pre-training Transformer (Vaswani et al., 2017) and large-scale pre-training
Revised: Pre-training Transformer (Vaswani et al., 2017) and large-scale pre-training

Original: Radford et al., 2018;
Revised: Radford et al., 2018;

Original: Devlin et al., 2019; Lewis et al., 2020) have gained much traction in the broader ﬁeld of natural language processing (NLP).
Revised: Devlin et al., 2019; Lewis et al., 2020) have gained much traction in the broader ﬁeld of natural language processing (NLP).

Original: Sequence-based pre-training with Transformer has also proven to be effective in learning source code representations.
Revised: Sequence-based pre-training with Transformer has also proven to be effective in learning source code representations.

Original: In contrast, techniques for pre-training with graphs are less well-developed, especially for learning graph representations of source code.
Revised: In contrast, techniques for pre-training with graphs are less well-developed, especially for learning graph representations of source code.

Original: Existing work on graph pre
Revised: Existing work on graph pre

Original: -training are mainly based on attribute and edge prediction (see (Hu et al., 2020) and references therein).
Revised: -training are mainly based on attribute and edge prediction (see (Hu et al., 2020) and references therein).

Original: Distinct from these works, our pre-training task is conceptually related to structural generative modeling of source code (Brockschmidt et al., 2018;
Revised: Distinct from these works, our pre-training task is conceptually related to structural generative modeling of source code (Brockschmidt et al., 2018;

Original: Alon et al., 2020) for tree generation, just as pre-training techniques in BERT and BART derived from language modeling and denoising for sequence generation in NLP.
Revised: Alon et al., 2020) for tree generation, just as pre-training techniques in BERT and BART derived from language modeling and denoising for sequence generation in NLP.

Original: However, we focus on pre-training G RAPHIX to learn hierarchical tree structures of code for program repair rather than on generating code.
Revised: However, we focus on pre-training G RAPHIX to learn hierarchical tree structures of code for program repair rather than on generating code.

Original:  M OTIVATING E XAMPLES
Revised:  M OTIVATING E XAMPLES

Original: We begin by motivating the graph edit approach with two examples of code changes.
Revised: We begin by motivating the graph edit approach with two examples of code changes.

Original: Listingdisplays two test samples from the Patches in the Wild benchmark, which are real-world Java code extracted from GitHub repositories.
Revised: Listingdisplays two test samples from the Patches in the Wild benchmark, which are real-world Java code extracted from GitHub repositories.

Original: In both examples, we can see that the code changes are much smaller compared to the method bodies.
Revised: In both examples, we can see that the code changes are much smaller compared to the method bodies.

Original: The ﬁx of the ﬁrst bug is straightforward with one edit that replaces || inside the condition with && to prevent the program from crashing.
Revised: The ﬁx of the ﬁrst bug is straightforward with one edit that replaces || inside the condition with && to prevent the program from crashing.

Original: For the second bug, the change is more structural and hierarchical than merely replacing the token Object with void and removing return keyword.
Revised: For the second bug, the change is more structural and hierarchical than merely replacing the token Object with void and removing return keyword.

Original: Rather, the corresponding ﬁx can be summarized by a sequence of four tree edits: (i) replace the return type
Revised: Rather, the corresponding ﬁx can be summarized by a sequence of four tree edits: (i) replace the return type

Original: Object with void type, (ii) remove the ReturnStmt , (iii) add an ExpressionStmt at the same position, (iv) copy the AssignExpr in the original code over the child of the newly added expression statement.
Revised: Object with void type, (ii) remove the ReturnStmt , (iii) add an ExpressionStmt at the same position, (iv) copy the AssignExpr in the original code over the child of the newly added expression statement.

Original: More generally, each edit is associated with a speciﬁc node and consists of several primitive actions such as node deletion, addition or copy and type or value prediction.
Revised: More generally, each edit is associated with a speciﬁc node and consists of several primitive actions such as node deletion, addition or copy and type or value prediction.

Original: These examples illustrate the beneﬁts of the edit-based approach over the pure translation-based approach in three aspects.
Revised: These examples illustrate the beneﬁts of the edit-based approach over the pure translation-based approach in three aspects.

Original: First, by modeling the edits, the model maintains the global code context public static List<String> getDomains(String consumerKey) throws APIManagementException { String list =
Revised: First, by modeling the edits, the model maintains the global code context public static List<String> getDomains(String consumerKey) throws APIManagementException { String list =

Original: ApiMgtDAO.getAuthorizedDomainsByConsumerKey(consumerKey); if ((list !
Revised: ApiMgtDAO.getAuthorizedDomainsByConsumerKey(consumerKey); if ((list !

Original: = null )
Revised: = null )

Original: || && (!(list.isEmpty())))
Revised: || && (!(list.isEmpty())))

Original: { return Arrays.asList(list.split(",")); } return null ; }
Revised: { return Arrays.asList(list.split(",")); } return null ; }

Original: Object void setData(Object newData) { return data = newData; } instead of decoding from scratch.
Revised: Object void setData(Object newData) { return data = newData; } instead of decoding from scratch.

Original: Second, the edit sequence is often much shorter than the code sequence per se.
Revised: Second, the edit sequence is often much shorter than the code sequence per se.

Original: Finally, with the structural edits, the decoding can be guided by an underlying syntax language.
Revised: Finally, with the structural edits, the decoding can be guided by an underlying syntax language.

Original: For example, the child of ExpressionStmt must be an expression (e.g., AssignExpr in the second bug ﬁx) and must not be other production rules such as ReturnStmt .
Revised: For example, the child of ExpressionStmt must be an expression (e.g., AssignExpr in the second bug ﬁx) and must not be other production rules such as ReturnStmt .

Original: Another advantage is in terms of an end-to-end framework for bug detection and ﬁxing in the sense that such an edit model can predict whether or not a program is buggy and localize the bug more precisely.
Revised: Another advantage is in terms of an end-to-end framework for bug detection and ﬁxing in the sense that such an edit model can predict whether or not a program is buggy and localize the bug more precisely.

Original:  A PPROACH
Revised:  A PPROACH

Original: At a high level, we want to model p ( g f | θ, g b ) where g b is the buggy code and g f is the ﬁxed code.
Revised: At a high level, we want to model p ( g f | θ, g b ) where g b is the buggy code and g f is the ﬁxed code.

Original: Instead of encoding g b and generating g f from scratch, we learn a sequence of T tree-edit actions { a t } Tt =1 that transforms g b to g f as motivated in Section 3.
Revised: Instead of encoding g b and generating g f from scratch, we learn a sequence of T tree-edit actions { a t } Tt =1 that transforms g b to g f as motivated in Section 3.

Original: Indirectly, we learn the model p ( a 1: T | g b ) from a labeled dataset of samples, each with buggy code and the corresponding ground-truth edit, by minimizing a cross-entropy loss.
Revised: Indirectly, we learn the model p ( a 1: T | g b ) from a labeled dataset of samples, each with buggy code and the corresponding ground-truth edit, by minimizing a cross-entropy loss.

Original: Figure 1 illustrates the overall architecture.
Revised: Figure 1 illustrates the overall architecture.

Original: [Figure]
Revised: [Figure]

Original: One may recover the original token-based generation model when each action corresponds to adding a new token and T is the length of g b .
Revised: One may recover the original token-based generation model when each action corresponds to adding a new token and T is the length of g b .

Original: The key departure from such a model is a more hierarchical program representation that allows shorter and more structural edits between the input g b and the output g f .
Revised: The key departure from such a model is a more hierarchical program representation that allows shorter and more structural edits between the input g b and the output g f .

Original: Inspired by (Dinella et al., 2020; Yao et al., 2021) which build on syntax structures and tree differencing-based edits, we represent the buggy code as graph g b that is constructed from the program’s syntax tree and extra data ﬂow edges between certain leaf nodes.
Revised: Inspired by (Dinella et al., 2020; Yao et al., 2021) which build on syntax structures and tree differencing-based edits, we represent the buggy code as graph g b that is constructed from the program’s syntax tree and extra data ﬂow edges between certain leaf nodes.

Original: Our model is sequential in nature with a multi-head graph encoder and an autoregressive tree decoder.
Revised: Our model is sequential in nature with a multi-head graph encoder and an autoregressive tree decoder.

Original: At each time step t , the encoder encodes the current graph state g t , and the decoder computes the graph edit history s t .
Revised: At each time step t , the encoder encodes the current graph state g t , and the decoder computes the graph edit history s t .

Original: Conditioned on the current state, the decoder then predicts and performs tree edit actions.
Revised: Conditioned on the current state, the decoder then predicts and performs tree edit actions.

Original: We discuss the baseline encoder, edit operations and ASDL-guided decoder in greater details in Appendix A. Next, we elaborate more on the multi-head graph encoder and our proposed pre-training strategy.
Revised: We discuss the baseline encoder, edit operations and ASDL-guided decoder in greater details in Appendix A. Next, we elaborate more on the multi-head graph encoder and our proposed pre-training strategy.

Original: 4. M ULTI - HEAD GRAPH ENCODER
Revised: 4. M ULTI - HEAD GRAPH ENCODER

Original: Earlier work (Yin et al., 2018; Allamanis et al., 2018; Brockschmidt et al., 2018; Hellendoorn et al., 2019) has explored graph neural networks on rich graph structures of programs including syntax trees and dependency graphs for program representations.
Revised: Earlier work (Yin et al., 2018; Allamanis et al., 2018; Brockschmidt et al., 2018; Hellendoorn et al., 2019) has explored graph neural networks on rich graph structures of programs including syntax trees and dependency graphs for program representations.

Original: We follow (Dinella et al., 2020) and represent each program using its abstract syntax tree: the core graph is built upon the syntax tree with bidirectional edges between parent and child nodes, edges between adjacent leaf nodes and additional edges between value nodes to partially capture data ﬂow information (Dinella et al., 2020;
Revised: We follow (Dinella et al., 2020) and represent each program using its abstract syntax tree: the core graph is built upon the syntax tree with bidirectional edges between parent and child nodes, edges between adjacent leaf nodes and additional edges between value nodes to partially capture data ﬂow information (Dinella et al., 2020;

Original: Allamanis et al., 2018).
Revised: Allamanis et al., 2018).

Original: Then, we use a graph isomorphism network (GIN, Xu et al.
Revised: Then, we use a graph isomorphism network (GIN, Xu et al.

Original: to compute a vector representation for each node and an aggregate graph representation.
Revised: to compute a vector representation for each node and an aggregate graph representation.

Original: Motivated by the success of multi-head attentions in (Vaswani et al., 2017; Veliˇckovi´c et al., 2018), we propose a multi-head graph encoder for the graph encoding in which each head is a GIN and the head outputs are concatenated into corresponding node and graph embeddings.
Revised: Motivated by the success of multi-head attentions in (Vaswani et al., 2017; Veliˇckovi´c et al., 2018), we propose a multi-head graph encoder for the graph encoding in which each head is a GIN and the head outputs are concatenated into corresponding node and graph embeddings.

Original: Using multiple graph heads allows the model to encode diverse aspects of the input program.
Revised: Using multiple graph heads allows the model to encode diverse aspects of the input program.

Original: Our architecture is different from the per-layer multi-head attention in Transformer (Vaswani et al., 2017) and graph attention networks (Veli ˇ ckovi ´
Revised: Our architecture is different from the per-layer multi-head attention in Transformer (Vaswani et al., 2017) and graph attention networks (Veli ˇ ckovi ´

Original: c et al., 2018) as well as multiple towers in message passing neural networks (Gilmer et al., 2017).
Revised: c et al., 2018) as well as multiple towers in message passing neural networks (Gilmer et al., 2017).

Original: We do not have cross-attention, hence having multiple heads per graph network layer is equivalent to adding more linear projections, which merely increase the GIN’s depth.
Revised: We do not have cross-attention, hence having multiple heads per graph network layer is equivalent to adding more linear projections, which merely increase the GIN’s depth.

Original: We choose to concatenate the resulting GIN representations at the end as an ensemble model and ﬁnd such a strategy effective in our ablation study (c.f. Appendix E).
Revised: We choose to concatenate the resulting GIN representations at the end as an ensemble model and ﬁnd such a strategy effective in our ablation study (c.f. Appendix E).

Original: The graph encoder is the component with the most parameters.
Revised: The graph encoder is the component with the most parameters.

Original: The encoder’s size is the function of the number of layers, the latent dimension and the number of heads.
Revised: The encoder’s size is the function of the number of layers, the latent dimension and the number of heads.

Original: We mainly experiment with a multi-head model that has 4 layers, 8 heads and the latent dimension d = 256 , which amounts to 32M parameters.
Revised: We mainly experiment with a multi-head model that has 4 layers, 8 heads and the latent dimension d = 256 , which amounts to 32M parameters.

Original: 4. P RE - TRAINING OBJECTIVE
Revised: 4. P RE - TRAINING OBJECTIVE

Original: We have described the architecture of G RAPHIX as a graph-based, autoregressive model.
Revised: We have described the architecture of G RAPHIX as a graph-based, autoregressive model.

Original: G RAPHIX is a supervised model and trained on labeled bug-ﬁx data using the teacher-forcing technique.
Revised: G RAPHIX is a supervised model and trained on labeled bug-ﬁx data using the teacher-forcing technique.

Original: Since the labeled dataset is relatively small, in order for the model to learn more inherent program structures, we devise a novel strategy to pre-train G RAPHIX on unlabeled source code and allow the knowledge to be transferred to the downstream program repair task.
Revised: Since the labeled dataset is relatively small, in order for the model to learn more inherent program structures, we devise a novel strategy to pre-train G RAPHIX on unlabeled source code and allow the knowledge to be transferred to the downstream program repair task.

Original: Pre-training task.
Revised: Pre-training task.

Original: Generalizing the masked language model pre-training idea in NLP (Devlin et al., 2019; Lewis et al., 2020), we devise a pre-training technique for the tree-structured data, namely deleted sub-tree reconstruction .
Revised: Generalizing the masked language model pre-training idea in NLP (Devlin et al., 2019; Lewis et al., 2020), we devise a pre-training technique for the tree-structured data, namely deleted sub-tree reconstruction .

Original: To that end, we take the ASTs from a collection of unlabeled source code.
Revised: To that end, we take the ASTs from a collection of unlabeled source code.

Original: For each AST, we randomly select a sub-tree of a certain size rooted at node n .
Revised: For each AST, we randomly select a sub-tree of a certain size rooted at node n .

Original: We replace n with a dummy node, and remove all the subsequent nodes and edges in the sub-tree.
Revised: We replace n with a dummy node, and remove all the subsequent nodes and edges in the sub-tree.

Original: Given the dummy node’s location, G RAPHIX is then trained to reconstruct the sub-tree using the other code context in the sequential manner.
Revised: Given the dummy node’s location, G RAPHIX is then trained to reconstruct the sub-tree using the other code context in the sequential manner.

Original: The decoder expands the partial AST one node at a time in the depth-ﬁrst traversal order (Maddison & Tarlow, 2014; Bielik et al., 2016), using the production rules in the underlying abstract language.
Revised: The decoder expands the partial AST one node at a time in the depth-ﬁrst traversal order (Maddison & Tarlow, 2014; Bielik et al., 2016), using the production rules in the underlying abstract language.

Original: Similar to one tree edit in our bug-ﬁxing task, each expansion step corresponds to a classiﬁcation problem, and therefore we use the same cross-entropy objective for training without changing the network architecture.
Revised: Similar to one tree edit in our bug-ﬁxing task, each expansion step corresponds to a classiﬁcation problem, and therefore we use the same cross-entropy objective for training without changing the network architecture.

Original: Our pre-training task can be viewed as a generalization of the masked language model objective in
Revised: Our pre-training task can be viewed as a generalization of the masked language model objective in

Original: BERT (Devlin et al., 2019) and a specialization of the denoising objective in BART (Lewis et al., 2020) to the tree structure of code.
Revised: BERT (Devlin et al., 2019) and a specialization of the denoising objective in BART (Lewis et al., 2020) to the tree structure of code.

Original: The graph reconstruction task is conceptually similar to the graph generative models (Li et al., 2018;
Revised: The graph reconstruction task is conceptually similar to the graph generative models (Li et al., 2018;

Original: Brockschmidt et al., 2018) for expression generation and the structural language model of code (Alon et al., 2020) for any-code completion.
Revised: Brockschmidt et al., 2018) for expression generation and the structural language model of code (Alon et al., 2020) for any-code completion.

Original: Unlike these works, we are interested in pre-training rather than code generation/completion.
Revised: Unlike these works, we are interested in pre-training rather than code generation/completion.

Original: Note that our pre-training technique is speciﬁc to ASTs with an underlying syntax language and not applicable to arbitrary graph structures.
Revised: Note that our pre-training technique is speciﬁc to ASTs with an underlying syntax language and not applicable to arbitrary graph structures.

Original: The pre-training strategy, however, can be extended to work with other types of graph corruptions such as multiple sub-tree additions and replacements as well as objectives such as link prediction and so on.
Revised: The pre-training strategy, however, can be extended to work with other types of graph corruptions such as multiple sub-tree additions and replacements as well as objectives such as link prediction and so on.

Original: We defer these directions to future work.
Revised: We defer these directions to future work.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate G RAPHIX on the Patches in the Wild benchmark and compare its performance to existing translation-based and edit-based baselines.
Revised: We evaluate G RAPHIX on the Patches in the Wild benchmark and compare its performance to existing translation-based and edit-based baselines.

Original: We conduct a series of experiments and studies on the network architectures (i.e., model size and single vs. multiple heads) and the data types (short vs. long function, and abstract vs. concrete code) to strengthen our evaluation.
Revised: We conduct a series of experiments and studies on the network architectures (i.e., model size and single vs. multiple heads) and the data types (short vs. long function, and abstract vs. concrete code) to strengthen our evaluation.

Original: 5. D ATASET
Revised: 5. D ATASET

Original: We use the Patches in the Wild Java bug-ﬁx benchmark, which was curated and popularized by Tufano et al.
Revised: We use the Patches in the Wild Java bug-ﬁx benchmark, which was curated and popularized by Tufano et al.

Original: (2019b).
Revised: (2019b).

Original: The dataset consists of 123,804 bug-ﬁx pairs extracted from GitHub repositories.
Revised: The dataset consists of 123,804 bug-ﬁx pairs extracted from GitHub repositories.

Original: It has two subsets: the small subset contains 58,350 methods/functions with less than 50 tokens whereas each of the 65,454 medium functions is between 50 and 100 tokens.
Revised: It has two subsets: the small subset contains 58,350 methods/functions with less than 50 tokens whereas each of the 65,454 medium functions is between 50 and 100 tokens.

Original: To build the ﬁnal dataset, the authors extracted about 10M GitHub commits whose messages textually match one of these (“ﬁx”, “bug”, “error”, and “exception”) patterns.
Revised: To build the ﬁnal dataset, the authors extracted about 10M GitHub commits whose messages textually match one of these (“ﬁx”, “bug”, “error”, and “exception”) patterns.

Original: Finally, the pair of code before and after each commit is considered one bug-ﬁx sample with duplicates being removed.
Revised: Finally, the pair of code before and after each commit is considered one bug-ﬁx sample with duplicates being removed.

Original: Each subset was originally divided into training, validation and test buckets, and we use the same splits in our evaluation.
Revised: Each subset was originally divided into training, validation and test buckets, and we use the same splits in our evaluation.

Original: Code abstraction.
Revised: Code abstraction.

Original: Tufano et al.
Revised: Tufano et al.

Original: (2019b) promoted the code abstraction idea where they abstracted away types, names and literal values in each method with generic names such as METHOD_1,
Revised: (2019b) promoted the code abstraction idea where they abstracted away types, names and literal values in each method with generic names such as METHOD_1,

Original: VAR_2, INT_3.
Revised: VAR_2, INT_3.

Original: Most existing work employs this abstracted code benchmark with an only exception of (Drain et al., 2021).
Revised: Most existing work employs this abstracted code benchmark with an only exception of (Drain et al., 2021).

Original: For convenience, we call it abstract benchmark.
Revised: For convenience, we call it abstract benchmark.

Original: We also conduct extensive experiments on the concrete version that comes from the same dataset (Tufano et al., 2019b) without performing the code abstraction procedure.
Revised: We also conduct extensive experiments on the concrete version that comes from the same dataset (Tufano et al., 2019b) without performing the code abstraction procedure.

Original: 5. D ATA PROCESSING FOR G RAPHIX
Revised: 5. D ATA PROCESSING FOR G RAPHIX

Original: Our proposed model G RAPHIX and its baseline H OPPITY (Dinella et al., 2020) operate on the
Revised: Our proposed model G RAPHIX and its baseline H OPPITY (Dinella et al., 2020) operate on the

Original: AST level, so we need two additional processing steps: code parsing and ground-truth graph edit construction.
Revised: AST level, so we need two additional processing steps: code parsing and ground-truth graph edit construction.

Original: Given each pair of buggy and ﬁxed methods, we run JavaParser 2 to parse the source code into a pair of ASTs and serialize the output ASTs into a JSON format.
Revised: Given each pair of buggy and ﬁxed methods, we run JavaParser 2 to parse the source code into a pair of ASTs and serialize the output ASTs into a JSON format.

Original: We discardedsamples (out of 123,804 samples) that JavaParser was unable to parse.
Revised: We discardedsamples (out of 123,804 samples) that JavaParser was unable to parse.

Original: Those few samples should not affect the evaluation.
Revised: Those few samples should not affect the evaluation.

Original: Ground-truth edit construction.
Revised: Ground-truth edit construction.

Original: Ground-truth edit sequences are the sole supervision signals that enable the learning of our graph edit model.
Revised: Ground-truth edit sequences are the sole supervision signals that enable the learning of our graph edit model.

Original: Each edit includes a pre-deﬁned operation type (see
Revised: Each edit includes a pre-deﬁned operation type (see

Original: Appendix A.2 for more details), the edit location, the node type and value.
Revised: Appendix A.2 for more details), the edit location, the node type and value.

Original: H OPPITY (Dinella et al., 2020) uses JSON differencing algorithm 3 to create the ground-truth tree edit sequence for each pair of buggy and ﬁxed code.
Revised: H OPPITY (Dinella et al., 2020) uses JSON differencing algorithm 3 to create the ground-truth tree edit sequence for each pair of buggy and ﬁxed code.

Original: Since G RAPHIX is a grammar-driven tree transformation (Yao et al., 2021), we implement a grammar-aware and shortest distance tree differencing algorithm, inspired by a dynamic program in Yao et al.
Revised: Since G RAPHIX is a grammar-driven tree transformation (Yao et al., 2021), we implement a grammar-aware and shortest distance tree differencing algorithm, inspired by a dynamic program in Yao et al.

Original: (Algorithm 3).
Revised: (Algorithm 3).

Original: One could also use the GumTree differencing tool 4 .
Revised: One could also use the GumTree differencing tool 4 .

Original: We show an example of two types of ground-truth edit sequences from the two differencing algorithms and statistics highlighting the advantages of the grammar-aware tree differencing in Appendix B.
Revised: We show an example of two types of ground-truth edit sequences from the two differencing algorithms and statistics highlighting the advantages of the grammar-aware tree differencing in Appendix B.

Original: Pre-training data.
Revised: Pre-training data.

Original: We use the CodeSearchNet dataset released by Husain et al.
Revised: We use the CodeSearchNet dataset released by Husain et al.

Original: for our pre-training purpose.
Revised: for our pre-training purpose.

Original: The entire dataset consists of 6.4M functions in six programming languages extracted from public non-fork GitHub repositories.
Revised: The entire dataset consists of 6.4M functions in six programming languages extracted from public non-fork GitHub repositories.

Original: Since our focus is on the Patches in the Wild Java benchmark, we use the Java portion of CodeSearchNet with a total of 1.5M functions.
Revised: Since our focus is on the Patches in the Wild Java benchmark, we use the Java portion of CodeSearchNet with a total of 1.5M functions.

Original: We ﬁlter out those functions that are not parsable or have more than 600 AST nodes.
Revised: We ﬁlter out those functions that are not parsable or have more than 600 AST nodes.

Original: The remaining functions can be considered ﬁxed code and are used the pre-training task.
Revised: The remaining functions can be considered ﬁxed code and are used the pre-training task.

Original: During pre-training, we select sub-trees between 2 and 6 descendants for the deletion and construct a sequence of addition operations as the ground-truth edits.
Revised: During pre-training, we select sub-trees between 2 and 6 descendants for the deletion and construct a sequence of addition operations as the ground-truth edits.

Original: More details on implementation are provided in Appendix C.
Revised: More details on implementation are provided in Appendix C.

Original: 5. B ASELINES AND METRICS
Revised: 5. B ASELINES AND METRICS

Original: We compare G RAPHIX to translation-based models including an LTSTM model (Tufano et al., 2019b) as well as state-of-the-art pre-trained Transformer models including notably PL-BART (Ahmad et al., 2021) and CodeT5 (Wang et al., 2021).
Revised: We compare G RAPHIX to translation-based models including an LTSTM model (Tufano et al., 2019b) as well as state-of-the-art pre-trained Transformer models including notably PL-BART (Ahmad et al., 2021) and CodeT5 (Wang et al., 2021).

Original: We also compare our model against edit-based models including Copy That!
Revised: We also compare our model against edit-based models including Copy That!

Original: (Panthaplackel et al., 2021) and H OPPITY (Dinella et al., 2020) of which the latter is directly related to our work.
Revised: (Panthaplackel et al., 2021) and H OPPITY (Dinella et al., 2020) of which the latter is directly related to our work.

Original: H OPPITY applies to Javascript and only works on short single-node edits (up to 3).
Revised: H OPPITY applies to Javascript and only works on short single-node edits (up to 3).

Original: That is, it does not handle a non-trivial sub-tree addition.
Revised: That is, it does not handle a non-trivial sub-tree addition.

Original: For comparison, we modify its implementation to handle Java source code, and in order to add a sub-tree, we ﬂatten the sub-tree from its root in the top-down and left-right order and create a series of single-node addition edits.
Revised: For comparison, we modify its implementation to handle Java source code, and in order to add a sub-tree, we ﬂatten the sub-tree from its root in the top-down and left-right order and create a series of single-node addition edits.

Original: To train the model, we use the teacher forcing technique so that at each intermediate step of the addition, the model is given the location of the parent node as well as type and value in the previous step.
Revised: To train the model, we use the teacher forcing technique so that at each intermediate step of the addition, the model is given the location of the parent node as well as type and value in the previous step.

Original: We name this enhanced baseline e-H OPPITY for convenience.
Revised: We name this enhanced baseline e-H OPPITY for convenience.

Original: Metrics.
Revised: Metrics.

Original: We use the standard top-1 exact match accuracy (EM) as the metric and compare the performance of all the above models on the small and medium test sets.
Revised: We use the standard top-1 exact match accuracy (EM) as the metric and compare the performance of all the above models on the small and medium test sets.

Original: Note that EMs for eH OPPITY and G RAPHIX are computed based on AST matching, which is equivalent to sequence matching in the other models.
Revised: Note that EMs for eH OPPITY and G RAPHIX are computed based on AST matching, which is equivalent to sequence matching in the other models.

Original: For G RAPHIX and e-H OPPITY , we use beam search of size 5 during inference and select the candidate with the highest score normalized by the edit sequence length.
Revised: For G RAPHIX and e-H OPPITY , we use beam search of size 5 during inference and select the candidate with the highest score normalized by the edit sequence length.

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: In this section, G RAPHIX refers to the 8-head model with 32M parameters that is not pre-trained.
Revised: In this section, G RAPHIX refers to the 8-head model with 32M parameters that is not pre-trained.

Original: G RAPHIX -P refers to G RAPHIX that is pre-trained and then ﬁne-tuned on the respective datasets.
Revised: G RAPHIX -P refers to G RAPHIX that is pre-trained and then ﬁne-tuned on the respective datasets.

Original: We present the results on the abstract and concrete benchmark in Table 1 and Table 2, respectively
Revised: We present the results on the abstract and concrete benchmark in Table 1 and Table 2, respectively

Original: [Table]
Revised: [Table]

Original: Each table consists of three groups of results: the ﬁrst group represents the translation-based methods without pre-training, the second group is for edit-based methods and the last group includes pretrained models including G RAPHIX -P. Note that DeepDebug (Drain et al., 2021) was not evaluated on the abstract code while most were not on the concrete code.
Revised: Each table consists of three groups of results: the ﬁrst group represents the translation-based methods without pre-training, the second group is for edit-based methods and the last group includes pretrained models including G RAPHIX -P. Note that DeepDebug (Drain et al., 2021) was not evaluated on the abstract code while most were not on the concrete code.

Original: [Table]
Revised: [Table]

Original: From Table 1, we can see that our models signiﬁcantly outperform the other edit-based baselines and are on a par with larger pre-trained models.
Revised: From Table 1, we can see that our models signiﬁcantly outperform the other edit-based baselines and are on a par with larger pre-trained models.

Original: Speciﬁcally, without pre-training G RAPHIX improves
Revised: Speciﬁcally, without pre-training G RAPHIX improves

Original: CodeBERT by 1.8% on the small subset and 4% on the medium subset.
Revised: CodeBERT by 1.8% on the small subset and 4% on the medium subset.

Original: We observe a similar improvement of G RAPHIX over BART.
Revised: We observe a similar improvement of G RAPHIX over BART.

Original: When pre-trained on CodeSearchNet Java, G RAPHIX -P slightly outperforms PL-BART, CodeT5-small on the small but underperforms them on the medium .
Revised: When pre-trained on CodeSearchNet Java, G RAPHIX -P slightly outperforms PL-BART, CodeT5-small on the small but underperforms them on the medium .

Original: CodeT5-base with 220M parameters is currently the state-of-the-art performance on both datasets.
Revised: CodeT5-base with 220M parameters is currently the state-of-the-art performance on both datasets.

Original: [Table]
Revised: [Table]

Original: Table 2 shows the results of the models on the concrete benchmark.
Revised: Table 2 shows the results of the models on the concrete benchmark.

Original: The PL-BART result, which was not reported in the original paper, is obtained by ﬁne-tuning the released pre-trained PL-BART model on the concrete code.
Revised: The PL-BART result, which was not reported in the original paper, is obtained by ﬁne-tuning the released pre-trained PL-BART model on the concrete code.

Original: We observe a similar improvement on the concrete benchmark for G RAPHIX -P over G RAPHIX , which is signiﬁcantly better the edit baselines, LSTM and the Transformer model.
Revised: We observe a similar improvement on the concrete benchmark for G RAPHIX -P over G RAPHIX , which is signiﬁcantly better the edit baselines, LSTM and the Transformer model.

Original: G RAPHIX -P is slightly better than DeepDebug (Drain et al., 2021) on the small set but worse on the medium .
Revised: G RAPHIX -P is slightly better than DeepDebug (Drain et al., 2021) on the small set but worse on the medium .

Original: Compared to PL-BART, G RAPHIX -P performs comparably on the small dataset and outperforms PL-BART by a good margin on the medium set.
Revised: Compared to PL-BART, G RAPHIX -P performs comparably on the small dataset and outperforms PL-BART by a good margin on the medium set.

Original: In short, our model works comparably to DeepDebug and PL-BART on the concrete benchmark despite much fewer parameters.
Revised: In short, our model works comparably to DeepDebug and PL-BART on the concrete benchmark despite much fewer parameters.

Original: Comparing the results in Table 1 and Table 2, we see a negligible drop about 0.2 – 0.5% in accuracy of G RAPHIX from the abstract to concrete code.
Revised: Comparing the results in Table 1 and Table 2, we see a negligible drop about 0.2 – 0.5% in accuracy of G RAPHIX from the abstract to concrete code.

Original: On the contrary, the accuracy of GRU with span copy (Panthaplackel et al., 2021) decreases by almost half.
Revised: On the contrary, the accuracy of GRU with span copy (Panthaplackel et al., 2021) decreases by almost half.

Original: This demonstrates that G RAPHIX is insensitive to the naming of variables, types and APIs.
Revised: This demonstrates that G RAPHIX is insensitive to the naming of variables, types and APIs.

Original: We also observe a similar trend for PL-BART.
Revised: We also observe a similar trend for PL-BART.

Original: These observations suggest that the code abstraction may not be necessary.
Revised: These observations suggest that the code abstraction may not be necessary.

Original: Additionally, we notice that while the pre-training provides an additional 10% relative gain on the small subsets, G RAPHIX -P does not achieve similar gains on the medium set, which is inherently longer and harder.
Revised: Additionally, we notice that while the pre-training provides an additional 10% relative gain on the small subsets, G RAPHIX -P does not achieve similar gains on the medium set, which is inherently longer and harder.

Original: We hypothesize that the synthetic edits used for pre-training might be more aligned with the small dataset than the medium , and more sampling strategies may be needed to bridge the gap.
Revised: We hypothesize that the synthetic edits used for pre-training might be more aligned with the small dataset than the medium , and more sampling strategies may be needed to bridge the gap.

Original:  A NECDOTAL E XAMPLES
Revised:  A NECDOTAL E XAMPLES

Original: In addition to several state-of-the-art results in EM accuracy, G RAPHIX demonstrates strong inductive biases in learning complex bug-ﬁx patterns.
Revised: In addition to several state-of-the-art results in EM accuracy, G RAPHIX demonstrates strong inductive biases in learning complex bug-ﬁx patterns.

Original: In this section, we showcase several bug examples that
Revised: In this section, we showcase several bug examples that

Original: G RAPHIX is able to detect and ﬁx.
Revised: G RAPHIX is able to detect and ﬁx.

Original: We show more such examples in Appendix F.
Revised: We show more such examples in Appendix F.

Original: [Table]
Revised: [Table]

Original: samples small
Revised: samples small

Original: (bottom).
Revised: (bottom).

Original: @Override public void resolveAnaphora() { List<Proposition> props = VariableStorage.getPopostionList(); int i = 0; for (CQuantifer quant : this .getQuantifers())
Revised: @Override public void resolveAnaphora() { List<Proposition> props = VariableStorage.getPopostionList(); int i = 0; for (CQuantifer quant : this .getQuantifers())

Original: { if (i < = (props.size())) props.get(i).setLinkedId(((String)
Revised: { if (i < = (props.size())) props.get(i).setLinkedId(((String)

Original: (quant.getVar().getSourceId()))); i++; } } public static int mul( int n1, int n2) { return n1 + * n2; }
Revised: (quant.getVar().getSourceId()))); i++; } } public static int mul( int n1, int n2) { return n1 + * n2; }

Original: Listing out
Revised: Listing out

Original: We manually analyze the generated ﬁxes of G RAPHIX on the test concrete benchmark.
Revised: We manually analyze the generated ﬁxes of G RAPHIX on the test concrete benchmark.

Original: Since longer edits tend to have lower probability, we normalize the prediction score of each generated ﬁx by the length of the corresponding generated edit sequence.
Revised: Since longer edits tend to have lower probability, we normalize the prediction score of each generated ﬁx by the length of the corresponding generated edit sequence.

Original: More speciﬁcally, we divide the negative log-likelihood by the edit length and exponentiate the result to obtain the normalized score.
Revised: More speciﬁcally, we divide the negative log-likelihood by the edit length and exponentiate the result to obtain the normalized score.

Original: Then, we manually check 500 correct and incorrect ﬁxes with highest scores in each of the small and medium sets.
Revised: Then, we manually check 500 correct and incorrect ﬁxes with highest scores in each of the small and medium sets.

Original: By inspecting the generated edits, we can easily spot out the code changes in each case.
Revised: By inspecting the generated edits, we can easily spot out the code changes in each case.

Original: We ﬁnd that similar ﬁxes have similar edits and scores.
Revised: We ﬁnd that similar ﬁxes have similar edits and scores.

Original: This analysis enables us to recognize interesting bug-ﬁx patterns as well as noises in the data.
Revised: This analysis enables us to recognize interesting bug-ﬁx patterns as well as noises in the data.

Original: The patterns range from syntactic ﬁxes such as using equals() instead of == or simplifying logic statements to semantic ones such as like adding null check conditions, ﬁxing variable misuses and so on.
Revised: The patterns range from syntactic ﬁxes such as using equals() instead of == or simplifying logic statements to semantic ones such as like adding null check conditions, ﬁxing variable misuses and so on.

Original: Here, we show some exemplary samples of ﬁx patches that match the ground-truth in the benchmark.
Revised: Here, we show some exemplary samples of ﬁx patches that match the ground-truth in the benchmark.

Original: Listing 2 displays two examples of potential bugs where the developers fail to validate the input/a class ﬁeld.
Revised: Listing 2 displays two examples of potential bugs where the developers fail to validate the input/a class ﬁeld.

Original: G RAPHIX can not only detect missing input validations (i.e., checking null ), it can also return an intended behavior in the second example when the input is null ).
Revised: G RAPHIX can not only detect missing input validations (i.e., checking null ), it can also return an intended behavior in the second example when the input is null ).

Original: Listing 3 shows two other examples where G RAPHIX ﬁxes an off-by-one error and a logical bug by apparently understanding the method name mul .
Revised: Listing 3 shows two other examples where G RAPHIX ﬁxes an off-by-one error and a logical bug by apparently understanding the method name mul .

Original: We show several instances of incorrect ﬁxes generated by G RAPHIX in Appendix G.
Revised: We show several instances of incorrect ﬁxes generated by G RAPHIX in Appendix G.

Original: We note that extra enclosed parentheses () in the examples are artifacts of our parser.
Revised: We note that extra enclosed parentheses () in the examples are artifacts of our parser.

Original: Finally, G RAPHIX and
Revised: Finally, G RAPHIX and

Original: G RAPHIX -P share similar inductive biases and hence share similar sets of bug-ﬁx patterns.
Revised: G RAPHIX -P share similar inductive biases and hence share similar sets of bug-ﬁx patterns.

Original:  D ISCUSSION AND F UTURE W ORK
Revised:  D ISCUSSION AND F UTURE W ORK

Original: In this paper, we consider the problem of generating bug ﬁxes using a neural model.
Revised: In this paper, we consider the problem of generating bug ﬁxes using a neural model.

Original: We present
Revised: We present

Original: G RAPHIX , a multi-head graph edit model that is pre-trained based on deleted sub-tree reconstruction.
Revised: G RAPHIX , a multi-head graph edit model that is pre-trained based on deleted sub-tree reconstruction.

Original: G RAPHIX establishes the high performance on a Java benchmark.
Revised: G RAPHIX establishes the high performance on a Java benchmark.

Original: Anecdotal ﬁxes demonstrate that the model is able to detect and suggest ﬁxes for a diverse set of bugs and program issues.
Revised: Anecdotal ﬁxes demonstrate that the model is able to detect and suggest ﬁxes for a diverse set of bugs and program issues.

Original: In particular,
Revised: In particular,

Original: G RAPHIX is about 2 × smaller than state-of-the-art Transformer models.
Revised: G RAPHIX is about 2 × smaller than state-of-the-art Transformer models.

Original: Finally, G RAPHIX performs equally well on generating abstract as well as concrete ﬁxes.
Revised: Finally, G RAPHIX performs equally well on generating abstract as well as concrete ﬁxes.

Original: Practical use.
Revised: Practical use.

Original: The problem of bug detection is nuanced, and several times ML models can be replaced by simple rules.
Revised: The problem of bug detection is nuanced, and several times ML models can be replaced by simple rules.

Original: We have shown that G RAPHIX can capture semantic bug ﬁxes through dozens of anecdotal examples (see Appendix F for more).
Revised: We have shown that G RAPHIX can capture semantic bug ﬁxes through dozens of anecdotal examples (see Appendix F for more).

Original: In addition, we demonstrate in Appendix
Revised: In addition, we demonstrate in Appendix

Original: D that one can calibrate the model score to achieve an increased precision up to 57% by trading-off the coverage of bug detection.
Revised: D that one can calibrate the model score to achieve an increased precision up to 57% by trading-off the coverage of bug detection.

Original: We anticipate several directions in the future work: (i) pre-train based on multiple sub-tree additions and replacements (ii) incorporate more program dependencies into the graph representation and (iii) leverage Transformer and GNNs in a uniﬁed architecture for the global code context in the sequence.
Revised: We anticipate several directions in the future work: (i) pre-train based on multiple sub-tree additions and replacements (ii) incorporate more program dependencies into the graph representation and (iii) leverage Transformer and GNNs in a uniﬁed architecture for the global code context in the sequence.

Original: Appendix A E NCODER AND D ECODER
Revised: Appendix A E NCODER AND D ECODER

Original: This section describes the technical formulation of the single-head encoder and the auto-regressive decoder.
Revised: This section describes the technical formulation of the single-head encoder and the auto-regressive decoder.

Original: For more details into these two components, we refer the reader to (Dinella et al., 2020; Yao et al., 2021).
Revised: For more details into these two components, we refer the reader to (Dinella et al., 2020; Yao et al., 2021).

Original: A. S INGLE - HEAD ENCODER
Revised: A. S INGLE - HEAD ENCODER

Original: Formally, given a graph g
Revised: Formally, given a graph g

Original: =
Revised: =

Original: ( V, E ) with a set of nodes V and edges E , a single-head graph encoder gives a d -dimensional representation of the graph g and representations of individual nodes v ∈ V .
Revised: ( V, E ) with a set of nodes V and edges E , a single-head graph encoder gives a d -dimensional representation of the graph g and representations of individual nodes v ∈ V .

Original: Similar to (Dinella et al., 2020), we compute the graph presentations for each edge type (i.e., AST edges, last-variable-use edges and successive edges) as follows:
Revised: Similar to (Dinella et al., 2020), we compute the graph presentations for each edge type (i.e., AST edges, last-variable-use edges and successive edges) as follows:

Original: [Equation]
Revised: [Equation]

Original: where K denotes the total number of edge types, L is the number of propagations.
Revised: where K denotes the total number of edge types, L is the number of propagations.

Original: W l,k∈ R d × d and
Revised: W l,k∈ R d × d and

Original: W
Revised: W

Original: [Equation]
Revised: [Equation]

Original: k∈ R
Revised: k∈ R

Original: dK ×
Revised: dK ×

Original: d are the network weights.
Revised: d are the network weights.

Original: N k ( v ) denotes a set of neighbors of v that are connected to v with the edge type k .
Revised: N k ( v ) denotes a set of neighbors of v that are connected to v with the edge type k .

Original: These formulae give the node embeddings n = { h ( L ) v } v ∈ V for all nodes v ∈ V while the graph representation is an aggregation of h ( l ) v for all l .
Revised: These formulae give the node embeddings n = { h ( L ) v } v ∈ V for all nodes v ∈ V while the graph representation is an aggregation of h ( l ) v for all l .

Original: More precisely, we use max pooling to combine h ( l ) v for each l and average the L + 1 resulting vectors to compute the ﬁnal graph representation g .
Revised: More precisely, we use max pooling to combine h ( l ) v for each l and average the L + 1 resulting vectors to compute the ﬁnal graph representation g .

Original: The initial embedding h (0) v vector of node v is set as the sum of the embedding of its node type and node value if the value exists.
Revised: The initial embedding h (0) v vector of node v is set as the sum of the embedding of its node type and node value if the value exists.

Original: The node type and value embeddings are stored in separate tables.
Revised: The node type and value embeddings are stored in separate tables.

Original: A. T REE EDIT OPERATIONS
Revised: A. T REE EDIT OPERATIONS

Original: G RAPHIX is a sequential decision process that iteratively performs tree edits on the partial tree at each step.
Revised: G RAPHIX is a sequential decision process that iteratively performs tree edits on the partial tree at each step.

Original: Following (Brockschmidt et al., 2018; Yao et al., 2021), we employ a grammar-based tree decoder to ensure the grammatical correctness of each edit action.
Revised: Following (Brockschmidt et al., 2018; Yao et al., 2021), we employ a grammar-based tree decoder to ensure the grammatical correctness of each edit action.

Original: To that end, we construct an abstract language based on JavaParser 5 .
Revised: To that end, we construct an abstract language based on JavaParser 5 .

Original: The grammar includes a set of primitive and composite types (c.f.
Revised: The grammar includes a set of primitive and composite types (c.f.

Original: (Rabinovich et al., 2017; Yin & Neubig, 2017)).
Revised: (Rabinovich et al., 2017; Yin & Neubig, 2017)).

Original: Each composite type (e.g., stmt ) has a collection of production rules such as ExpressionStmt(expr expression) where each rule is represented as a constructor with a list of ﬁeld arguments.
Revised: Each composite type (e.g., stmt ) has a collection of production rules such as ExpressionStmt(expr expression) where each rule is represented as a constructor with a list of ﬁeld arguments.

Original: Each argument has the ﬁeld type (e.g., expr ), the ﬁeld name (e.g., expression ) and the cardinality of the ﬁeld as single , optional , or sequential .
Revised: Each argument has the ﬁeld type (e.g., expr ), the ﬁeld name (e.g., expression ) and the cardinality of the ﬁeld as single , optional , or sequential .

Original: The cardinality indicates the number of children a node accepts as grammatically valid children.
Revised: The cardinality indicates the number of children a node accepts as grammatically valid children.

Original: Based on the grammar, we deﬁne the following primitive operations:
Revised: Based on the grammar, we deﬁne the following primitive operations:

Original: DELETE operator takes a tree node n and removes it from the tree.
Revised: DELETE operator takes a tree node n and removes it from the tree.

Original: As a result, the corresponding sub-tree and the node n are removed.
Revised: As a result, the corresponding sub-tree and the node n are removed.

Original: However, removing a node that corresponds to a single ﬁeld results in a grammatically invalid tree.
Revised: However, removing a node that corresponds to a single ﬁeld results in a grammatically invalid tree.

Original: For instance, removing node ClassType in step 1 of Figure 1 causes the AST to have no type, which is invalid.
Revised: For instance, removing node ClassType in step 1 of Figure 1 causes the AST to have no type, which is invalid.

Original: To ensure the syntax validity at each edit step, after the removal, we add a Dummy node as a placeholder with no node type or value.
Revised: To ensure the syntax validity at each edit step, after the removal, we add a Dummy node as a placeholder with no node type or value.

Original: Another beneﬁt of Dummy node is to reduce the uncertainty of the node selection in the subsequent step.
Revised: Another beneﬁt of Dummy node is to reduce the uncertainty of the node selection in the subsequent step.

Original: ADD operator adds a node to the graph.
Revised: ADD operator adds a node to the graph.

Original: Under the grammar and the dummy node mechanism, the addition of a new node occurs in two cases.
Revised: Under the grammar and the dummy node mechanism, the addition of a new node occurs in two cases.

Original: In the ﬁrst case, we add a new child to a parent node of a sequential ﬁeld.
Revised: In the ﬁrst case, we add a new child to a parent node of a sequential ﬁeld.

Original: For example, the node of type (stmt* statements) may have an arbitrary number of children.
Revised: For example, the node of type (stmt* statements) may have an arbitrary number of children.

Original: In this case, adding a node requires the location of the parent node and the position to be added in its children list.
Revised: In this case, adding a node requires the location of the parent node and the position to be added in its children list.

Original: In the second case, we add a new node into the position of a
Revised: In the second case, we add a new node into the position of a

Original: Dummy node (adding a non-terminal node ExpressionStmt in Figure 1 in step 4).
Revised: Dummy node (adding a non-terminal node ExpressionStmt in Figure 1 in step 4).

Original: Similarly to the DELETE operation, once adding a non-terminal node, we follow the underlying grammar speciﬁed by the added production rule and instantiate the set of dummy, child nodes at the newly added node.
Revised: Similarly to the DELETE operation, once adding a non-terminal node, we follow the underlying grammar speciﬁed by the added production rule and instantiate the set of dummy, child nodes at the newly added node.

Original: The ADD operator is also used to populate empty terminal nodes with actual node types and possible values.
Revised: The ADD operator is also used to populate empty terminal nodes with actual node types and possible values.

Original: Note that while constructing the original tree, we also add Dummy at the empty ﬁelds.
Revised: Note that while constructing the original tree, we also add Dummy at the empty ﬁelds.

Original: COPY operator is introduced to simplify and shorten the edit sequence.
Revised: COPY operator is introduced to simplify and shorten the edit sequence.

Original: While making code changes, the developer often moves code one place to another.
Revised: While making code changes, the developer often moves code one place to another.

Original: Such changes can be translated directly on the
Revised: Such changes can be translated directly on the

Original: AST by copying the corresponding sub-tree from the input AST.
Revised: AST by copying the corresponding sub-tree from the input AST.

Original: In essence, this operator locates the target position similarly to the ADD operation and then copies the sub-tree from the original tree to the target position of the current tree in one single step.
Revised: In essence, this operator locates the target position similarly to the ADD operation and then copies the sub-tree from the original tree to the target position of the current tree in one single step.

Original: For example, the subtree for the assignment expression data = newData in Figure 1 at t = 2 is copied to the new position in step 6.
Revised: For example, the subtree for the assignment expression data = newData in Figure 1 at t = 2 is copied to the new position in step 6.

Original: Finally, the NO_OP action is used to automatically terminate the iterative tree editing procedure.
Revised: Finally, the NO_OP action is used to automatically terminate the iterative tree editing procedure.

Original: After this step, the remaining dummy nodes will be pruned.
Revised: After this step, the remaining dummy nodes will be pruned.

Original: A. A UTOREGRESSIVE TREE DECODER
Revised: A. A UTOREGRESSIVE TREE DECODER

Original: Given the graph representation g t of the program at step t and the edit history s t − 1 from the previous step, we use an LSTM-based tree decoder to update the state s t =
Revised: Given the graph representation g t of the program at step t and the edit history s t − 1 from the previous step, we use an LSTM-based tree decoder to update the state s t =

Original: LSTM ( g t | s t − 1 ) .
Revised: LSTM ( g t | s t − 1 ) .

Original: Based on the updated state s t , the model predict edit actions where each node edit includes a node selector, an operator predictor and a type/value predictor.
Revised: Based on the updated state s t , the model predict edit actions where each node edit includes a node selector, an operator predictor and a type/value predictor.

Original: The node selector selects node loc t , then the operator predictor decides which operator op t to apply the edit on loc t among the four possible operators { DELETE , ADD , COPY , NO_OP } .
Revised: The node selector selects node loc t , then the operator predictor decides which operator op t to apply the edit on loc t among the four possible operators { DELETE , ADD , COPY , NO_OP } .

Original: For the ADD operation, the type predictor further determines the node type/label type t for the newly added node.
Revised: For the ADD operation, the type predictor further determines the node type/label type t for the newly added node.

Original: For terminal nodes, the value predictor computes the node value value t .
Revised: For terminal nodes, the value predictor computes the node value value t .

Original: The probability of an edit is then decomposed as
Revised: The probability of an edit is then decomposed as

Original: [Equation]
Revised: [Equation]

Original: , loc t ) p ( value t | s t , op t , loc t , type t ) .
Revised: , loc t ) p ( value t | s t , op t , loc t , type t ) .

Original: [Equation]
Revised: [Equation]

Original: We provide more details into one decoding step below:
Revised: We provide more details into one decoding step below:

Original: Node selection.
Revised: Node selection.

Original: For node selection, we employ a pointer network (Vinyals et al., 2015) since different graphs have different number of nodes.
Revised: For node selection, we employ a pointer network (Vinyals et al., 2015) since different graphs have different number of nodes.

Original: For ADD , the selector either selects a sequential-ﬁeld node or a Dummy node.
Revised: For ADD , the selector either selects a sequential-ﬁeld node or a Dummy node.

Original: In the ﬁrst case, it determines the parent of the node to be added and its left sibling.
Revised: In the ﬁrst case, it determines the parent of the node to be added and its left sibling.

Original: Therefore, we further decomposte p ( loc t | s t ) =
Revised: Therefore, we further decomposte p ( loc t | s t ) =

Original: p ( sibling t | s t , parent t ) p ( loc t | s t , parent t ) .
Revised: p ( sibling t | s t , parent t ) p ( loc t | s t , parent t ) .

Original: If parent t does not have any children, the left sibling node is considered as parent t itself.
Revised: If parent t does not have any children, the left sibling node is considered as parent t itself.

Original: We compute the logits by taking inner product between the edit history s t and the node embeddings n t .
Revised: We compute the logits by taking inner product between the edit history s t and the node embeddings n t .

Original: We select the node that maximizes the inner product.
Revised: We select the node that maximizes the inner product.

Original: After selecting the node, we update the state with an additional LSTM call s t =
Revised: After selecting the node, we update the state with an additional LSTM call s t =

Original: LSTM ( emb ( loc t ) | s t ) where emb ( loc t ) is the node embedding of the selected node loc t .
Revised: LSTM ( emb ( loc t ) | s t ) where emb ( loc t ) is the node embedding of the selected node loc t .

Original: Operator prediction: The operator prediction is a regular classiﬁcation problem with four labels { DELETE , ADD , COPY , NO_OP } .
Revised: Operator prediction: The operator prediction is a regular classiﬁcation problem with four labels { DELETE , ADD , COPY , NO_OP } .

Original: We calculate the probability of operator op t using a softmax layer such that p ( op t | s t , loc t ) =
Revised: We calculate the probability of operator op t using a softmax layer such that p ( op t | s t , loc t ) =

Original: softmax ( W op s t + b op ) .
Revised: softmax ( W op s t + b op ) .

Original: After that, we again update the state via s t =
Revised: After that, we again update the state via s t =

Original: LSTM ( emb ( op t ) | s t ) where emb ( op t ) is the embeding of the selected operator op t .
Revised: LSTM ( emb ( op t ) | s t ) where emb ( op t ) is the embeding of the selected operator op t .

Original: Type prediction.
Revised: Type prediction.

Original: This module assigns a type type t to the newly added node in the AST.
Revised: This module assigns a type type t to the newly added node in the AST.

Original: Since the number of types is speciﬁed a priori by the underlying syntax language, we simply treat the type predictor as a classiﬁcation problem over all possible types while masking out the impossible choices.
Revised: Since the number of types is speciﬁed a priori by the underlying syntax language, we simply treat the type predictor as a classiﬁcation problem over all possible types while masking out the impossible choices.

Original: We calculate the probability of operator op t using a softmax layer such that p ( type t | s t , loc t ) =
Revised: We calculate the probability of operator op t using a softmax layer such that p ( type t | s t , loc t ) =

Original: softmax ( W type s t + b type ) .
Revised: softmax ( W type s t + b type ) .

Original: We update the state via s t =
Revised: We update the state via s t =

Original: LSTM ( emb ( type t ) | s t ) where emb ( type t ) is the embedding of the newly predicted type type t .
Revised: LSTM ( emb ( type t ) | s t ) where emb ( type t ) is the embedding of the newly predicted type type t .

Original: Value Prediction.
Revised: Value Prediction.

Original: If the newly added node is a terminal node, the value predictor assigns to it a value value t .
Revised: If the newly added node is a terminal node, the value predictor assigns to it a value value t .

Original: The possible values are chosen from the union of the local values present in the input (i.e., local value table) and the global values that frequently appear in other programs.
Revised: The possible values are chosen from the union of the local values present in the input (i.e., local value table) and the global values that frequently appear in other programs.

Original: The local value embeddings are computed by the graph encoder while the global value embeddings are stored globally in the value embedding table.
Revised: The local value embeddings are computed by the graph encoder while the global value embeddings are stored globally in the value embedding table.

Original: We compute the scores by simply taking inner product between the state s t and the value embeddings and select the one maximizing the score.
Revised: We compute the scores by simply taking inner product between the state s t and the value embeddings and select the one maximizing the score.

Original: Finally, we update the state via s t =
Revised: Finally, we update the state via s t =

Original: LSTM ( emb ( value t ) | s t ) where emb ( value t ) is the embedding of the value value t .
Revised: LSTM ( emb ( value t ) | s t ) where emb ( value t ) is the embedding of the value value t .

Original: The decoding process stops once it encounters NO_OP or reaches the maximum number edits.
Revised: The decoding process stops once it encounters NO_OP or reaches the maximum number edits.

Original: B C OMPARISON OF T REE D IFFERENCING A LGORITHMS
Revised: B C OMPARISON OF T REE D IFFERENCING A LGORITHMS

Original: Table 3 shows the advantages of structural, grammar-aware tree differencing with copy operations in
Revised: Table 3 shows the advantages of structural, grammar-aware tree differencing with copy operations in

Original: G RAPHIX over JSON differencing used in H OPPITY .
Revised: G RAPHIX over JSON differencing used in H OPPITY .

Original: [Table]
Revised: [Table]

Original: public boolean create
Revised: public boolean create

Original: [Table]
Revised: [Table]

Original: POJO.
Revised: POJO.

Original: [Table]
Revised: [Table]

Original: user) throws SQLException
Revised: user) throws SQLException

Original: [Table]
Revised: [Table]

Original: prepareStatement
Revised: prepareStatement

Original: [Table]
Revised: [Table]

Original: INSERT INTO User ...")
Revised: INSERT INTO User ...")

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Remove the statement in line 10: return true ; 2.
Revised: Remove the statement in line 10: return true ; 2.

Original: Remove the scope: System.out 3.
Revised: Remove the scope: System.out 3.

Original: Remove the arguments in println 3.
Revised: Remove the arguments in println 3.

Original: Remove the name println 4.
Revised: Remove the name println 4.

Original: Replace the remaining MethodCallExpr with BooleanLiteralExpr 5.
Revised: Replace the remaining MethodCallExpr with BooleanLiteralExpr 5.

Original: Add true 6.
Revised: Add true 6.

Original: Replace ExpressionStmt with ReturnStmt
Revised: Replace ExpressionStmt with ReturnStmt

Original: [Table]
Revised: [Table]

Original: Remove the statement in line 9: System.out.println(...); Listing
Revised: Remove the statement in line 9: System.out.println(...); Listing

Original: An
Revised: An

Original: example
Revised: example

Original: of code change where the println statement in line 9 is removed.
Revised: of code change where the println statement in line 9 is removed.

Original: The structural
Revised: The structural

Original: edit is
Revised: edit is

Original: straightforward with one remove step.
Revised: straightforward with one remove step.

Original: The JSON differencing edits are less natural: the algorithm ﬁrst removes line 10 and gradually changes line 9 to re-add the return true statement.
Revised: The JSON differencing edits are less natural: the algorithm ﬁrst removes line 10 and gradually changes line 9 to re-add the return true statement.

Original: We can see that the structural edit differencing results in much shorter edit sequences on average.
Revised: We can see that the structural edit differencing results in much shorter edit sequences on average.

Original: Examination of edits shows that structural edits also more meaningful.
Revised: Examination of edits shows that structural edits also more meaningful.

Original: We show an example of code change and edits in Listing 4.
Revised: We show an example of code change and edits in Listing 4.

Original: C I MPLEMENTATION AND T RAINING D ETAILS
Revised: C I MPLEMENTATION AND T RAINING D ETAILS

Original: We implement G RAPHIX with PyTorch based on the open source implementation of H OPPITY (Dinella et al., 2020).
Revised: We implement G RAPHIX with PyTorch based on the open source implementation of H OPPITY (Dinella et al., 2020).

Original: We train G RAPHIX on a machine with 8 V100 GPUs, each with 32GB memory.
Revised: We train G RAPHIX on a machine with 8 V100 GPUs, each with 32GB memory.

Original: We use batch size 5 on each GPU both in pre-training and ﬁne-tuning.
Revised: We use batch size 5 on each GPU both in pre-training and ﬁne-tuning.

Original: We use the Adam optimizer and a linear learning rate scheduler, used in Transformer (Vaswani et al., 2017).
Revised: We use the Adam optimizer and a linear learning rate scheduler, used in Transformer (Vaswani et al., 2017).

Original: We set the initial learning rate 10 − 4 and the warmup steps corresponding to 1 epoch.
Revised: We set the initial learning rate 10 − 4 and the warmup steps corresponding to 1 epoch.

Original: We pre-train G RAPHIX forepochs for 10 days.
Revised: We pre-train G RAPHIX forepochs for 10 days.

Original: We ﬁne-tune G RAPHIX for 20 epochs with the maximum number of edit steps being 20.
Revised: We ﬁne-tune G RAPHIX for 20 epochs with the maximum number of edit steps being 20.

Original: The ﬁne-tuning takes about 2 hours per epoch on the small subset and about 3.5 hours per epoch on the medium set.
Revised: The ﬁne-tuning takes about 2 hours per epoch on the small subset and about 3.5 hours per epoch on the medium set.

Original: We choose the best model checkpoint using the validation set and report its top-1 accuracy on the test set.
Revised: We choose the best model checkpoint using the validation set and report its top-1 accuracy on the test set.

Original: D H IGH PRECISION REGIME FOR G RAPHIX
Revised: D H IGH PRECISION REGIME FOR G RAPHIX

Original: As shown previously and in Appendix F, G RAPHIX is able to learn meaningful and interpretable code patterns, leading to a wide range of interesting bug ﬁxes.
Revised: As shown previously and in Appendix F, G RAPHIX is able to learn meaningful and interpretable code patterns, leading to a wide range of interesting bug ﬁxes.

Original: Indeed, the actual kinds of ﬁxes have not been studied thoroughly for the pre-trained Transformer models.
Revised: Indeed, the actual kinds of ﬁxes have not been studied thoroughly for the pre-trained Transformer models.

Original: Moreover, we show that our model can be of potential use in practice by trading-off the recall for an increased precision.
Revised: Moreover, we show that our model can be of potential use in practice by trading-off the recall for an increased precision.

Original: To demonstrate that, we show the impact of calibrating the prediction scores to achieve such a trade-off.
Revised: To demonstrate that, we show the impact of calibrating the prediction scores to achieve such a trade-off.

Original: [Table]
Revised: [Table]

Original: Table 4 suggests that we can achieve higher accuracy at the cost of lower prediction frequency.
Revised: Table 4 suggests that we can achieve higher accuracy at the cost of lower prediction frequency.

Original: For example, the model can suggest 418 most probable bug ﬁxes at 47.37% correct patches, at the threshold 0.7 on the prediction score.
Revised: For example, the model can suggest 418 most probable bug ﬁxes at 47.37% correct patches, at the threshold 0.7 on the prediction score.

Original: E A BLATION S TUDIES E. E FFECTS OF THE GRAPH MODEL SIZE AND PRE - TRAINING
Revised: E A BLATION S TUDIES E. E FFECTS OF THE GRAPH MODEL SIZE AND PRE - TRAINING

Original: In Table 5, we compare G RAPHIX with H OPPITY using about 1M parameters.
Revised: In Table 5, we compare G RAPHIX with H OPPITY using about 1M parameters.

Original: Here, G RAPHIXB refers to the proposed G RAPHIX model with 1M parameters.
Revised: Here, G RAPHIXB refers to the proposed G RAPHIX model with 1M parameters.

Original: The results show that G RAPHIX is signiﬁcantly better than H OPPITY because of the ASDL-guided decoding and the grammaraware shortest edit differencing.
Revised: The results show that G RAPHIX is signiﬁcantly better than H OPPITY because of the ASDL-guided decoding and the grammaraware shortest edit differencing.

Original: To be speciﬁc, G RAPHIX edits are signiﬁcantly shorter and more meaningful than JSON differencing edits (see Table 3 and Listing 4 in Appendix B for the comparison).
Revised: To be speciﬁc, G RAPHIX edits are signiﬁcantly shorter and more meaningful than JSON differencing edits (see Table 3 and Listing 4 in Appendix B for the comparison).

Original: Moreover, the grammar-constrained decoder reduces the uncertainty of predicting edit properties at each decoding step.
Revised: Moreover, the grammar-constrained decoder reduces the uncertainty of predicting edit properties at each decoding step.

Original: [Table]
Revised: [Table]

Original: We also study the effects of the graph model size and pre-training on G RAPHIX .
Revised: We also study the effects of the graph model size and pre-training on G RAPHIX .

Original: Table 5 suggests that the multi-head encoding leads to about 50% relative boost in accuracy on the small, abstract subset and almost 2 × boost on the abstract, medium .
Revised: Table 5 suggests that the multi-head encoding leads to about 50% relative boost in accuracy on the small, abstract subset and almost 2 × boost on the abstract, medium .

Original: The improvements are consistent on the concrete code.
Revised: The improvements are consistent on the concrete code.

Original: It can be seen that the pre-training gives an additional 10% relative gain on the small subsets but does not help on the medium sets.
Revised: It can be seen that the pre-training gives an additional 10% relative gain on the small subsets but does not help on the medium sets.

Original: We explore more pre-training options and model sizes in our future work.
Revised: We explore more pre-training options and model sizes in our future work.

Original: Next, we study the role of our proposed multi-head architecture.
Revised: Next, we study the role of our proposed multi-head architecture.

Original: E. R OLE OF THE MULTI - HEAD ARCHITECTURE
Revised: E. R OLE OF THE MULTI - HEAD ARCHITECTURE

Original: [Table]
Revised: [Table]

Original: Following (Dinella et al., 2020)
Revised: Following (Dinella et al., 2020)

Original: [Table]
Revised: [Table]

Original: we use the GIN depth of 4 for all the experiments.
Revised: we use the GIN depth of 4 for all the experiments.

Original: To understand the the impact of the number of heads compared to the latent dimension
Revised: To understand the the impact of the number of heads compared to the latent dimension

Original: [Table]
Revised: [Table]

Original: we experiment with the latent dimension d and the number of heads H such that the corresponding model sizes are comparable.
Revised: we experiment with the latent dimension d and the number of heads H such that the corresponding model sizes are comparable.

Original: It can be seen from Table 6 that the multi-head models outperform the corresponding single-head counterparts with comparable sizes, for both 4M and 32M models.
Revised: It can be seen from Table 6 that the multi-head models outperform the corresponding single-head counterparts with comparable sizes, for both 4M and 32M models.

Original: Moreover, increasing the model capacity with d improves the performance as d increases from 128 to 256 while the model with d = 566 tends to overﬁt the training data and performs poorly on the test set.
Revised: Moreover, increasing the model capacity with d improves the performance as d increases from 128 to 256 while the model with d = 566 tends to overﬁt the training data and performs poorly on the test set.

Original: In summary, our experiments show the beneﬁts of the proposed multi-head architecture.
Revised: In summary, our experiments show the beneﬁts of the proposed multi-head architecture.

Original: E. P ER - LAYER VERSUS LAST - LAYER MULTI - HEAD
Revised: E. P ER - LAYER VERSUS LAST - LAYER MULTI - HEAD

Original: We implemente a per-layer multi-head architecture with head averaging and GIN, following the idea in GAT (Veliˇckovi´c et al., 2018).
Revised: We implement a per-layer multi-head architecture with head averaging and GIN, following the idea in GAT (Veliˇckovi´c et al., 2018).

Original: In this architecture, each head computes a linear projection of the input before it is fed to the respective GIN layer, so without attention having multiple heads per layer is the same as adding two more consecutive linear layers before that GIN layer: the ﬁrst layer stacks the linear projection matrices into one weight matrix whereas the second has a ﬁxed weight and computes average.
Revised: In this architecture, each head computes a linear projection of the input before it is fed to the respective GIN layer, so without attention having multiple heads per layer is the same as adding two more consecutive linear layers before that GIN layer: the ﬁrst layer stacks the linear projection matrices into one weight matrix whereas the second has a ﬁxed weight and computes average.

Original: We take the average of the head outputs instead of the concatenation for a computational reason.
Revised: We take the average of the head outputs instead of the concatenation for a computational reason.

Original: We compare this new model with a single-head model of the same latent dimension ( d = 256 ), and our ensemble-style multi-head models (with H = 4 and H = 8 heads).
Revised: We compare this new model with a single-head model of the same latent dimension ( d = 256 ), and our ensemble-style multi-head models (with H = 4 and H = 8 heads).

Original: The results and conﬁguration of each model are shown in Table 7.
Revised: The results and conﬁguration of each model are shown in Table 7.

Original: [Table]
Revised: [Table]

Original: We can see this model performs slightly better than the single-head model thanks to the additional multi-head layers across the datasets, but it is worse than GRAPHIX with 4 heads at the end and 2M fewer parameters, except for the abstract, small dataset.
Revised: We can see this model performs slightly better than the single-head model thanks to the additional multi-head layers across the datasets, but it is worse than GRAPHIX with 4 heads at the end and 2M fewer parameters, except for the abstract, small dataset.

Original: Along with the previous empirical study on the width, this experiment demonstrates the beneﬁts of the proposed multi-head architecture over purely increasing the graph neural network’s depth or width.
Revised: Along with the previous empirical study on the width, this experiment demonstrates the beneﬁts of the proposed multi-head architecture over purely increasing the graph neural network’s depth or width.

Original: F M ORE A NECDOTAL E XAMPLES
Revised: F M ORE A NECDOTAL E XAMPLES

Original: We show some more bug ﬁxing and code refactoring patterns learned by G RAPHIX .
Revised: We show some more bug ﬁxing and code refactoring patterns learned by G RAPHIX .

Original: In some cases, we show the pairs of buggy code and ﬁxed code.
Revised: In some cases, we show the pairs of buggy code and ﬁxed code.

Original: In other cases, we only highlight the changes with added text in green and deleted text in red.
Revised: In other cases, we only highlight the changes with added text in green and deleted text in red.

Original: A few long statements are replaced with “ . . . ”.
Revised: A few long statements are replaced with “ . . . ”.

Original: F.
Revised: F.

Original: S IMPLIFYING LOGIC EXPRESSIONS
Revised: S IMPLIFYING LOGIC EXPRESSIONS

Original: Before
Revised: Before

Original: After small/ public boolean isEmpty() { if ((first) =
Revised: After small/ public boolean isEmpty() { if ((first) =

Original: = null ) { return true ; } return false ; } public boolean isEmpty() { return (first) =
Revised: = null ) { return true ; } return false ; } public boolean isEmpty() { return (first) =

Original: = null ; } medium/ @Override public boolean onTouchEvent(android.view.
Revised: = null ; } medium/ @Override public boolean onTouchEvent(android.view.

Original: MotionEvent ev) { super .onTouchEvent(ev); dragHelper.processTouchEvent(ev);
Revised: MotionEvent ev) { super .onTouchEvent(ev); dragHelper.processTouchEvent(ev);

Original: ViewGroup parent =
Revised: ViewGroup parent =

Original: ((ViewGroup) (findBottomView( this , x, y).getParent())); return false || (parent
Revised: ((ViewGroup) (findBottomView( this , x, y).getParent())); return false || (parent

Original: ==
Revised: ==

Original: ( this )); }
Revised: ( this )); }

Original: F.
Revised: F.

Original: F IXING OFF - BY - ONE ERRORS
Revised: F IXING OFF - BY - ONE ERRORS

Original: Before
Revised: Before

Original: After small/ @Override public boolean hasNext() { return ((cursor) + 1) < (batches); } @Override public boolean hasNext() { return (cursor) < (batches); } small/ public boolean hasNext() { return (frameIndex) <
Revised: After small/ @Override public boolean hasNext() { return ((cursor) + 1) < (batches); } @Override public boolean hasNext() { return (cursor) < (batches); } small/ public boolean hasNext() { return (frameIndex) <

Original: ((count) - 1); } public boolean hasNext() { return (frameIndex) <
Revised: ((count) - 1); } public boolean hasNext() { return (frameIndex) <

Original: (count); medium/ public static boolean isPowerOfTwo( long number) { if (number < = 0) { throw new IllegalArgumentException(...); } if ((number & (-number))
Revised: (count); medium/ public static boolean isPowerOfTwo( long number) { if (number < = 0) { throw new IllegalArgumentException(...); } if ((number & (-number))

Original: =
Revised: =

Original: = number) { return true ; } return false ; }U SING EQUALS OF COMPARISON medium/ public int run() { int exponent = 1000;
Revised: = number) { return true ; } return false ; } medium/ public int run() { int exponent = 1000;

Original: BigInteger base = BigInteger.valueOf(2); BigInteger value =
Revised: BigInteger base = BigInteger.valueOf(2); BigInteger value =

Original: BigInteger.
Revised: BigInteger.

Original: ZERO; int sum = 0; value = base.pow(exponent);
Revised: ZERO; int sum = 0; value = base.pow(exponent);

Original: String str
Revised: String str

Original: =
Revised: =

Original: value.toString(); for ( int i = 0; i < ((str.length()) - 1) (str.length() ; i++) sum += (( int ) ((str.charAt(i)) - '0')); return sum; }small/
Revised: value.toString(); for ( int i = 0; i < ((str.length()) - 1) (str.length() ; i++) sum += (( int ) ((str.charAt(i)) - '0')); return sum; }

Original:  public TimelineConfig
Revised: F. U SING EQUALS INSTEAD OF == FOR S TRING COMPARISON small/ public TimelineConfig

Original: findChannelById(String id) { for (TimelineConfig channel : channels) { if (channel.getId() .equals(id) )
Revised: findChannelById(String id) { for (TimelineConfig channel : channels) { if (channel.getId() .equals(id) )

Original: return channel; } return null ; } small/ private Customer findCustomer(String customerCode) { for (Customer c : customers) { if (c.getCode() .equals(customerCode) )
Revised: return channel; } return null ; } small/ private Customer findCustomer(String customerCode) { for (Customer c : customers) { if (c.getCode() .equals(customerCode) )

Original: { return c; } } return null ; } small/ public boolean checkUsername(String username) { for (Model.
Revised: { return c; } } return null ; } small/ public boolean checkUsername(String username) { for (Model.

Original: User user : userRepository.findAll()) { if (user.getName() .equals(username) )
Revised: User user : userRepository.findAll()) { if (user.getName() .equals(username) )

Original: { return false ; } } return true ; 
Revised: { return false ; } } return true ; }

Revised: F.

Original:  medium/ private static List<GroupData> generateGroups( int count) {
Revised: C ORRECTING RETURNED OBJECTS medium/ private static List<GroupData> generateGroups( int count) {

Original: List<GroupData> groups = new ArrayList<GroupData>(); for ( int i = 0; i < count; i++) {
Revised: List<GroupData> groups = new ArrayList<GroupData>(); for ( int i = 0; i < count; i++) {

Original: groups.add( new GroupData().withName(...)); } return null groups; } medium/ @Override public String toString() {
Revised: groups.add( new GroupData().withName(...)); } return null groups; } medium/ @Override public String toString() {

Original: String str =
Revised: String str =

Original: "Symbol Table list:"; for ( int i = nestinglevel; (-1) <
Revised: "Symbol Table list:"; for ( int i = nestinglevel; (-1) <

Original: (nestinglevel); i++) { str +=
Revised: (nestinglevel); i++) { str +=

Original: ("Nesting level " + i) + ":\n"; str += tables[i].toString(); }return str; }
Revised: ("Nesting level " + i) + ":\n"; str += tables[i].toString(); }return str; }

Original: F. F IXINGCONDITIONS
Revised: F. F IXING

Original:  medium/ public View getViewWithConf(String viewName) { if ((viewName != null ) && ( !
Revised: CONDITIONS medium/ public View getViewWithConf(String viewName) { if ((viewName != null ) && ( !

Original: (viewName.isEmpty())))
Revised: (viewName.isEmpty())))

Original: { for (fi.nls.oskari.domain.map.view.
Revised: { for (fi.nls.oskari.domain.map.view.

Original: View item : list) { if (viewName.equals(item.getName()))
Revised: View item : list) { if (viewName.equals(item.getName()))

Original: { return item; } } } return null ; } medium/ private double getStepMovement(genetics.
Revised: { return item; } } } return null ; } medium/ private double getStepMovement(genetics.

Original: MusicPhenotype p) { double steps = 0; double intervalCount = 0; for (java.util.
Revised: MusicPhenotype p) { double steps = 0; double intervalCount = 0; for (java.util.

Original: ArrayList<
Revised: ArrayList<

Original: Integer> measure : p.melodyIntervals) { for ( int interval : measure) { interval = Math.abs(interval); intervalCount++; if ((interval >= 1) || && (interval <= 2)) { steps += 1; } } } if (intervalCount == 0) { return 0; } return steps / intervalCount; } medium/ public boolean 
Revised: Integer> measure : p.melodyIntervals) { for ( int interval : measure) { interval = Math.abs(interval); intervalCount++; if ((interval >= 1) || && (interval <= 2)) { steps += 1; } } } if (intervalCount == 0) { return 0; } return steps / intervalCount; } medium/ public boolean insertBudget

Original: G N EGATIVE E XAMPLES
Revised: G N EGATIVE E XAMPLES

Original: While learning meaningful frequent change patterns, G RAPHIX also conﬁdently suggests incomplete and/or incorrect bug ﬁxes.
Revised: While learning meaningful frequent change patterns, G RAPHIX also conﬁdently suggests incomplete and/or incorrect bug ﬁxes.

Original: We provide three such examples below.
Revised: We provide three such examples below.

Original: For each example, we show the buggy input, the generated ﬁx by G RAPHIX and the ground-truth ﬁxed code.
Revised: For each example, we show the buggy input, the generated ﬁx by G RAPHIX and the ground-truth ﬁxed code.

Revised: In Listing 5, G RAPHIX suggests a quick ﬁx that replaces || with && but does not go beyond adding logical !

Revised: to the second condition.

Revised: Since G RAPHIX can add !

Revised: in medium/9887 , Section F.5 when trained on the medium dataset, we suspect that the model might not have seen many !.isEmpty

Original: Buggy code @Override protected void onSaveInstanceState(android.os.
Revised: () patterns on the small set. Note that, even if the model could do that, the generated ﬁx and the ground-truth label with two nested if conditions would not match, despite the logical equivalence. [Table] @Override protected void onSaveInstanceState(android.os.

Original: Bundle savedInstance) { super .onSaveInstanceState(savedInstance); if (((listData) !
Revised: Bundle savedInstance) { super .onSaveInstanceState(savedInstance); if (((listData) !

Original: = null )
Revised: = null )

Original: || (listData.isEmpty()))
Revised: || (listData.isEmpty()))

Original: { savedInstance.putSerializable("HEADER", listData); } }
Revised: { savedInstance.putSerializable("HEADER", listData); } }

Original: Generated ﬁxed code @Override protected void onSaveInstanceState(android.os.
Revised: Generated ﬁxed code @Override protected void onSaveInstanceState(android.os.

Original: Bundle savedInstance) { super .onSaveInstanceState(savedInstance); if (((listData) !
Revised: Bundle savedInstance) { super .onSaveInstanceState(savedInstance); if (((listData) !

Original: = null ) && (listData.isEmpty())) { savedInstance.putSerializable("HEADER", listData); } }
Revised: = null ) && (listData.isEmpty savedInstance.putSerializable [Table]

Original: Ground-truth ﬁxed code @Override protected void onSaveInstanceState(android.os.
Revised: @Override protected void onSaveInstanceState android.os.

Original: Bundle savedInstance) { super .onSaveInstanceState(savedInstance); if ((listData) !
Revised: Bundle savedInstance) { super .onSaveInstanceState savedInstance

Original: = null) { if(!
Revised: ; if null { if

Original: (listData.isEmpty())) { savedInstance.putSerializable("HEADER", listData [Equation]
Revised: .putSerializable

Original: Listing 5: Test sample small/6783 in the benchmark.
Revised: Listing 5: Test sample small/ in the benchmark.

Original: In Listing 5, G RAPHIX suggests a quick ﬁx that replaces || with && but does not go beyond adding logical !
Revised: Listing 6 shows another incorrect ﬁx by G RAPHIX .

Original:  to  the second condition.
Revised: This example is nuanced because it could be difﬁcult to know in the ﬁrst place whether the original code is buggy or not, without more information.

Original: Since G RAPHIX can add ! in medium/9887 , Section F.5 when trained on the medium dataset, we suspect that the model might not have seen many !.isEmpty () patterns on the small set.
Revised: The last example, shown in Listing 7, is similar to the pattern G RAPHIX correctly learns in small/6886 , Section F.1 and many others we observe in our manual analysis.

Original: Note that, even if the model could do that, the generated ﬁx and the ground-truth label with two nested if conditions would not match, despite the logical equivalence.
Revised: In this case, the model simpliﬁes the logical return expression in the input code, but that does not match with the correct ﬁx. Again, it is difﬁcult to ﬁx this kind of issues unless we have a way to represent the global context in the class and more broadly the whole project.

Original: “Buggy” code private boolean isValidInLength(android.text.
Revised: “Buggy” code private boolean isValidInLength(android.text.

Original: Editable s) { return (!(respectPatternLength))
Revised: Editable s) { return (!(respectPatternLength))

Original: || ((s.length()) <
Revised: || ((s.length()) <

Original: = (maxLength)); }
Revised: = (maxLength)); }

Original: Generated ﬁxed code private boolean isValidInLength(android.text.
Revised: Generated ﬁxed code private boolean isValidInLength(android.text.

Original: Editable s) { return (!(respectPatternLength)) && ((s.length()) <
Revised: Editable s) { return (!(respectPatternLength)) && ((s.length()) <

Original: = (maxLength)); }
Revised: = (maxLength)); }

Original: Ground-truth ﬁxed code private boolean isValidInLength(android.text.
Revised: Ground-truth ﬁxed code private boolean isValidInLength(android.text.

Original: Editable s) { if ((respectPatternLength) && ((s.length()) > (maxLength))) return false ; return true ; }
Revised: Editable s) { if ((respectPatternLength) && ((s.length()) > (maxLength))) return false ; return true ; }

Original: Listing 6: Test sample small/9672 in the benchmark.
Revised: Listing 6: Test sample small/9672 in the benchmark.

Original: Listing 6 shows another incorrect ﬁx by G RAPHIX .

Original: This example is nuanced because it could be difﬁcult to know in the ﬁrst place whether the original code is buggy or not, without more information.

Original: The last example, shown in Listing 7, is similar to the pattern G RAPHIX correctly learns in small/6886 ,

Original: Section F.1 and many others we observe in our manual analysis.

Original: In this case, the model simpliﬁes the logical return expression in the input code, but that does not match with the correct ﬁx.

Original: Again, it is difﬁcult to ﬁx this kind of issues unless we have a way to represent the global context in the class and more broadly the whole project.

Original: Buggy code @Override public boolean onKeyUp( int keyCode, KeyEvent event)
Revised: Buggy code @Override public boolean onKeyUp( int keyCode, KeyEvent event)

Original: { if ((keyCode == (KEYCODE_VOLUME_DOWN))
Revised: { if ((keyCode == (KEYCODE_VOLUME_DOWN))

Original: || (keyCode == (KEYCODE_VOLUME_UP)))
Revised: || (keyCode == (KEYCODE_VOLUME_UP)))

Original: { return true ; } return false ;
Revised: { return true ; } return false ;

Original: Generated ﬁxed code @Override public boolean onKeyUp( int keyCode, KeyEvent event) { return (keyCode == (KEYCODE_VOLUME_DOWN))
Revised: Generated ﬁxed code @Override public boolean onKeyUp( int keyCode, KeyEvent event) { return (keyCode == (KEYCODE_VOLUME_DOWN))

Original: || (keyCode == (KEYCODE_VOLUME_UP))
Revised: || (keyCode == (KEYCODE_VOLUME_UP))

Original: Ground-truth ﬁxed code @Override public boolean onKeyUp( int keyCode, KeyEvent event) { if ((keyCode == (KEYCODE_VOLUME_DOWN))
Revised: Ground-truth ﬁxed code @Override public boolean onKeyUp( int keyCode, KeyEvent event) { if ((keyCode == (KEYCODE_VOLUME_DOWN))

Original: || (keyCode == (KEYCODE_VOLUME_UP)))
Revised: || (keyCode == (KEYCODE_VOLUME_UP)))

Original: { return true ; } return super .onKeyUp(keyCode, event); }
Revised: { return true ; } return super .onKeyUp(keyCode, event); }

Original: Listing 7: Test sample small/7677 in the 
Revised: Listing 7: Test sample small/7677 in the benchmark.

Original: H E XAMPLES OF H OPPITY
Revised: H E XAMPLES OF H OPPITY

Original: Finally, we show representative examples learned by e-H OPPITY in the decreasing order of the scores.
Revised: Finally, we show representative examples learned by e-H OPPITY in the decreasing order of the scores.

Original: small/ protected public final int getRetryMaxAttempts() { return retryMaxAttempts; } small/ public java.lang.
Revised: small/ protected public final int getRetryMaxAttempts() { return retryMaxAttempts; } small/ public java.lang.

Original: String[] getText() { return this.
Revised: String[] getText() { return this.

Original: text; } small/ @Override public projectx.persistence.entities.
Revised: text; } small/ @Override public projectx.persistence.entities.

Original: Category findByid( long id) { return db.findCategoryById(id); } small/ public models.
Revised: Category findByid( long id) { return db.findCategoryById(id); } small/ public models.

Original: T1Entity save(models.
Revised: T1Entity save(models.

Original: T1Entity entity) throws ClientException, ParseException { if (entity == null ) return null ; if (!
Revised: T1Entity entity) throws ClientException, ParseException { if (entity == null ) return null ; if (!

Original: (isAuthenticated()))
Revised: (isAuthenticated()))

Original: return null ; models.
Revised: return null ; models.

Original: T1Entity response = postService.save(entity); return response; return postService.save(entity); } small/ @Override public float computeBonus() { float bonus =
Revised: T1Entity response = postService.save(entity); return response; return postService.save(entity); } small/ @Override public float computeBonus() { float bonus =

Original: (( this .salary)
Revised: (( this .salary)

Original: * ( this .pctBonus))
Revised: * ( this .pctBonus))

Original: + 1000; return bonus; } medium/ @java.lang.
Revised: + 1000; return bonus; } medium/ @java.lang.

Original: Override public boolean onOptionsItemSelected(android.view.
Revised: Override public boolean onOptionsItemSelected(android.view.

Original: MenuItem item) { boolean retval = true ; switch (item.getItemId()) { case android.R.id.home : onBackPressed(); break ; case R.id.change_password : changeAccountPassword(account); break ; case R.id.delete_account : openAccountRemovalConfirmationDialog(account); break ; default : retval = super .onOptionsItemSelected(item); break; } return retval; }
Revised: MenuItem item) { boolean retval = true ; switch (item.getItemId()) { case android.R.id.home : onBackPressed(); break ; case R.id.change_password : changeAccountPassword(account); break ; case R.id.delete_account : openAccountRemovalConfirmationDialog(account); break ; default : retval = super .onOptionsItemSelected(item); break; } return retval; }