Enhancing Language Model with Unit Test Techniques for Efficient Regular Expression Generation (2023)

Abstract: Recent research has investigated the use of generative language models to produce regular expressions with semantic-based approaches. However, these approaches have shown shortcomings in practical applications, particularly in terms of functional correctness, which refers to the ability to reproduce the intended function inputs by the user. To address this issue, we present a novel method called Unit-Test Driven Reinforcement Learning (UTD-RL). Our approach differs from previous methods by taking into account the crucial aspect of functional correctness and transforming it into a differentiable gradient feedback using policy gradient techniques. In which functional correctness can be evaluated through Unit Tests, a testing method that ensures regular expressions meets its design and performs as intended. Experiments conducted on three public datasets demonstrate the effectiveness of the proposed method in generating regular expressions. This method has been employed in a regulatory scenario where regular expressions can be utilized to ensure that all online content is free from non-compliant elements, thereby significantly reducing the workload of relevant personnel.

## introduction regular expressions are an essential tool for processing text in an efficient, flexible, and powerful manner @xcite . for instance, an individual whose work involves reviewing the language used in an application to prevent the display of violent or pornographic content to underage users. manually checking each line can be a time-consuming task. therefore, the use of regular expressions can greatly streamline this process. nevertheless, writing and debugging regular expressions can be a daunting task for those without expertise, as the syntax can often be obscure and unintuitive @xcite . the use of natural language to generate regular expressions has been explored in several works to figure 1 : pipeline of our works. and the whole pipeline consists of 3 steps: the first step will generate prompt from the original context; followed by the sft with the prompt generated from the first step; finally unit-test driven reinforcement learning is implemented bridge the gap for the public in utilizing regular expressions. for instance, ranta et al. @xcite developed a rule-based system that generates regular expressions from template input. subsequently, locascio et al. @xcite proposed the use of lstm-based sequence to sequence models to generate regular expressions based on contextual inputs. furthermore, with the advancement of large language models, researchers have discovered that the performance can be improved by employing supervised fine-tuning (sft) @xcite on large language models (llms). nev-ertheless, regular expressions generated by these models often encounter compilation failures and inadequately capture the intended functionality of the input requirements, which is a critical aspect in practical applications. to address this, researchers have explored the use of semantic correctness @xcite as a criterion. however, adopting such a method does not completely resolve the aforementioned issues. we posit that the disregard for functional significance in input specification may be a significant factor contributing to these challenges. therefore, this paper emphasizes the importance of functional correctness. to enhance the functional correctness of the generated regular expression, it is important to consider the practical context in which it will be used. generally, assessing its practical applicability requires conducting "unit test". specifically, if the generated regular expression can accurately extract the desired results from a given sequence of inputs, it can be considered to meet the functional requirements of the user. therefore, in this paper, we propose unit-test driven reinforcement learning (utd-rl). this approach utilizes policy gradient techniques @xcite to learn from the feedback provided by the unit test results, enabling the model to adjust its pattern generation process to better align with the intended functionality. as a result, it shows promise in improving the effectiveness of regular expression generation in practical applications. experimental results demonstrate that regular expressions generated by this method can better adhere to the input requirements, resulting in a significant improvement in the performance of the generated regular expression with respect to unit test. as mentioned earlier, we consider functional correctness to be the most crucial factor in this task. however, we have observed that the previous evaluation method, which computes equivalence by converting each regular expression to a minimal deterministic finite automaton (dfa) and leveraging the fact that minimal dfas are guaranteed to be the same for semantically equivalent regular expressions, is inadequate for assessing the functional correctness of the generated regular expression in relation to the input requirements. therefore, in this paper, we propose the adoption of "unit test" as an alternative method for evaluating the generated regular expression, in addition to utilizing dfa. to sum up our contributions: ## related work recent research has focused on automating the generation of regular expressions from natural language, employing both non-deep learning and deep learning approaches. early researchers highlighted the ability to encode regular expressions into finite state networks @xcite . ranta et al. @xcite capitalized on this property and developed a rule-based technique for converting formatted language specifications into regular expressions. sequentially, locascio et al. @xcite first introduced an lstm-based sequenceto-sequence model (deep regex) that translates contextual information into regular expressions using a syntax-based objective: maximum likelihood estimation (mle). zhong and bhatia @xcite optimized performance by employing policy gradient techniques @xcite to train the model with a semantics-based objective. similarly, park et al. @xcite applied semantic correctness as the reinforcement learning reward. however, experiments conducted on these models revealed significant overfitting on public datasets resulting in limited generalizability to other input requirements. we speculate that lstm lacking the capacity for induction and deduction compared to the advanced large language models available today. recently, large language models (llms) trained on extensive text corpora from diverse domains have exhibited their capability to perform zero-shot tasks, including code generation. this zero-shot ability emerged when models reached an adequate scale @xcite . researchers utilizing pre-trained llms and fine-tuning them on pertinent datasets have achieved remarkable outcomes. for example, codex @xcite , a fine-tuned model on gpt-3 @xcite , outperforms prior state-of-the-art models on code generation. copilot, a highly renowned code suggestion tool within the github community, employs codex as its foundational model. furthermore, codegeex @xcite , a multilingual code generation model equipped with 13 billion parameters, attains the highest average performance on publicly available datasets. ## experimental setup in this section, we evaluate our work on different pre-trained language models to verify its effectiveness. additionally, we conduct test case analysis and present case studies to provide further insights. ## practical application in our context, the app hosts numerous registered merchants. in compliance with market regulatory requirements, these registered merchants are obligated to undergo internal compliance reviews before publishing new advertisement landing pages or text content. this is done to ensure that the content does not contain any non-compliant elements. given the large number of merchants involved and the complexity of the rules, the conventional approach relied heavily on manual creation of regular expressions to identify non-compliant text scenarios. for instance, one requirement for advertisement landing pages was the exclusion of promotional expressions. unfortunately, this approach often resulted in significant time and labor costs associated with the development and testing of regular expressions. now a new solution has been introduced: an automated workflow that utilizes the large language model trained with utd-rl. to make it more specific, this language model is capable of generating production-ready regular expressions and automatically conducting unit test, thereby enabling an automated workflow that greatly facilitates the public's use of regular expressions. the process is depicted in figure 4 . subsequently, a unit test is implemented to assess the validity of the regular expression. if the outcome of the unit test exceeds the threshold, the regular expression is considered valid. conversely, the input prompt is concatenated with the failed cases to regenerate the regular expression. ## conclusion in conclusion, ensuring the functional correctness of regular expressions is crucial in practical appli-cations. this paper proposes the use of utd-rl to effectively utilize the outcomes of unit test as rewards for the model, thereby enhancing the functional correctness. furthermore, "unit test" are employed to assess the functional correctness of the generated regular expressions. this paper solely focuses on evaluating the effectiveness of the proposed method in the generation of regular expressions. however, it is believed that this approach can be extended to generate any corpus that necessitates functional specifications (e.g., python code generation, sql generation, etc.). future research will investigate the applicability of this method in these domains, and we encourage interested researchers to experiment with this approach.