PatternSmith

PatternSmith is a decoder-only Transformer that converts natural-language descriptions into regular expressions.

Model details

  • Architecture: GPT-style decoder-only Transformer
  • Framework: PyTorch
  • Parameters: 7,242,624
  • Tokenizer: GPT-2 BPE (tiktoken)
  • Vocabulary size: 50,257
  • Context length: 128
  • Transformer blocks: 4
  • Attention heads: 4
  • Embedding dimension: 128
  • Feed-forward dimension: 512
  • Training: Supervised next-token prediction
  • Initialization: Random weights

Training dataset

NL-RX-Turk, cleaned and split into:

  • Training: 6,245 examples
  • Validation: 961 examples
  • Test: 2,402 examples

Evaluation results

Metric Result
Best validation loss 0.2044
Validation perplexity 1.2268
Test exact match 37.14%
Semantic equivalence 55.54%
Python regex compilation 96.96%

Semantic equivalence was evaluated using regexbench BRICS FULLMATCH.

Intended use

Research and experimentation with natural-language-to-regex generation. This model is a portfolio project, not a production-ready regex generation system.

Limitations

  • Exact-match accuracy is limited.
  • Syntactically valid regex does not guarantee semantic correctness.
  • Predictions should be validated against representative test cases.
  • The model may fail on complex or ambiguous descriptions.

Checkpoint

The trained PyTorch checkpoint is available as best.pt in this repository.

Source code

GitHub: https://github.com/ramu1904/PatternSmith

The model architecture and inference implementation are available in the GitHub repository.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support