JetBrains-Research/cmg-codet5-with-history

CMG/CMC: CodeT5 (with history)

This is the checkpoint for CodeT5 model, fine-tuned for the commit message generation (and/or completion) task as part of the paper "From Commit Message Generation to History-Aware Commit Message Completion", ASE 2023.

Details

🔍 For further details, please refer to:

• Paper: https://arxiv.org/abs/2308.07655
• Repository: https://github.com/JetBrains-Research/commit_message_generation

• This model is based on Salesforce/codet5-base checkpoint from 📜 CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation.

• This model was trained with commit diffs as well as WITH commit message history.

• This model was trained on the CommitChronicle dataset introduced in our study.

• Our hyperparameter setting is mostly based on 📜 RACE: Retrieval-augmented Commit Message Generation. The exact values are provided below:

  Hyperparameter	Value
  Encoder context max length	512
  Decoder context max length	512
  Number of training epochs	1
  Batch size	32
  Optimizer	AdamW
  Warmup	Linear
  Number of warmup steps	100
  Peak learning rate	0.00002

Available checkpoints

We also released checkpoints for other models fine-tuned as part of our study.

• Models trained with commit message history:
  • CodeT5: 🤗 JetBrains-Research/cmg-codet5-with-history (this model)
  • CodeReviewer: 🤗 JetBrains-Research/cmg-codereviewer-with-history
  • RACE: 🤗 JetBrains-Research/cmg-race-with-history
• Models trained without commit message history:
  • CodeT5: 🤗 JetBrains-Research/cmg-codet5-without-history
  • CodeReviewer: 🤗 JetBrains-Research/cmg-codereviewer-without-history
  • RACE: 🤗 JetBrains-Research/cmg-race-without-history

Citation

TODO