---
language:
- code
- en
license: apache-2.0
tags:
- commit_message_generation
- code
datasets:
- JetBrains-Research/commit-chronicle
---

# CMG/CMC: RACE (with history)

This is the checkpoint for [RACE](https://aclanthology.org/2022.emnlp-main.372.pdf) model, fine-tuned for the commit message generation (and/or completion) task as part of the paper "From Commit Message Generation to History-Aware Commit Message Completion", ASE 2023.

## Details

> 🔍 For further details, please refer to:
> * **Paper**: TODO
> * **Repository**: [https://github.com/JetBrains-Research/commit_message_generation](https://github.com/JetBrains-Research/commit_message_generation)


* This model is based on the fine-tuned CodeT5 checkpoint [`JetBrains-Research/cmg-codet5-with-history`](https://huggingface.co/JetBrains-Research/cmg-codet5-with-history) and uses RACE architecture introduced in 📜 [RACE: Retrieval-Augmented Commit Message Generation](https://aclanthology.org/2022.emnlp-main.372.pdf).
  * Note: Requires a custom model class. Check [our implementation](https://github.com/JetBrains-Research/commit_message_generation/blob/appendix_cmg/src/model/configurations/utils/race.py) or [the replication package](https://github.com/DeepSoftwareAnalytics/RACE) provided by RACE authors. 
* This model was trained with commit diffs as well as WITH commit message history.
* This model was trained on the CommitChronicle dataset introduced in our study. 
* Our hyperparameter setting is mostly based on 📜 [RACE: Retrieval-augmented Commit Message Generation](https://aclanthology.org/2022.emnlp-main.372/).
 The exact values are provided below:
    
    |       Hyperparameter       |                                                                    Value                                                                    |
    |:--------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------:|
    | Encoder context max length |                                                                     512                                                                     |
    | Decoder context max length |                                                                     512                                                                     |
    | Number of training epochs  |                                                                      1                                                                      |
    |         Batch size         |                                                                     32                                                                      |
    |         Optimizer          |                  [AdamW](https://pytorch.org/docs/1.12/generated/torch.optim.AdamW.html?highlight=adamw#torch.optim.AdamW)                  |
    |           Warmup           | [Linear](https://huggingface.co/docs/transformers/v4.21.3/en/main_classes/optimizer_schedules#transformers.get_linear_schedule_with_warmup) |
    |   Number of warmup steps   |                                                                     100                                                                     |
    |     Peak learning rate     |                                                                   0.00002                                                                   |


## Available checkpoints

We also released checkpoints for other models fine-tuned as part of our study.

* Models trained *with commit message history*:
  * **CodeT5:** 🤗 [`JetBrains-Research/cmg-codet5-with-history`](https://huggingface.co/JetBrains-Research/cmg-codet5-with-history)
  * **CodeReviewer:** 🤗 [`JetBrains-Research/cmg-codereviewer-with-history`](https://huggingface.co/JetBrains-Research/cmg-codereviewer-with-history) 
  * **RACE:** 🤗 [`JetBrains-Research/cmg-race-with-history`](https://huggingface.co/JetBrains-Research/cmg-race-with-history) (this model)
* Models trained *without commit message history*:
  * **CodeT5:** 🤗 [`JetBrains-Research/cmg-codet5-without-history`](https://huggingface.co/JetBrains-Research/cmg-codet5-without-history)
  * **CodeReviewer:** 🤗 [`JetBrains-Research/cmg-codereviewer-without-history`](https://huggingface.co/JetBrains-Research/cmg-codereviewer-without-history)
  * **RACE:** 🤗 [`JetBrains-Research/cmg-race-without-history`](https://huggingface.co/JetBrains-Research/cmg-race-without-history)

## Citation

```
TODO
```