Add Baichuan2 support
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

Add a Chinese open-source LLM Baichuan2 (7B/13B) support.

**Collection**: NLP

# Changelog 
- Add Baichuan2 model configuration.
- Add Baichuan2 hf-to-nemo conversion.

# Usage
```python
# Model conversion from huggingface to nemo
python ./scripts/nlp_language_modeling/convert_hf_baichuan2_to_nemo.py --in-file '/mount/data/Baichuan2-7B-Base' --out-file '/mount/data/Baichuan2-7B-Base/baichuan2.nemo'
```
```python
# Inference
python ./examples/nlp/language_modeling/megatron_gpt_eval.py \
    --config-name=megatron_baichuan2_inference \
    gpt_model_file=${BAICHUAN2_NEMO_MODEL} \
    trainer.devices=1 \
    trainer.num_nodes=1 \
    tensor_model_parallel_size=1 pipeline_model_parallel_size=1 
```
```python
# Pretrain
python3 -u examples/nlp/language_modeling/megatron_gpt_pretraining.py \
  --config-name=megatron_baichuan2_config \
  trainer.devices=8 \
  trainer.num_nodes=1 \
  trainer.val_check_interval=100 \
  trainer.precision=bf16 \
  trainer.max_steps=1000 \
  trainer.max_epochs=1 \
  model.megatron_amp_O2=True \
  model.sequence_parallel=True \
  model.tensor_model_parallel_size=2 \
  model.pipeline_model_parallel_size=1 \
  model.global_batch_size=16 \
  model.micro_batch_size=2 
```
```python
# Parameter efficient fine-tuning
python examples/nlp/language_modeling/tuning/megatron_gpt_peft_tuning.py \
    trainer.max_steps=200 \
    trainer.devices=2 \
    trainer.val_check_interval=200 \
    exp_manager.exp_dir=exp \
    exp_manager.name=test \
    model.tensor_model_parallel_size=1 \
    model.restore_from_path=${BAICHUAN2_NEMO_MODEL} \
    model.global_batch_size=32 \
    model.micro_batch_size=4 \
    model.data.train_ds.file_names=${TRAIN} \
    model.data.validation_ds.file_names=${VALID} \
    model.answer_only_loss=True \
    model.peft.peft_scheme="lora" \
    model.data.train_ds.concat_sampling_probabilities=[1.0] 2>&1|tee /mount/data/test.log
```
```python
# SFT
python ./examples/nlp/language_modeling/tuning/megatron_gpt_sft.py \
    trainer.devices=8 \
    trainer.val_check_interval=200 \
    model.restore_from_path=${BAICHUAN2_NEMO_MODEL} \
    model.tensor_model_parallel_size=${TP_SIZE} \
    model.pipeline_model_parallel_size=${PP_SIZE} \
    model.answer_only_loss=True \
    model.global_batch_size=16 \
    model.micro_batch_size=2 \
    model.data.train_ds.file_names=${TRAIN} \
    model.data.train_ds.concat_sampling_probabilities=[1.0] \
    model.data.validation_ds.file_names=${VALID} \
    model.data.validation_ds.global_batch_size=64 \
    model.data.validation_ds.micro_batch_size=8 \
    model.megatron_amp_O2=True \
    exp_manager.exp_dir=exp \
    exp_manager.name=test \
    trainer.precision=bf16-mixed 2>&1|tee /mount/data/log/test.log
```
```python
# peft model evaluation
CUDA_VISIBLE_DEVICES='0,1' python ./examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py \
        trainer.devices=2 \
        trainer.num_nodes=1 \
        trainer.precision=32 \
        model.restore_from_path=${BAICHUAN2_NEMO_MODEL} \
        model.peft.restore_from_path=${LORA_MODEL} \
        model.data.test_ds.file_names=["path/to/your/dataset.jsonl"] \
        model.data.test_ds.global_batch_size=32 \
        model.data.test_ds.micro_batch_size=16 \
        model.tensor_model_parallel_size=1 \
        model.pipeline_model_parallel_size=1 \
        inference.greedy=True \
        inference.outfile_path=..
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)


jenkins

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.