---
language:
  - en
  - ja
library_name: transformers
pipeline_tag: text-generation
license: llama3
model_type: llama
---

# Llama3-Preferred-MedSwallow-70B

## Model Description

Llama3-Preferred-MedSwallow-70B is a finetuned model based on [tokyotech-llm/Llama-3-Swallow-70B-v0.1](https://huggingface.co/tokyotech-llm/Llama-3-Swallow-70B-v0.1), which has undergone continued pretraining on an original corpus of medical-related text.
For more details, please refer to our blog post at https://tech.preferred.jp/ja/blog/llama3-preferred-medswallow-70b/.
The model is released under the [META LLAMA 3 COMMUNITY LICENSE](https://llama.meta.com/llama3/license/).

## Model Performance

The table below shows the performance on the Japanese national medical licensing examinations from 2018 to 2022 ([IgakuQA](https://github.com/jungokasai/IgakuQA)).
| Model ID                                                                                                            |   Average |   2018 |   2019 |   2020 |   2021 |   2022 |
|:--------------------------------------------------------------------------------------------------------------------|-------------------:|-------:|-------:|-------:|-------:|-------:|
| **Llama3-Preferred-MedSwallow-70B**          |              **395.2** |    **407** |    **390** |    **391** |    393 |    **395** |
GPT-4                                                                                                                |              388.8 |    382 |    385 |    387 |    **398** |    392 |
| [Llama-3-Swallow-70B-v0.1](https://huggingface.co/tokyotech-llm/Llama-3-Swallow-70B-v0.1)                                                                         |              348.6 |    353 |    347 |    353 |    345 |    345 |
| [Meta-Llama-3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B)                                                                                 |              334.6 |    353 |    340 |    348 |    314 |    318 |
| [Qwen2-72B](https://huggingface.co/Qwen/Qwen2-72B)                                   |              331.2 |    320 |    325 |    325 |    326 |    360 |
| [gemma-2-27b](https://huggingface.co/google/gemma-2-27b)                                   |              316 |    337 |    298 |    327 |    296 |    322 |
| [Swallow-70b-NVE-hf](https://huggingface.co/tokyotech-llm/Swallow-70b-NVE-hf)                                                  |              291.6 |    283 |    280 |    300 |    295 |    300 |
| [Swallow-MX-8x7b-NVE-v0.1](https://huggingface.co/tokyotech-llm/Swallow-MX-8x7b-NVE-v0.1)                                    |              280.8 |    262 |    273 |    291 |    284 |    294 |
ChatGPT                                                                                                             |              273.2 |    266 |    250 |    266 |    297 |    287 |

## Limitations

The model was developed for research purposes and is not intended for clinical diagnosis.
It is the users responsibility to ensure compliance with applicable rules and regulations.

## Contributors

Preferred Networks, Inc.
- Junichiro Iwasawa
- Keita Suzuki
- Wataru Kawakami

## License

[META LLAMA 3 COMMUNITY LICENSE](https://llama.meta.com/llama3/license/)