mkshing's picture
initial commit
254bcbe
metadata
language:
  - ja
tags:
  - japanese-stablelm
  - causal-lm
pipeline_tag: text-generation
license: apache-2.0

Japanese-StableLM-Instruct-Alpha-7B-v2

japanese-stablelm-icon

"A parrot able to speak Japanese, ukiyoe, edo period" โ€” Stable Diffusion XL

Model Description

japanese-stablelm-instruct-alpha-7b-v2 is a 7B parameter decoder-only language models pre-trained built on top of the Japanese-StableLM-Base-Alpha-7B model and further fine-tuned on various instruction-following datasets.

Usage

First install additional dependencies in requirements.txt:

pip install sentencepiece einops

Then start generating text with japanese-stablelm-instruct-alpha-7b-v2 by using the following code snippet:

import torch
from transformers import LlamaTokenizer, AutoModelForCausalLM

tokenizer = LlamaTokenizer.from_pretrained(
    "novelai/nerdstash-tokenizer-v1", additional_special_tokens=["โ–โ–"]
)
model = AutoModelForCausalLM.from_pretrained(
    "stabilityai/japanese-stablelm-instruct-alpha-7b-v2",
    trust_remote_code=True,
    torch_dtype=torch.float16,
    variant="fp16",
)
model.eval()

if torch.cuda.is_available():
    model = model.to("cuda")

def build_prompt(user_query, inputs="", sep="\n\n### "):
    sys_msg = "ไปฅไธ‹ใฏใ€ใ‚ฟใ‚นใ‚ฏใ‚’่ชฌๆ˜Žใ™ใ‚‹ๆŒ‡็คบใจใ€ๆ–‡่„ˆใฎใ‚ใ‚‹ๅ…ฅๅŠ›ใฎ็ต„ใฟๅˆใ‚ใ›ใงใ™ใ€‚่ฆๆฑ‚ใ‚’้ฉๅˆ‡ใซๆบ€ใŸใ™ๅฟœ็ญ”ใ‚’ๆ›ธใใชใ•ใ„ใ€‚"
    p = sys_msg
    roles = ["ๆŒ‡็คบ", "ๅฟœ็ญ”"]
    msgs = [": \n" + user_query, ": \n"]
    if inputs:
        roles.insert(1, "ๅ…ฅๅŠ›")
        msgs.insert(1, ": \n" + inputs)
    for role, msg in zip(roles, msgs):
        p += sep + role + msg
    return p

# Infer with prompt without any additional input
user_inputs = {
    "user_query": "ไธŽใˆใ‚‰ใ‚ŒใŸใ“ใจใ‚ใ–ใฎๆ„ๅ‘ณใ‚’ๅฐๅญฆ็”Ÿใงใ‚‚ๅˆ†ใ‹ใ‚‹ใ‚ˆใ†ใซๆ•™ใˆใฆใใ ใ•ใ„ใ€‚",
    "inputs": "ๆƒ…ใ‘ใฏไบบใฎใŸใ‚ใชใ‚‰ใš"
}
prompt = build_prompt(**user_inputs)

input_ids = tokenizer.encode(
    prompt, 
    add_special_tokens=False, 
    return_tensors="pt"
)

tokens = model.generate(
    input_ids.to(device=model.device),
    max_new_tokens=256,
    temperature=1,
    top_p=0.95,
    do_sample=True,
)

out = tokenizer.decode(tokens[0][input_ids.shape[1]:], skip_special_tokens=True).strip()
print(out)
"""
ใ€Œๆƒ…ใ‘ใฏไบบใฎใŸใ‚ใชใ‚‰ใšใ€ใฏใ€ใ€Œๆƒ…ใ‘ใ‚’ใ‹ใ‘ใ‚‹ใจใใฎไบบใฎใŸใ‚ใซใชใ‚‰ใชใ„ใ€ใจใ„ใ†ๆ„ๅ‘ณใงใฏใ‚ใ‚Šใพใ›ใ‚“ใ€‚
ใ“ใฎใ“ใจใ‚ใ–ใฏใ€ใ‚‚ใจใ‚‚ใจใ€Œ่ชฐใ‹ใฎใŸใ‚ใซ่กŒๅ‹•ใ™ใ‚‹ใจใใฎ่กŒๅ‹•ใŒๅ›žใ‚Šๅ›žใฃใฆ่‡ชๅˆ†ใซ่ฟ”ใฃใฆใใ‚‹ใ€ใจใ„ใ†ใ“ใจใ‚’่ชฌใ„ใŸใ“ใจใ‚ใ–ใงใ™ใ€‚
"""

Model Details

  • Model type: japanese-stablelm-instruct-alpha-7b-v2 is an auto-regressive language model based on the NeoX transformer architecture.
  • Language(s): Japanese
  • Library: GPT-NeoX
  • License: This model is licensed under Apache License, Version 2.0.

Training

Parameters Hidden Size Layers Heads Sequence Length
7B 4096 32 32 1024

Training Dataset

japanese-stablelm-instruct-alpha-7b-v2 is fine-tuned on a combination of following datasets:

Use and Limitations

Intended Use

This model is intended to be used by the open-source community in chat-like applications in adherence with Apache-2.0 license.

Limitations and bias

Although the aforementioned datasets help to steer the base language models into "safer" distributions of text, not all biases and toxicity can be mitigated through fine-tuning. We ask that users be mindful of such potential issues that can arise in generated responses. Do not treat model outputs as substitutes for human judgment or as sources of truth. Please use responsibly.

Authors

Acknowledgements

We are utilizing the v1 version of the novelai-tokenizer, introduced by NovelAI, because it processes both Japanese and English text both effectively and efficiently. We extend our gratitude to NovelAI for allowing us to use their remarkable work. For more details about the tokenizer, please refer to their blog post.

We are grateful for the contributions of the EleutherAI Polyglot-JA team in helping us to collect a large amount of pre-training data in Japanese. Polyglot-JA members includes Hyunwoong Ko (Project Lead), Fujiki Nakamura (originally started this project when he committed to the Polyglot team), Yunho Mo, Minji Jung, KeunSeok Im, and Su-Kyeong Jang.

We are also appreciative of AI Novelist/Sta (Bit192, Inc.) and the numerous contributors from Stable Community Japan for assisting us in gathering a large amount of high-quality Japanese textual data for model training.

How to cite

@misc{JapaneseStableLMInstructAlpha7Bv2, 
      url={[https://huggingface.co/stabilityai/japanese-stablelm-instruct-alpha-7b-v2](https://huggingface.co/stabilityai/japanese-stablelm-instruct-alpha-7b-v2)}, 
      title={Japanese StableLM Instruct Alpha 7B v2}, 
      author={Lee, Meng and Nakamura, Fujiki and Shing, Makoto and McCann, Paul and Akiba, Takuya and Orii, Naoki}
}

Citations

@misc{alpaca,
  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
  title = {Stanford Alpaca: An Instruction-following LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
}
@software{gpt-neox-library,
  title = {{GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch}},
  author = {Andonian, Alex and Anthony, Quentin and Biderman, Stella and Black, Sid and Gali, Preetham and Gao, Leo and Hallahan, Eric and Levy-Kramer, Josh and Leahy, Connor and Nestler, Lucas and Parker, Kip and Pieler, Michael and Purohit, Shivanshu and Songz, Tri and Phil, Wang and Weinbach, Samuel},
  url = {https://www.github.com/eleutherai/gpt-neox},
  doi = {10.5281/zenodo.5879544},
  month = {8},
  year = {2021},
  version = {0.0.1},
}