cateto commited on
Commit
c364724
โ€ข
1 Parent(s): 0104ad9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md CHANGED
@@ -1,3 +1,55 @@
1
  ---
2
  license: cc-by-3.0
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-3.0
3
+ language:
4
+ - ko
5
+ pipeline_tag: text-generation
6
  ---
7
+ # korean-gpt-neox-125M
8
+
9
+ ## Model Details
10
+
11
+ ### Model Description
12
+
13
+ <!-- Provide a longer summary of what this model is. -->
14
+
15
+ - **Developed by:** [cateto](http://github.com/cateto)
16
+ - **Model type:** [gpt-neox](https://github.com/EleutherAI/gpt-neox)
17
+ - **Language(s) (NLP):** Korean
18
+
19
+ ## Uses
20
+
21
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
22
+
23
+ ### Direct Use
24
+
25
+ ```python
26
+ # Import the transformers library
27
+ from transformers import AutoModelForCausalLM, AutoTokenizer
28
+
29
+ tokenizer = AutoTokenizer.from_pretrained("cateto/korean-gpt-neox-125M")
30
+
31
+ model = AutoModelForCausalLM.from_pretrained("cateto/korean-gpt-neox-125M")
32
+
33
+ # Get user input
34
+ user_input = "์šฐ๋ฆฌ๋Š” ์•ž์œผ๋กœ ๋”๋‚˜์€ ๋ฏธ๋ž˜๋ฅผ"
35
+
36
+ # Encode the prompt using the tokenizer
37
+ input_ids = tokenizer.encode(user_input, return_tensors="pt")
38
+
39
+ # Generate chatbot output using the model
40
+ output_ids = model.generate(
41
+ input_ids,
42
+ num_beams=4,
43
+ repetition_penalty=1.5,
44
+ no_repeat_ngram_size=3
45
+ )
46
+
47
+ # Decode chatbot output ids as text
48
+ bot_output = tokenizer.decode(output_ids.tolist()[0], skip_special_tokens=True)
49
+
50
+ # Print chatbot output
51
+ print(f"์ถœ๋ ฅ ## ", bot_output)
52
+
53
+ # ์ถœ๋ ฅ ## ์šฐ๋ฆฌ๋Š” ์•ž์œผ๋กœ ๋”๋‚˜์€ ๋ฏธ๋ž˜๋ฅผ ํ–ฅํ•ด ๋‚˜์•„๊ฐˆ ์ˆ˜ ์žˆ๋‹ค.
54
+ ```
55
+