renillhuang commited on
Commit
d28971c
β€’
1 Parent(s): 386af23

Update README_ko.md

Browse files
Files changed (1) hide show
  1. README_ko.md +33 -4
README_ko.md CHANGED
@@ -32,7 +32,7 @@
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
- - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
@@ -265,10 +265,39 @@ CUDA_VISIBLE_DEVICES=0 python demo/text_generation_base.py --model OrionStarAI/O
265
  CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-14B-Chat --tokenizer OrionStarAI/Orion-14B-Chat --prompt μ•ˆλ…•. 이름이 λ­μ˜ˆμš”
266
 
267
  ```
 
268
 
269
- ## 4.4. μ˜ˆμ‹œ λ…ΈμΆœ
 
270
 
271
- ### 4.4.1. μž‘λ‹΄
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
272
 
273
  `````
274
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
@@ -295,7 +324,7 @@ Orion-14BοΌšμ˜ˆμ „μ— μž­μ΄λΌλŠ” μ–΄λ¦° μ†Œλ…„μ΄ μžˆμ—ˆλ‹€. κ·ΈλŠ” μž‘μ€ 마
295
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
296
  `````
297
 
298
- ### 4.4.2. ν•œμΌ
299
 
300
  `````
301
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください
 
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
+ - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)[<img src="./assets/imgs/vllm.png" alt="vllm" height="20"/>](#vllm) [<img src="./assets/imgs/llama_cpp.png" alt="llamacpp" height="20"/>](#llama-cpp)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
 
265
  CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-14B-Chat --tokenizer OrionStarAI/Orion-14B-Chat --prompt μ•ˆλ…•. 이름이 λ­μ˜ˆμš”
266
 
267
  ```
268
+ ## 4.4. vLLM 좔둠을 톡해
269
 
270
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
271
+ https://github.com/vllm-project/vllm
272
 
273
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
274
+ https://github.com/vllm-project/vllm/pull/2539
275
+
276
+
277
+ <a name="llama-cpp"></a><br>
278
+ ## 4.5. llama.cpp 좔둠을 톡해
279
+
280
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
281
+ https://github.com/ggerganov/llama.cpp
282
+
283
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
284
+ https://github.com/ggerganov/llama.cpp/pull/5118
285
+
286
+ - GGUF ν˜•μ‹μœΌλ‘œ λ³€ν™˜ν•˜λŠ” 방법
287
+
288
+ ```shell
289
+ python convert-hf-to-gguf.py path/to/Orion-14B-Chat --outfile chat.gguf
290
+ ```
291
+
292
+ - λͺ¨λΈ μΆ”λ‘  방법
293
+
294
+ ```shell
295
+ ./main --frequency-penalty 0.5 --frequency-penalty 0.5 --top-k 5 --top-p 0.9 -m chat.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
296
+ ```
297
+
298
+ ## 4.6. μ˜ˆμ‹œ λ…ΈμΆœ
299
+
300
+ ### 4.6.1. μž‘λ‹΄
301
 
302
  `````
303
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
 
324
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
325
  `````
326
 
327
+ ### 4.6.2. ν•œμΌ
328
 
329
  `````
330
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください