renillhuang commited on
Commit
8f474a2
β€’
1 Parent(s): a9189fd

Update README_ko.md

Browse files
Files changed (1) hide show
  1. README_ko.md +34 -4
README_ko.md CHANGED
@@ -32,7 +32,7 @@
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
- - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
@@ -266,9 +266,39 @@ CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-
266
 
267
  ```
268
 
269
- ## 4.4. μ˜ˆμ‹œ λ…ΈμΆœ
270
 
271
- ### 4.4.1. μž‘λ‹΄
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
272
 
273
  `````
274
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
@@ -295,7 +325,7 @@ Orion-14BοΌšμ˜ˆμ „μ— μž­μ΄λΌλŠ” μ–΄λ¦° μ†Œλ…„μ΄ μžˆμ—ˆλ‹€. κ·ΈλŠ” μž‘μ€ 마
295
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
296
  `````
297
 
298
- ### 4.4.2. ν•œμΌ
299
 
300
  `````
301
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください
 
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
+ - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)[<img src="./assets/imgs/vllm.png" alt="vllm" height="20"/>](#vllm) [<img src="./assets/imgs/llama_cpp.png" alt="llamacpp" height="20"/>](#llama-cpp)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
 
266
 
267
  ```
268
 
269
+ ## 4.4. vLLM 좔둠을 톡해
270
 
271
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
272
+ https://github.com/vllm-project/vllm
273
+
274
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
275
+ https://github.com/vllm-project/vllm/pull/2539
276
+
277
+
278
+ <a name="llama-cpp"></a><br>
279
+ ## 4.5. llama.cpp 좔둠을 톡해
280
+
281
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
282
+ https://github.com/ggerganov/llama.cpp
283
+
284
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
285
+ https://github.com/ggerganov/llama.cpp/pull/5118
286
+
287
+ - GGUF ν˜•μ‹μœΌλ‘œ λ³€ν™˜ν•˜λŠ” 방법
288
+
289
+ ```shell
290
+ python convert-hf-to-gguf.py path/to/Orion-14B-Chat --outfile chat.gguf
291
+ ```
292
+
293
+ - λͺ¨λΈ μΆ”λ‘  방법
294
+
295
+ ```shell
296
+ ./main --frequency-penalty 0.5 --frequency-penalty 0.5 --top-k 5 --top-p 0.9 -m chat.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
297
+ ```
298
+
299
+ ## 4.6. μ˜ˆμ‹œ λ…ΈμΆœ
300
+
301
+ ### 4.6.1. μž‘λ‹΄
302
 
303
  `````
304
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
 
325
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
326
  `````
327
 
328
+ ### 4.6.2. ν•œμΌ
329
 
330
  `````
331
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください