renillhuang commited on
Commit
e67931b
β€’
1 Parent(s): b8a6cac

Update README_ko.md

Browse files
Files changed (1) hide show
  1. README_ko.md +38 -4
README_ko.md CHANGED
@@ -32,7 +32,7 @@
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
- - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
@@ -267,9 +267,43 @@ CUDA_VISIBLE_DEVICES=0 python demo/text_generation.py --model OrionStarAI/Orion-
267
 
268
  ```
269
 
270
- ## 4.4. μ˜ˆμ‹œ λ…ΈμΆœ
271
 
272
- ### 4.4.1. μž‘λ‹΄
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
273
 
274
  `````
275
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
@@ -296,7 +330,7 @@ Orion-14BοΌšμ˜ˆμ „μ— μž­μ΄λΌλŠ” μ–΄λ¦° μ†Œλ…„μ΄ μžˆμ—ˆλ‹€. κ·ΈλŠ” μž‘μ€ 마
296
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
297
  `````
298
 
299
- ### 4.4.2. ν•œμΌ
300
 
301
  `````
302
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください
 
32
  - [πŸ“– λͺ¨ν˜• μ†Œκ°œ](#model-introduction)
33
  - [πŸ”— λ‹€μš΄λ‘œλ“œ 경둜](#model-download)
34
  - [πŸ”– 평가결과](#model-benchmark)
35
+ - [πŸ“Š λͺ¨ν˜• 좔리](#model-inference)[<img src="./assets/imgs/vllm.png" alt="vllm" height="20"/>](#vllm) [<img src="./assets/imgs/llama_cpp.png" alt="llamacpp" height="20"/>](#llama-cpp)
36
  - [πŸ“œ μ„±λͺ… ν•©μ˜](#declarations-license)
37
  - [πŸ₯‡ κΈ°μ—… μ†Œκ°œ](#company-introduction)
38
 
 
267
 
268
  ```
269
 
270
+ ## 4.4. vLLM 좔둠을 톡해
271
 
272
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
273
+ https://github.com/vllm-project/vllm
274
+
275
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
276
+ https://github.com/vllm-project/vllm/pull/2539
277
+
278
+
279
+ <a name="llama-cpp"></a><br>
280
+ ## 4.5. llama.cpp 좔둠을 톡해
281
+
282
+ - ν”„λ‘œμ νŠΈ μ£Όμ†Œ<br>
283
+ https://github.com/ggerganov/llama.cpp
284
+
285
+ - ν’€ λ¦¬ν€˜μŠ€νŠΈ<br>
286
+ https://github.com/ggerganov/llama.cpp/pull/5118
287
+
288
+ - GGUF ν˜•μ‹μœΌλ‘œ λ³€ν™˜ν•˜λŠ” 방법
289
+
290
+ ```shell
291
+ python convert-hf-to-gguf.py path/to/Orion-14B-Chat --outfile chat.gguf
292
+ ```
293
+
294
+ - λͺ¨λΈ μΆ”λ‘  방법
295
+
296
+ ```shell
297
+ ./main --frequency-penalty 0.5 --frequency-penalty 0.5 --top-k 5 --top-p 0.9 -m chat.gguf -p "Building a website can be done in 10 simple steps:\nStep 1:" -n 400 -e
298
+ ```
299
+
300
+
301
+
302
+
303
+
304
+ ## 4.6. μ˜ˆμ‹œ λ…ΈμΆœ
305
+
306
+ ### 4.6.1. μž‘λ‹΄
307
 
308
  `````
309
  μ‚¬μš©μžοΌšμ•ˆλ…•,이름이 λ­μ˜ˆμš”
 
330
  이 μ΄μ•ΌκΈ°λŠ” μ €ν¬μ—κ²Œ μš©κΈ°μ™€ 결심이 μžˆλ‹€λ©΄ λͺ¨λ“  어렀움을 κ·Ήλ³΅ν•˜κ³  μžμ‹ μ˜ κΏˆμ„ 이룰 수 μžˆλ‹€λŠ” 것을 μ•Œλ €μ€€λ‹€.
331
  `````
332
 
333
+ ### 4.6.2. ν•œμΌ
334
 
335
  `````
336
  η”¨ζˆ·οΌšθ‡ͺ己を紹介してください