CountingMstar commited on
Commit
2914830
β€’
1 Parent(s): e90a1b6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md CHANGED
@@ -82,4 +82,82 @@ You can load the trained model through the training process described above and
82
  Thank you.
83
 
84
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
82
  Thank you.
83
 
84
 
85
+ ---
86
+ # AI Tutor BERT (인곡지λŠ₯ κ³Όμ™Έ μ„ μƒλ‹˜ BERT)
87
+ 이 λͺ¨λΈμ€ 인곡지λŠ₯(AI) κ΄€λ ¨ μš©μ–΄ 및 μ„€λͺ…을 νŒŒμΈνŠœλ‹(fine-tuning)ν•œ BERT λͺ¨λΈμž…λ‹ˆλ‹€.
88
+
89
+
90
+ 졜근 인곡지λŠ₯에 κ΄€ν•œ 관심이 λ†’μ•„μ§€λ©΄μ„œ λ§Žμ€ μ‚¬λžŒμ΄ 인곡지λŠ₯ κ΄€λ ¨ μˆ˜μ—… 및 ν”„λ‘œμ νŠΈλ₯Ό μ§„ν–‰ν•˜κ³  μžˆμŠ΅λ‹ˆλ‹€. κ·ΈλŸ¬λ‚˜ 인곡지λŠ₯ κ΄€λ ¨ λŒ€ν•™μ›μƒμœΌλ‘œμ„œ μ΄λŸ¬ν•œ μˆ˜μš”μ— λΉ„ν•΄ 인곡지λŠ₯ μ΄ˆλ³΄μžλ“€μ΄ 잘 μ•Œμ•„λ“€μ„ 수 μžˆλŠ” μœ μš©ν•œ μžλ£ŒλŠ” ν”μΉ˜ μ•ŠμŠ΅λ‹ˆλ‹€. λ”λΆˆμ–΄ 각자의 μˆ˜μ€€κ³Ό 뢄야에 κ°œμΈν™”λœ κ°•μ˜ λ˜ν•œ λΆ€μ‘±ν•œ μƒν™©μ΄μ–΄μ„œ λ§Žμ€ μ‚¬λžŒλ“€μ΄ 인곡지λŠ₯ ν•™μŠ΅μ„ μ‹œμž‘ν•˜κΈ° μ–΄λ €μ›Œν•˜κ³  μžˆμŠ΅λ‹ˆλ‹€. μ΄λŸ¬ν•œ 문제λ₯Ό ν•΄κ²°ν•˜κ³ μž, 저희 νŒ€μ€ 인곡지λŠ₯ μš©μ–΄ λ„λ©”μΈμ—μ„œ κ³Όμ™Έ μ„ μƒλ‹˜ 역할을 ν•˜λŠ” μ–Έμ–΄λͺ¨λΈμ„ λ§Œλ“€μ—ˆμŠ΅λ‹ˆλ‹€. λͺ¨λΈμ˜ μ’…λ₯˜, ν•™μŠ΅ 데이터셋, μ‚¬μš©λ²• 등이 μ•„λž˜μ— μ„€λͺ…λ˜μ–΄ μžˆμœΌλ‹ˆ μžμ„Ένžˆ μ½μ–΄λ³΄μ‹œκ³ , κΌ­ μ‚¬μš©ν•΄ λ³΄μ‹œκΈ° λ°”λžλ‹ˆλ‹€.
91
+
92
+
93
+ ## Model
94
+ https://huggingface.co/bert-base-uncased
95
+
96
+
97
+ λͺ¨λΈμ˜ 경우 μžμ—°μ–΄ 처리 λͺ¨λΈ 쀑 κ°€μž₯ 유λͺ…ν•œ Googleμ—μ„œ κ°œλ°œν•œ BERTλ₯Ό μ‚¬μš©ν–ˆμŠ΅λ‹ˆλ‹€. μžμ„Έν•œ μ„€λͺ…은 μœ„ μ‚¬μ΄νŠΈλ₯Ό μ°Έκ³ ν•˜μ‹œκΈ° λ°”λžλ‹ˆλ‹€. μ§ˆμ˜μ‘λ‹΅μ΄ 주인 κ³Όμ™Έ μ„ μƒλ‹˜λ‹΅κ²Œ, BERT μ€‘μ—μ„œλ„ μ§ˆμ˜μ‘λ‹΅μ— νŠΉν™”λœ Question and Answering λͺ¨λΈμ„ μ‚¬μš©ν•˜μ˜€μŠ΅λ‹ˆλ‹€. λΆˆλŸ¬μ˜€λŠ” 법은 λ‹€μŒκ³Ό κ°™μŠ΅λ‹ˆλ‹€.
98
+ ```
99
+ from transformers import BertForQuestionAnswering
100
+
101
+ model = BertForQuestionAnswering.from_pretrained("bert-base-uncased")
102
+ ```
103
+
104
+ ## Dataset
105
+ ### Wikipedia
106
+ https://en.wikipedia.org/wiki/Main_Page
107
+ ### activeloop
108
+ https://www.activeloop.ai/resources/glossary/arima-models/
109
+ ### Adrien Beaulieu
110
+ https://product.house/100-ai-glossary-terms-explained-to-the-rest-of-us/
111
+
112
+
113
+ ```
114
+ Context: 'Feature engineering or feature extraction or feature discovery is the process of extracting features (characteristics, properties, attributes) from raw data. Due to deep learning networks, such as convolutional neural networks, that are able to learn features by themselves, domain-specific-based feature engineering has become obsolete for vision and speech processing. Other examples of features in physics include the construction of dimensionless numbers such as Reynolds number in fluid dynamics; then Nusselt number in heat transfer; Archimedes number in sedimentation; construction of first approximations of the solution such as analytical strength of materials solutions in mechanics, etc..'
115
+
116
+ Question: 'What is large language model?'
117
+
118
+ Answer: 'A large language model (LLM) is a type of language model notable for its ability to achieve general-purpose language understanding and generation.'
119
+ ```
120
+
121
+
122
+ ν•™μŠ΅ 데이터셋은 인곡지λŠ₯ κ΄€λ ¨ λ¬Έλ§₯, 질문, 그리고 응닡 μ΄λ ‡κ²Œ 3κ°€μ§€λ‘œ ꡬ성이 λ˜μ–΄μžˆμŠ΅λ‹ˆλ‹€. 응닡(μ •λ‹΅) λ°μ΄ν„°λŠ” λ¬Έλ§₯ 데이터 μ•ˆμ— ν¬ν•¨λ˜μ–΄ 있고, λ¬Έλ§₯ λ°μ΄ν„°μ˜ λ¬Έμž₯ μˆœμ„œλ₯Ό λ°”κΏ”μ£Όμ–΄ 데이터λ₯Ό μ¦κ°•ν•˜μ˜€μŠ΅λ‹ˆλ‹€. 질문 λ°μ΄ν„°λŠ” μ£Όμ œκ°€ λ˜λŠ” 인곡지λŠ₯ μš©μ–΄λ‘œ μ„€μ •ν–ˆμŠ΅λ‹ˆλ‹€. μœ„μ˜ μ˜ˆμ‹œλ₯Ό λ³΄μ‹œλ©΄ μ΄ν•΄ν•˜μ‹œκΈ° νŽΈν•˜μ‹€ κ²λ‹ˆλ‹€. 총 데이터 μˆ˜λŠ” 3300μ—¬ 개둜 data 폴더에 pickle 파일 ν˜•νƒœλ‘œ μ €μž₯λ˜μ–΄ 있고, λ°μ΄ν„°λŠ” Wikipedia 및 λ‹€λ₯Έ μ‚¬μ΄νŠΈλ“€μ„ μ—μ„œ html을 μ΄μš©ν•˜μ—¬ μΆ”μΆœ 및 κ°€κ³΅ν•˜μ—¬ μ œμž‘ν•˜μ˜€μŠ΅λ‹ˆλ‹€. ν•΄λ‹Ή μΆœμ²˜λŠ” μœ„μ™€ κ°™μŠ΅λ‹ˆλ‹€.
123
+
124
+
125
+ ## Training and Result
126
+ https://github.com/CountingMstar/AI_BERT/blob/main/MY_AI_BERT_final.ipynb
127
+
128
+
129
+ ν•™μŠ΅ 방식은 data ν΄λ”μ˜ 데이터와 BERT Question and Answering λͺ¨λΈμ„ λΆˆμ–΄μ™€ μ§„ν–‰λ©λ‹ˆλ‹€. μžμ„Έν•œ λͺ¨λΈ ν•™μŠ΅ 및 μ‚¬μš©λ²•μ€ μœ„μ˜ 링크에 μ„€λͺ…λ˜μ–΄ μžˆμŠ΅λ‹ˆλ‹€.
130
+
131
+ ```
132
+ N_EPOCHS = 10
133
+ optim = AdamW(model.parameters(), lr=5e-5)
134
+ ```
135
+
136
+
137
+ 에포크(epoch)λŠ” 10을 μ‚¬μš©ν–ˆμœΌλ©°, μ•„λ‹΄ μ˜΅ν‹°λ§ˆμ΄μ Έμ™€ λŸ¬λ‹λ ˆμ΄νŠΈλŠ” 5e-5λ₯Ό μ‚¬μš©ν–ˆμŠ΅λ‹ˆλ‹€.
138
+
139
+
140
+
141
+ <img src="https://github.com/CountingMstar/AI_BERT/assets/90711707/72142ff8-f5c8-47ea-9f19-1e6abb4072cd" width="500" height="400"/>
142
+ <img src="https://github.com/CountingMstar/AI_BERT/assets/90711707/2dd78573-34eb-4ce9-ad4d-2237fc7a5b1e" width="500" height="400"/>
143
+
144
+
145
+ κ²°κ³ΌλŠ” μœ„ κ·Έλž˜ν”„λ“€κ³Ό 같이 λ§ˆμ§€λ§‰ 에포크 κΈ°μ€€ loss = 6.917126256477786, accuracy = 0.9819078947368421둜 μƒλ‹Ήνžˆ ν•™μŠ΅μ΄ 잘 된 λͺ¨μŠ΅μ„ λ³΄μ—¬μ€λ‹ˆλ‹€.
146
+
147
+
148
+
149
+ ## How to use?
150
+
151
+
152
+ ```
153
+ model = torch.load("./models/AI_BERT_final_10.pth")
154
+ ```
155
+
156
+ μœ„ ν•™μŠ΅ 과정을 톡해 ν•™μŠ΅λœ λͺ¨λΈμ„ λΆˆλŸ¬μ™€ μ‚¬μš©ν•˜μ‹œλ©΄ λ©λ‹ˆλ‹€.
157
+
158
+
159
+ κ°μ‚¬ν•©λ‹ˆλ‹€.
160
+
161
+
162
+
163
  Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference