TomatenMarc commited on
Commit
87a8dbd
1 Parent(s): 2476254

Upload 10 files

Browse files
Files changed (1) hide show
  1. README.md +12 -13
README.md CHANGED
@@ -141,14 +141,13 @@ Parameters of the fit()-Method:
141
  "lr": 4e-05
142
  },
143
  "scheduler": "WarmupLinear",
144
- "warmup_steps": 66,
145
- "weight_decay": 0.06
146
  }
147
  ```
148
 
149
  ## Evaluation
150
 
151
- We applied a 6-fold (In-Topic) cross-validation method to demonstrate WRAP's optimal performance. This involved using the same dataset and parameters
152
  described in the *Training* section, where we trained on k-1 splits and made predictions using the kth split.
153
 
154
  Additionally, we assessed its ability to generalize across the 6 topics (Cross-Topic) of TACO. Each of the k topics was utilized for testing, while
@@ -156,19 +155,19 @@ the remaining k-1 topics were used for training purposes.
156
 
157
  In total, the WRAP classifier performs as follows:
158
 
159
- ### Content Management
160
 
161
- | Macro-F1 | Inference | Information | Multiclass |
162
- |-------------|-----------|-------------|------------|
163
- | In-Topic | 87.71% | 85.34% | 75.80% |
164
- | Cross-Topic | 86.71% | 84.58% | 73.92% |
165
 
166
- ### Classification
167
 
168
- | Micro-F1 | Reason | Statement | Notification | None |
169
- |-------------|--------|-----------|--------------|--------|
170
- | In-Topic | 77.82% | 61.10% | 80.56% | 83.71% |
171
- | Cross-Topic | 76.52% | 58.99% | 78.43% | 81.73% |
172
 
173
  # Environmental Impact
174
 
 
141
  "lr": 4e-05
142
  },
143
  "scheduler": "WarmupLinear",
144
+ "warmup_steps": 66
 
145
  }
146
  ```
147
 
148
  ## Evaluation
149
 
150
+ We applied a 6-fold (Closed-Topic) cross-validation method to demonstrate WRAP's optimal performance. This involved using the same dataset and parameters
151
  described in the *Training* section, where we trained on k-1 splits and made predictions using the kth split.
152
 
153
  Additionally, we assessed its ability to generalize across the 6 topics (Cross-Topic) of TACO. Each of the k topics was utilized for testing, while
 
155
 
156
  In total, the WRAP classifier performs as follows:
157
 
158
+ ### Binary Classification Tasks
159
 
160
+ | Macro-F1 | Inference | Information | Multi-Class |
161
+ |--------------|-----------|-------------|-------------|
162
+ | Closed-Topic | 86.62% | 86.30% | 75.29% |
163
+ | Cross-Topic | 86.27% | 84.90% | 73.54% |
164
 
165
+ ### Multi-Class Classification Task
166
 
167
+ | Micro-F1 | Reason | Statement | Notification | None |
168
+ |--------------|--------|-----------|--------------|--------|
169
+ | Closed-Topic | 78.14% | 60.96% | 79.36% | 82.72% |
170
+ | Cross-Topic | 77.05% | 58.33% | 78.45% | 80.33% |
171
 
172
  # Environmental Impact
173