DontPlanToEnd
commited on
Commit
•
1b85b20
1
Parent(s):
e61cdd9
Update app.py
Browse files
app.py
CHANGED
@@ -218,9 +218,9 @@ with GraInter:
|
|
218 |
<h2 style="margin-bottom: 0; font-size: 1.8em;">About</h2>
|
219 |
<strong>UGI:</strong> Uncensored General Intelligence. A measurement of the amount of uncensored/controversial information an LLM knows and is willing to tell the user. It is calculated from the average score of 5 subjects LLMs commonly refuse to talk about. The leaderboard is made of roughly 65 questions/tasks, measuring both willingness to answer and accuracy in fact-based controversial questions. I'm choosing to keep the questions private so people can't train on them and devalue the leaderboard.
|
220 |
|
221 |
-
**W/10:** Willingness/10. A more narrow subset of the UGI questions,
|
222 |
<br>
|
223 |
-
**I/10:** Intelligence/10.
|
224 |
<br><br>
|
225 |
A high UGI but low W/10 could mean for example that the model can provide a lot of accurate sensitive information, but will refuse to form the information into something it sees as dangerous. Or that it answers questions correctly, but appends a paragraph to its answer explaining why the question is immoral to ask.
|
226 |
<br><br>
|
|
|
218 |
<h2 style="margin-bottom: 0; font-size: 1.8em;">About</h2>
|
219 |
<strong>UGI:</strong> Uncensored General Intelligence. A measurement of the amount of uncensored/controversial information an LLM knows and is willing to tell the user. It is calculated from the average score of 5 subjects LLMs commonly refuse to talk about. The leaderboard is made of roughly 65 questions/tasks, measuring both willingness to answer and accuracy in fact-based controversial questions. I'm choosing to keep the questions private so people can't train on them and devalue the leaderboard.
|
220 |
|
221 |
+
**W/10:** Willingness/10. A more narrow subset of the UGI questions, solely focused on measuring how far a model can be pushed before going against its instructions, refusing to answer, or adding an ethical disclaimer to its response.
|
222 |
<br>
|
223 |
+
**I/10:** Intelligence/10. The average score of the UGI questions with the highest correlation with parameter size. This metric shows how much a model's knowledge and reasoning play a role in its UGI score.
|
224 |
<br><br>
|
225 |
A high UGI but low W/10 could mean for example that the model can provide a lot of accurate sensitive information, but will refuse to form the information into something it sees as dangerous. Or that it answers questions correctly, but appends a paragraph to its answer explaining why the question is immoral to ask.
|
226 |
<br><br>
|