Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -18,9 +18,10 @@ These factors are output in the form of percentiles based on text the model has
|
|
| 18 |
This tool is not 100% accurate and can incorrectly flag texts as human or AI written when they are not, so it should not be used as a sole measure for cheating detection.
|
| 19 |
|
| 20 |
# Usage
|
| 21 |
-
To run this project, visit https://huggingface.co/spaces/atyshka/ai-detector. The
|
| 22 |
On the right, you will see 3 outputs:
|
| 23 |
|
|
|
|
| 24 |
First, you will see the prediction from the model (Human or AI) and the confidence score.
|
| 25 |
Next, you will see the factors that contributed to this decision.
|
| 26 |
The percentages do not represent how much each feature contributed to the decision, rather, they are percentiles indicating whether the feature is high or low compared to other text the model has seen.
|
|
@@ -67,7 +68,7 @@ I hope that this added interpretability makes the model less of a black box for
|
|
| 67 |
The LLM detector does not work well with short spans of text, as there is not sufficient data to make a strong inference.
|
| 68 |
It also works best in the original domain of Wikipedia generation, as other domains such as fiction may contain different distributions of word length and perplexity.
|
| 69 |
Text that an LLM has paraphrased from a vocabulary-rich sample is particularly hard to detect, because the model will reuse these high-perplexity words.
|
| 70 |
-
In general, longer prompts fool the classifier more, because they
|
| 71 |
I also suspect that state-of-the-art approaches using GPT 3.5/4 or PaLM may be harder to detect than the much simpler Curie GPT-3 model.
|
| 72 |
Ideally, I would use these models for training my classifier, however, this would incur significant expense.
|
| 73 |
Finally, the classifier can be tricked with some simple modification of LLM-written text, by adding rare words or long/short sentences.
|
|
|
|
| 18 |
This tool is not 100% accurate and can incorrectly flag texts as human or AI written when they are not, so it should not be used as a sole measure for cheating detection.
|
| 19 |
|
| 20 |
# Usage
|
| 21 |
+
To run this project, visit https://huggingface.co/spaces/atyshka/ai-detector. The interface allows you to enter any text and click "Submit" to evaluate it.
|
| 22 |
On the right, you will see 3 outputs:
|
| 23 |
|
| 24 |
+
|
| 25 |
First, you will see the prediction from the model (Human or AI) and the confidence score.
|
| 26 |
Next, you will see the factors that contributed to this decision.
|
| 27 |
The percentages do not represent how much each feature contributed to the decision, rather, they are percentiles indicating whether the feature is high or low compared to other text the model has seen.
|
|
|
|
| 68 |
The LLM detector does not work well with short spans of text, as there is not sufficient data to make a strong inference.
|
| 69 |
It also works best in the original domain of Wikipedia generation, as other domains such as fiction may contain different distributions of word length and perplexity.
|
| 70 |
Text that an LLM has paraphrased from a vocabulary-rich sample is particularly hard to detect, because the model will reuse these high-perplexity words.
|
| 71 |
+
In general, longer prompts fool the classifier more, because they significantly alter the word distribution.
|
| 72 |
I also suspect that state-of-the-art approaches using GPT 3.5/4 or PaLM may be harder to detect than the much simpler Curie GPT-3 model.
|
| 73 |
Ideally, I would use these models for training my classifier, however, this would incur significant expense.
|
| 74 |
Finally, the classifier can be tricked with some simple modification of LLM-written text, by adding rare words or long/short sentences.
|