Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -15,7 +15,8 @@ widget:
|
|
| 15 |
# NewsGPT
|
| 16 |
|
| 17 |
## Model Description
|
| 18 |
-
The model is
|
|
|
|
| 19 |
|
| 20 |
## Training Data
|
| 21 |
The model's training data consists of ~13,000,000 English articles from ~90 outlets, which each consists of a headline (title) and a subheading (description). The articles were collected from the [Sciride News Mine](http://sciride.org/news.html), after which some additional cleaning was performed on the data, such as removing duplicate articles and removing repeated "outlet tags" appearing before or after headlines such as "| Daily Mail Online".
|
|
|
|
| 15 |
# NewsGPT
|
| 16 |
|
| 17 |
## Model Description
|
| 18 |
+
The model is similar to [gpt2](https://huggingface.co/gpt2) in that it shares its size, architecture, tokenizer algorithm and Causal Language Modeling objective.
|
| 19 |
+
The model parameters of a [GPT2LMHeadModel](https://huggingface.co/docs/transformers/v4.26.1/en/model_doc/gpt2#transformers.GPT2LMHeadModel) model were randomly initialized and pre-trained from scratch using a dataset consisting only of news.
|
| 20 |
|
| 21 |
## Training Data
|
| 22 |
The model's training data consists of ~13,000,000 English articles from ~90 outlets, which each consists of a headline (title) and a subheading (description). The articles were collected from the [Sciride News Mine](http://sciride.org/news.html), after which some additional cleaning was performed on the data, such as removing duplicate articles and removing repeated "outlet tags" appearing before or after headlines such as "| Daily Mail Online".
|