Deploy TinyModel1 from GitHub Actions

Browse files

Files changed (8) hide show

README.md +25 -21
artifact.json +20 -5
config.json +1 -1
eval_report.json +209 -0
misclassified_sample.jsonl +100 -0
model.safetensors +1 -1
tokenizer.json +1199 -1199
tokenizer_config.json +2 -2

README.md CHANGED Viewed

@@ -3,14 +3,13 @@ license: apache-2.0
 library_name: transformers
 pipeline_tag: text-classification
 datasets:
-  - ag_news
 language:
   - en
 tags:
   - tiny
   - bert
   - text-classification
-  - ag-news
 ---
 <div align="center">
@@ -19,7 +18,7 @@ tags:
 # TinyModel1
-**TinyModel1** is a compact **encoder** model for **news topic classification**, trained from scratch on the [AG News](https://huggingface.co/datasets/fancyzhx/ag_news) dataset. It targets fast CPU/GPU inference, simple deployment behind a router or API, and use as a **baseline** before larger or domain-specific models.
 ## Links
@@ -35,6 +34,7 @@ tags:
 |:--|:--|
 | **Task** | Text classification (single-label, 4 classes) |
 | **Labels** | World, Sports, Business, Sci/Tech |
 | **Architecture** | Tiny BERT-style encoder (`BertForSequenceClassification`) |
 | **Parameters** | 1,339,268 (~1.34M) |
 | **Max sequence length** | 128 tokens (training & inference) |
@@ -44,13 +44,13 @@ tags:
 ## Model overview
-This release fits a **small footprint** so you can run batch or interactive classification without heavy GPUs. Training uses a WordPiece tokenizer fit on the training split and a shallow BERT stack suited to short news sentences.
 ### **Core capabilities**
-- **Topic routing** — assign one of four coarse news categories for search, feeds, or moderation triage.
-- **Low latency** — small parameter count keeps inference suitable for edge and serverless setups.
-- **Fine-tuning base** — swap labels or add data for your domain while keeping the same architecture.
 ---
@@ -58,8 +58,8 @@ This release fits a **small footprint** so you can run batch or interactive clas
 | Setting | Value |
 |:--|:--|
-| **Train samples** | 3000 |
-| **Eval samples** | 600 |
 | **Epochs** | 2 |
 | **Batch size** | 16 |
 | **Learning rate** | 0.0001 |
@@ -71,10 +71,14 @@ This release fits a **small footprint** so you can run batch or interactive clas
 | Metric | Value |
 |:--|:--|
-| **Eval accuracy** | 0.5583 |
-| **Final train loss** | 1.1562 |
-Metrics are computed on the held-out eval split configured above; treat them as a **sanity-check baseline**, not a production SLA.
 ---
@@ -87,38 +91,38 @@ from transformers import pipeline
 clf = pipeline(
     "text-classification",
-    model="TinyModel1",  # or local path after save
     tokenizer="TinyModel1",
     top_k=None,
 )
-text = "Markets rose after the central bank held rates steady."
 print(clf(text))
 ```
-Use `top_k=None` (or your Transformers version’s equivalent) to obtain scores for **all** labels. Replace `"TinyModel1"` with your Hugging Face model id (for example `HyperlinksSpace/TinyModel1`) when loading from the Hub.
 ---
 ## Training data
-- **Dataset:** [fancyzhx/ag_news](https://huggingface.co/datasets/fancyzhx/ag_news) (4-class news topics).
 - **Preprocessing:** tokenizer trained on training texts; sequences truncated to 128 tokens.
 ---
 ## Intended use
-- Prototyping **routing**, **tagging**, and **dashboard** features over English news-style text.
 - Teaching and benchmarking small-classification setups.
-- Starting point for **domain adaptation** (finance, sports, etc.) with your own labels.
 ---
 ## Limitations
-- **Accuracy** is modest by design; do not rely on it for high-stakes decisions without validation on your data.
-- **English-oriented** news wording; other languages or social-style text may degrade.
-- **Four fixed classes**; not suitable as a general-purpose language model.
 ---

 library_name: transformers
 pipeline_tag: text-classification
 datasets:
+  - fancyzhx/ag_news
 language:
   - en
 tags:
   - tiny
   - bert
   - text-classification
 ---
 <div align="center">
 # TinyModel1
+**TinyModel1** is a compact **encoder** model for **news topic classification, trained on the AG News dataset. It targets fast CPU/GPU inference and use as a baseline.**
 ## Links
 |:--|:--|
 | **Task** | Text classification (single-label, 4 classes) |
 | **Labels** | World, Sports, Business, Sci/Tech |
+| **Dataset** | `fancyzhx/ag_news` |
 | **Architecture** | Tiny BERT-style encoder (`BertForSequenceClassification`) |
 | **Parameters** | 1,339,268 (~1.34M) |
 | **Max sequence length** | 128 tokens (training & inference) |
 ## Model overview
+Trained with a WordPiece tokenizer fit on the training split and a shallow BERT stack. Replace the dataset and labels via `scripts/train_tinymodel1_classifier.py` for your own taxonomy.
 ### **Core capabilities**
+- **Text routing** — assign one class per input for search, feeds, or triage.
+- **Low latency** — small parameter count suits edge and serverless setups.
+- **Fine-tuning base** — swap labels or data for your domain while keeping the same architecture.
 ---
 | Setting | Value |
 |:--|:--|
+| **Train samples (cap)** | 3000 |
+| **Eval samples (cap)** | 600 |
 | **Epochs** | 2 |
 | **Batch size** | 16 |
 | **Learning rate** | 0.0001 |
 | Metric | Value |
 |:--|:--|
+| **Accuracy** | 0.5150 |
+| **Macro F1** | 0.4262 |
+| **Weighted F1** | 0.4240 |
+| **Final train loss** | 1.1535 |
+Per-class F1 and the confusion matrix are saved in `eval_report.json` in this model directory.
+Metrics are computed on the held-out eval subset (see `eval_report.json` → `reproducibility`); treat them as a **sanity-check baseline**, not a production SLA.
 ---
 clf = pipeline(
     "text-classification",
+    model="TinyModel1",
     tokenizer="TinyModel1",
     top_k=None,
 )
+text = "Your input text here."
 print(clf(text))
 ```
+Use `top_k=None` (or your Transformers version’s equivalent) for scores for **all** labels. Replace `"TinyModel1"` with your Hub model id when loading from the Hub.
 ---
 ## Training data
+- **Dataset:** `fancyzhx/ag_news` (text column mapped for training; see `artifact.json`).
 - **Preprocessing:** tokenizer trained on training texts; sequences truncated to 128 tokens.
 ---
 ## Intended use
+- Prototyping **routing**, **tagging**, and **dashboard** features over short text.
 - Teaching and benchmarking small-classification setups.
+- Starting point for **domain adaptation** with your own labels.
 ---
 ## Limitations
+- **Accuracy** is modest by design; validate on your data before high-stakes use.
+- **Not a general-purpose language model** — classification head only; for generation use an LM.
+- **Tokenizer and labels** are tied to this training run; mismatched inputs may degrade.
 ---

artifact.json CHANGED Viewed

@@ -1,8 +1,13 @@
 {
   "name": "TinyModel1",
-  "version": "0.2.0",
   "task": "text-classification",
-  "dataset": "ag_news",
   "base_model": "tinymodel1-bert-scratch",
   "labels": [
     "World",
@@ -10,12 +15,22 @@
     "Business",
     "Sci/Tech"
   ],
-  "eval_accuracy": 0.5583,
-  "train_loss": 1.1562,
   "num_parameters": 1339268,
   "max_train_samples": 3000,
   "max_eval_samples": 600,
   "epochs": 2,
   "batch_size": 16,
-  "learning_rate": 0.0001
 }

 {
   "name": "TinyModel1",
+  "version": "0.3.0",
   "task": "text-classification",
+  "dataset": "fancyzhx/ag_news",
+  "dataset_config": null,
+  "train_split": "train",
+  "eval_split": "test",
+  "text_column": "text",
+  "label_column": "label",
   "base_model": "tinymodel1-bert-scratch",
   "labels": [
     "World",
     "Business",
     "Sci/Tech"
   ],
+  "eval_accuracy": 0.515,
+  "eval_macro_f1": 0.4262,
+  "eval_weighted_f1": 0.424,
+  "eval_per_class_f1": {
+    "World": 0.4268,
+    "Sports": 0.7114,
+    "Business": 0.0,
+    "Sci/Tech": 0.5665
+  },
+  "train_loss": 1.1535,
   "num_parameters": 1339268,
   "max_train_samples": 3000,
   "max_eval_samples": 600,
+  "max_seq_length": 128,
   "epochs": 2,
   "batch_size": 16,
+  "learning_rate": 0.0001,
+  "seed": 42
 }

config.json CHANGED Viewed

@@ -34,7 +34,7 @@
   "pad_token_id": 0,
   "problem_type": "single_label_classification",
   "tie_word_embeddings": true,
-  "transformers_version": "5.5.4",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 8000

   "pad_token_id": 0,
   "problem_type": "single_label_classification",
   "tie_word_embeddings": true,
+  "transformers_version": "5.8.0",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 8000

eval_report.json ADDED Viewed

	@@ -0,0 +1,209 @@

+{
+  "reproducibility": {
+    "seed": 42,
+    "dataset": "fancyzhx/ag_news",
+    "dataset_config": null,
+    "train_split": "train",
+    "eval_split": "test",
+    "text_column": "text",
+    "label_column": "label",
+    "max_train_samples": 3000,
+    "max_eval_samples": 600,
+    "note": "Train and eval rows are the first N after shuffle(seed) of each split; see texts/eval-reproducibility.md."
+  },
+  "metrics": {
+    "accuracy": 0.515,
+    "macro_f1": 0.426181,
+    "weighted_f1": 0.424034,
+    "per_class_f1": {
+      "World": 0.426778,
+      "Sports": 0.711443,
+      "Business": 0.0,
+      "Sci/Tech": 0.566502
+    },
+    "confusion_matrix": [
+      [
+        51,
+        82,
+        0,
+        15
+      ],
+      [
+        1,
+        143,
+        1,
+        2
+      ],
+      [
+        18,
+        13,
+        0,
+        121
+      ],
+      [
+        21,
+        17,
+        0,
+        115
+      ]
+    ],
+    "confusion_matrix_axis": "rows=true class, columns=predicted class",
+    "label_order": [
+      "World",
+      "Sports",
+      "Business",
+      "Sci/Tech"
+    ]
+  },
+  "dataset_quality": {
+    "class_distribution": {
+      "train": {
+        "counts_by_label": {
+          "World": 771,
+          "Sports": 742,
+          "Business": 691,
+          "Sci/Tech": 796
+        },
+        "proportions_by_label": {
+          "World": 0.257,
+          "Sports": 0.247333,
+          "Business": 0.230333,
+          "Sci/Tech": 0.265333
+        },
+        "total": 3000
+      },
+      "eval": {
+        "counts_by_label": {
+          "World": 148,
+          "Sports": 147,
+          "Business": 152,
+          "Sci/Tech": 153
+        },
+        "proportions_by_label": {
+          "World": 0.246667,
+          "Sports": 0.245,
+          "Business": 0.253333,
+          "Sci/Tech": 0.255
+        },
+        "total": 600
+      }
+    }
+  },
+  "error_analysis": {
+    "top_confusions": [
+      {
+        "true_label": "Business",
+        "predicted_label": "Sci/Tech",
+        "count": 121
+      },
+      {
+        "true_label": "World",
+        "predicted_label": "Sports",
+        "count": 82
+      },
+      {
+        "true_label": "Sci/Tech",
+        "predicted_label": "World",
+        "count": 21
+      },
+      {
+        "true_label": "Business",
+        "predicted_label": "World",
+        "count": 18
+      },
+      {
+        "true_label": "Sci/Tech",
+        "predicted_label": "Sports",
+        "count": 17
+      },
+      {
+        "true_label": "World",
+        "predicted_label": "Sci/Tech",
+        "count": 15
+      },
+      {
+        "true_label": "Business",
+        "predicted_label": "Sports",
+        "count": 13
+      },
+      {
+        "true_label": "Sports",
+        "predicted_label": "Sci/Tech",
+        "count": 2
+      },
+      {
+        "true_label": "Sports",
+        "predicted_label": "World",
+        "count": 1
+      },
+      {
+        "true_label": "Sports",
+        "predicted_label": "Business",
+        "count": 1
+      }
+    ]
+  },
+  "calibration": {
+    "max_prob_histogram": {
+      "num_bins": 10,
+      "bins": [
+        {
+          "bin_low": 0.0,
+          "bin_high": 0.1,
+          "count": 0
+        },
+        {
+          "bin_low": 0.1,
+          "bin_high": 0.2,
+          "count": 0
+        },
+        {
+          "bin_low": 0.2,
+          "bin_high": 0.3,
+          "count": 1
+        },
+        {
+          "bin_low": 0.3,
+          "bin_high": 0.4,
+          "count": 24
+        },
+        {
+          "bin_low": 0.4,
+          "bin_high": 0.5,
+          "count": 237
+        },
+        {
+          "bin_low": 0.5,
+          "bin_high": 0.6,
+          "count": 175
+        },
+        {
+          "bin_low": 0.6,
+          "bin_high": 0.7,
+          "count": 163
+        },
+        {
+          "bin_low": 0.7,
+          "bin_high": 0.8,
+          "count": 0
+        },
+        {
+          "bin_low": 0.8,
+          "bin_high": 0.9,
+          "count": 0
+        },
+        {
+          "bin_low": 0.9,
+          "bin_high": 1.0,
+          "count": 0
+        }
+      ],
+      "note": "Each eval example contributes one max softmax probability (winner class)."
+    }
+  },
+  "routing": {
+    "fallback_behavior": "At inference, if the maximum softmax probability is below `min_confidence`, treat the prediction as low-confidence: route to human review, a secondary model, or a safe default class\u2014choose per product.",
+    "min_confidence": null,
+    "comment": "`min_confidence` is not set by training; typical starting range is 0.5\u20130.7 for routing. Tune on a validation set using `max_prob` histogram and error analysis."
+  }
+}

misclassified_sample.jsonl ADDED Viewed

	@@ -0,0 +1,100 @@

+{"text": "Indian board plans own telecast of Australia series The Indian cricket board said on Wednesday it was making arrangements on its own to broadcast next month #39;s test series against Australia, which is under threat because of a raging TV rights dispute.", "true_label": "Sports", "predicted_label": "World", "max_prob": 0.485279}
+{"text": "Stocks Higher on Drop in Jobless Claims A sharp drop in initial unemployment claims and bullish forecasts from Nokia and Texas Instruments sent stocks slightly higher in early trading Thursday.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.445942}
+{"text": "Stocks Higher on Drop in Jobless Claims A sharp drop in initial unemployment claims and bullish forecasts from Nokia and Texas Instruments sent stocks higher in early trading Thursday.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.449751}
+{"text": "REVIEW: 'Half-Life 2' a Tech Masterpiece (AP) AP - It's been six years since Valve Corp. perfected the first-person shooter with \"Half-Life.\" Video games have come a long way since, with better graphics and more options than ever. Still, relatively few games have mustered this one's memorable characters and original science fiction story.", "true_label": "Sci/Tech", "predicted_label": "World", "max_prob": 0.409015}
+{"text": "China's inflation rate slows sharply but problems remain (AFP) AFP - China's inflation rate eased sharply in October as government efforts to cool the economy began to really bite, with food prices, one of the main culprits, showing some signs of slowing, official data showed.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.467996}
+{"text": "ADV: Try Currency Trading Risk-Free 30 Days 24-hour commission-free trading, 100-to-1 leverage of your capital, and Dealbook Fx 2 - our free advanced trading software. Sign up for our free 30-day trial and receive one-on-one training.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.496622}
+{"text": "OPEC Can Raise Output Capacity by 1 Mln Barrels/Day (Update1) The Organization of Petroleum Exporting Countries, which supplies a third of the world #39;s crude oil, can raise production capacity by 1 million barrels a day by year-end, OPEC President Purnomo Yusgiantoro said.", "true_label": "Business", "predicted_label": "World", "max_prob": 0.444693}
+{"text": "Amazon Snaps Up China's Largest Web Retailer (NewsFactor) NewsFactor - Amazon.com (Nasdaq: AMZN) has said it will buy Joyo.com Limited -- a British Virgin Islands company that operates the largest Internet retail Web site in China -- for US #36;75 million.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.496007}
+{"text": "Bookies take bets on new Band Aid Bookies take bets on a new Band Aid single being Christmas No 1, expected to be confirmed by Midge Ure.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.601988}
+{"text": "Microsoft, EU Officials in Procedure Meeting Microsoft (Quote, Chart) and representatives of the European Union #39;s Competition Commission will sit down at the table together on Thursday, but it won #39;t be to eat.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.511439}
+{"text": "Nevada Implements 1st Touch-Screen Voting Machines With Paper-Trail By RACHEL KONRAD     CARSON CITY, Nev. (AP) -- In what could become a model for other states, Nevada voters on Tuesday became the first in the nation to cast ballots in a statewide election on computers that printed paper records of electronic ballots...", "true_label": "Sci/Tech", "predicted_label": "World", "max_prob": 0.520874}
+{"text": "UN aid plane shot at in Ivory Coast Abidjan - A United Nations World Food Programme (WFP) plane was met with gunfire and threats when it arrived in Man, western Ivory Coast, the UN said in a statement on Saturday.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.564946}
+{"text": "Vampire: The Masquerade - Bloodlines Review November 17, 2004 - Most of us who #39;ve been gamers for a while are familiar with the history behind Troika, which was formed from key members of the Black Isle group that made the Fallout, among other talented individuals.", "true_label": "Sci/Tech", "predicted_label": "Sports", "max_prob": 0.467514}
+{"text": "Israel Destroys Refugee Homes, Kills One GAZA CITY, Gaza Strip - A day after a mortar round killed an Israeli-American woman in a nearby settlement, the Israeli army charged into a Palestinian refugee camp Saturday, killing one person and tearing down 35 homes, witnesses and a U.N. aid official said...", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.495682}
+{"text": "Kerry assails Bush troop withdrawal plan (AFP) AFP - Democratic White House hopeful Senator John Kerry warned that President George W. Bush's plan to withdraw 70,000 troops from Europe and Asia would hinder the war on terrorism and embolden North Korea.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.528166}
+{"text": "TSA Deal Overpaid Boeing, Report Says Boeing Co. received at least \\$49 million in excessive profits on a \\$1.2 billion contract to supply explosives-detection systems to hundreds of the nation #39;s airports, the Department of Homeland ", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.463929}
+{"text": "Shares Gain While Oil Holds Near \\$50  LONDON (Reuters) - European stock markets rose and absorbed  three separate share placings on Wednesday, boosted by Wall  Street's strong finish while oil prices held close to \\$50 a  barrel ahead of U.S. oil inventory data.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.489279}
+{"text": "Red Hat replaces CFO Red Hat on Thursday named Charles Peters Jr. as executive vice president and chief financial officer. Peters replaces Kevin Thompson, who unexpectedly announced his resignation in June, a few days before the ", "true_label": "Business", "predicted_label": "Sports", "max_prob": 0.651315}
+{"text": "Jobless Claims Rise Unexpectedly  WASHINGTON (Reuters) - The number of Americans filing  initial claims for jobless pay grew unexpectedly last week to  357,000, Labor Department data showed on Thursday, but an  official said an increase in the week after a public holiday  was typical.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.478942}
+{"text": "Auto Parts Sector Falls on Delphi News Investors sold off shares of auto parts makers Friday after Delphi Corp. issued a profit warning and said it would cut nearly 5 percent of its work force next year.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.492736}
+{"text": "Hasbro's No Conehead Its Saturday Night Live version of \"Trivial Pursuit\" is good strategy for staying ahead of age compression.", "true_label": "Business", "predicted_label": "Sports", "max_prob": 0.627061}
+{"text": "Intel dauphin Otellini becomes king Intel #39;s board has given the go ahead for the long anticipated shift in power from current CEO Craig Barrett to current President Paul Otellini Come May 18, Otellini will take over the chipmaker and become its fifth ever CEO.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.452937}
+{"text": "India's Tata expands regional footprint via NatSteel buyout (AFP) AFP - India's Tata Iron and Steel Company Ltd. took a strategic step to expand its Asian footprint with the announcement it will buy the Asia-Pacific steel operations of Singapore's NatSteel Ltd.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.490368}
+{"text": "Ken Caminiti, 1996 NL MVP, Dies at Age 41 NEW YORK - Ken Caminiti, the 1996 National League MVP who later admitted using steroids during his major league career, died Sunday. He was 41...", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.680263}
+{"text": "The Next Shock: Not Oil, but Debt The American economic ship, which has weathered the recent run-up in crude oil prices, may be more vulnerable to sudden surges in the price of money.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.462241}
+{"text": "One held in Jakarta embassy blast probe Indonesian police said on Friday they had made their first arrest directly linked to last week #39;s deadly embassy bombing in Jakarta, detaining a man who delivered explosives to those blamed for the attack.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.579313}
+{"text": "Oil price rise adds to airlines woes as losses continue The global airline industry is forecast to have made net losses of \\$4.8bn this year, as the rise in the oil price has overwhelmed efforts by carriers to cut costs.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.49329}
+{"text": "United #39;s pension dilemma United Airlines says it likely will end funding for employee pension plans, a move that would be the largest ever default by a US company and could lead to a taxpayer-funded bailout rivaling the savings-and-loan fiasco of the 1980s.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.484973}
+{"text": "UN 'must ignore cloning ban call' The UK's Royal Society urges the UN to ignore a call by President Bush to ban all forms of human cloning.", "true_label": "Sci/Tech", "predicted_label": "World", "max_prob": 0.493441}
+{"text": "UPDATE 1-UPS buys CNF #39;s Menlo Worldwide United Parcel Service Inc. (UPS.N: Quote, Profile, Research) agreed to buy Menlo Worldwide Forwarding, a unit of CNF Inc. (CNF.N: Quote, Profile, Research) , for \\$150 million in ", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.495311}
+{"text": "Hewlett-Packard buys Synstar Palo Alto-based Hewlett-Packard Co. has bought IT services company Synstar plc, of Bracknell, England for about \\$293.3 million. Synstar has some 1,500 customers across Europe, selling IT support for various computer platforms.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.496921}
+{"text": "N.Korea called worst for press freedom Media watchdog Reporters Without Borders has labeled North Korea and Cuba the worst countries in terms of press freedom, with Denmark being the best.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.522785}
+{"text": "Headshake to the SETI Headfake Did the famous screensaver, SETIhome, uncover the first strong evidence for an extraterrestrial signal? The SETI Institute's Seth Shostak discusses how hyperbole can misrepresent the last addition to a list of stellar candidates.", "true_label": "Sci/Tech", "predicted_label": "World", "max_prob": 0.490704}
+{"text": "UnitedHealthcare pays \\$3.5M to settle Medicare fraud claim UnitedHealthcare Insurance Co., a branch of UnitedHealth Group, will pay \\$3.5 million to settle charges that it defrauded the Medicare program, the US Department of Justice announced Monday.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.452892}
+{"text": "Level 3 Acquires Sprint #39;s Wholesale Dial-Up Internet Business Level 3 today announced that it has purchased Sprint #39;s wholesale dial-up Internet access business for \\$34 million in cash. Sprint is one of the largest providers of wholesale dial-up service to ISPs in North America.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.501144}
+{"text": "Dodgers Nip Giants 3-2 in Crucial Series SAN FRANCISCO - Shawn Green can sit out Saturday knowing he was a huge help to the Dodgers during their crucial series against San Francisco. Green hit a two-run homer in Los Angeles' 3-2 victory over the Giants on Friday night, a day before the first baseman will miss a game to observe the Jewish holiday Yom Kippur...", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.662953}
+{"text": "CHUCK JAFFE BOSTON (CBS.MW) -- A lot of people got excited when Fidelity Investments announced recently that it was cutting fees on five index mutual funds.", "true_label": "Business", "predicted_label": "World", "max_prob": 0.461596}
+{"text": "Roundup: Pleasantly Perfect takes Pacific Classic Favored Pleasantly Perfect took charge down the stretch to win by a length in the 14th running of the \\$1 million Pacific Classic yesterday at Del Mar. Pleasantly ", "true_label": "Sports", "predicted_label": "Business", "max_prob": 0.275798}
+{"text": "Update 1: UAL Posts \\$274M Loss, Capping Bad Quarter Record fuel costs and low air fares contributed to a \\$274 million third-quarter loss for United Airlines #39; parent company, which warned again that labor costs must be slashed again soon in order for it to emerge from bankruptcy.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.492365}
+{"text": "Sharon 'at risk of assassination' Israel's opposition leader warns that the prime minister risks being assassinated over his Gaza disengagement plan.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.549887}
+{"text": "Arson attack on Jewish centre in Paris (AFP) AFP - A Jewish social centre in central Paris was destroyed by fire overnight in an anti-Semitic arson attack, city authorities said.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.548172}
+{"text": "Consumer spending jumped in September WASHINGTON - Consumers, who substantially slowed down their spending in late summer, roared back to life in September, boosting their purchases by 0.6 percent.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.486617}
+{"text": "Editorial: Sub incident shows China #39;s stripes After days of speculation and a chase by Japanese destroyers and a surveillance plane, it has finally been determined that the nuclear submarine that intruded into Japanese territorial water between Okinawa and Taiwan was Chinese.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.462702}
+{"text": "Rebound in US consumer spending US consumer spending rebounded in July, a sign the economy may be emerging from an early summer decline. Consumer spending rose 0.8 last month, boosted by car and retail sales.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.486717}
+{"text": "Oil Falls to 4-Month Low on Signs of Heating-Oil Supply Rise Crude oil fell to the lowest in more than four months on speculation that warm weather and increased refinery production bolstered US heating-oil stockpiles last week.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.483435}
+{"text": "Wall St.'s Nest Egg - the Housing Sector  NEW YORK (Reuters) - If there were any doubts that we're  still living in the era of the stay-at-home economy, the rows  of empty seats at the Athens Olympics should help erase them.", "true_label": "Business", "predicted_label": "Sports", "max_prob": 0.50777}
+{"text": "Deutsche Bank to Sell Scudder Business to Legg Mason Deutsche Bank AG of Germany plans to sell its New York, Philadelphia, Cincinnati and Chicago offices of Scudder Private Investment Counsel to Legg Mason Inc. for \\$55 million, plus payments of up to \\$26 million, the company said Monday.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.493886}
+{"text": "Oil Prices Sink to a Four-Month Low  LONDON (Reuters) - Oil prices sank to a four-month low  below \\$41 for U.S. crude on Wednesday after leading OPEC  producer Saudi Arabia questioned the need for the cartel to  curb supplies.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.476556}
+{"text": "Symantec Said to Be in Talks for Veritas Symantec, which produces the Norton line of computer products, is in talks to acquire Veritas Software, a maker of data backup programs.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.502015}
+{"text": "Gold Fields loses high court bid to halt Harmony takeover South African mining giant Gold Fields lost a high court bid to halt a hostile takeover by rival Harmony Gold, which is seeking to create the world #39;s biggest gold producer, a court official said.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.420627}
+{"text": "The Handbag? Total Knockoff. The Price Tag? All Too Real. It is as much a rite of the New York holiday season as sidewalk Santas or crowded Fifth Avenue sidewalks: the proliferation of hawkers selling counterfeit products like the fake Fendi handbag, the replica Rolex watch and the pirated DVD.", "true_label": "Business", "predicted_label": "World", "max_prob": 0.480361}
+{"text": "Bedding Blip? Go Back to Sleep This is getting old. Every time Hidden Gems selection Select Comfort (Nasdaq: SCSS) gets settled in and ready for a long night #39;s rest full of dreams of two-times-in-three-years capital gains, along ", "true_label": "Business", "predicted_label": "Sports", "max_prob": 0.568377}
+{"text": "Mars water tops science honours The discovery that salty, acidic water once flowed across the surface of Mars has topped a list of the 10 key scientific advances of 2004.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.331842}
+{"text": "France, Ivory Coast relations worsen Nine French peacekeepers were killed during an air raid by Ivorian bombers more than a week ago. France responded by destroying most of the Ivorian air force.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.518902}
+{"text": "WordsWorth Books files Chapter 11 WordsWorth Books, a Harvard Square institution for 29 years, yesterday filed for bankruptcy protection, as its owners seek a buyer or investor to help the independent bookseller compete with giant rivals like Amazon.com.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.480633}
+{"text": "Toys Will Be Toys Trust me, you don #39;t want to see desperate parents shopping over the holidays. I was at Circuit City (NYSE: CC) last week and saw a mother pleading with a sales clerk for a Nintendo DS portable video game system.", "true_label": "Sci/Tech", "predicted_label": "World", "max_prob": 0.383977}
+{"text": "Vodafone begins 3G service Vodafone launches its third-generation services for mobile phones, offering video calls, music downloads and games.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.504971}
+{"text": "Georgia Evolution Dispute Embarrasses Some Atlanta, Nov. 11 - First, Georgia #39;s education chief tried to take the word  quot;evolution quot; out of the state #39;s science curriculum.", "true_label": "Sci/Tech", "predicted_label": "Sports", "max_prob": 0.495901}
+{"text": "Oil Down 3 Pct. as U.S. Winter Stays Mild  NEW YORK (Reuters) - Oil prices slid 3 percent on Monday on  expectations that more mild U.S. weather at the start of the  New Year will limit heating oil demand.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.481833}
+{"text": "Lucent's history a case study in wild rides If you've ever wondered what ignited America's late-1990s obsession with telecom and tech stocks, you could well argue it all started with Lucent Technologies .", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.35487}
+{"text": "General Mills converting all US cereals to whole grain GOLDEN VALLEY, Minn. -- Breakfast cereal maker General Mills is converting all of its cereals to whole grain. The company says it becomes the first leading food company to make the move involving cereals.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.494894}
+{"text": "Taking Advantage of the Terminally Stupid Would you pay \\$4 for something that, at best, is worth a dime? Concord Communications shareholders would.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.394048}
+{"text": "West Japan on Guard for Aftershocks After Quakes  TOKYO (Reuters) - Residents of western Japan were warned of  possible aftershocks on Monday after two strong earthquakes the  previous day but authorities said the tremors were not directly  linked to a cycle of major seismic activity that hits the  region every century or so.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.58476}
+{"text": "Bush Blames Poorly Made Shirt for Bulge (AP) AP - President Bush sees the value  #151; and the humor  #151; in the idea that aides could secretly feed him advice through a radio receiver hidden on his back.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.61032}
+{"text": "Black Watch troops move into position The first units of a Black Watch battlegroup are due to arrive today in their new positions south of Baghdad as Tony Blair indicated that more British troops may replace them in the American-controlled zone before the end of the year.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.512313}
+{"text": "BSkyB sees profits rise after strong subscriber growth (AFP) AFP - British satellite broadcaster BSkyB said profit rose by 16 percent in the first quarter as the group enjoyed strong subscriber growth in the run-up to the key Christmas trading period.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.484029}
+{"text": "Al Qaeda Urge Wide-Scale Muslim Resistance Al Qaeda #39;s No. 2 man Ayman Zawahiri called for an all-out armed resistance in the Muslim world against the West and Jews whom he described as crusaders.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.638306}
+{"text": "Final Round in Cable-ISP Fight The U.S. Supreme Court has agreed to hear whether cable operators must give access to their lines to third-party ISPs. Michael Grebb reports from Washington.", "true_label": "Sci/Tech", "predicted_label": "Sports", "max_prob": 0.596346}
+{"text": "PeopleSofts big bash See you next year in Las Vegas , proclaimed a marquee at the PeopleSoft user conference in San Francisco in late September. It was one of many not-so-subtle attempts by the company to reassure its customers ", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.488586}
+{"text": "EBay #39;s Buy Of Rent.com May Lack Strategic Sense Standard  amp; Poor #39;s Equity Research said the purchase of Rent.com by eBay (nasdaq: EBAY - news - people ) could be a bit of a miscalculation.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.495739}
+{"text": "Sharon to Hold Tense Meeting With Settlers JERUSALEM - After more than a year of avoiding Jewish settlers, Israeli Prime Minister Ariel Sharon has decided to directly confront his former supporters in a meeting about his contentious plan to withdraw from the Gaza Strip and part of the West Bank.    Sharon invited settler leaders to meet with him in Jerusalem on Sunday, just a week before he presents his \"disengagement\" plan to parliament...", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.614542}
+{"text": "Black Market in British Treasure Sold on EBay (Reuters) Reuters - Pieces of Britain's past, including a\\second-century silver ring and a 500-year-old Tudor trade\\weight, are among artifacts being peddled daily on the Internet\\to the alarm of experts at the British Museum.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.410666}
+{"text": "Dollar Stuck, CPI Offers Little Direction  TOKYO (Reuters) - The dollar moved in tight ranges on  Wednesday as most investors shrugged off lower-than-expected  U.S. inflation data and stuck to the view the U.S. Federal  Reserve would continue raising rates.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.479432}
+{"text": "Chile's Lagos Receives Torture Commission Report &lt;p&gt;&lt;/p&gt;&lt;p&gt; By Ignacio Badal&lt;/p&gt;&lt;p&gt; SANTIAGO, Chile (Reuters) - Chilean President Ricardo Lagosreceived a chilling report on Wednesday from a governmentcommission that interviewed more than 30,000 victims tochronicle for the first time the systematic use of tortureduring Augusto Pinochet's 1973-1990 dictatorship.&lt;/p&gt;", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.442196}
+{"text": "Greenspan Worried Congress to Bar Options Expensing  WASHINGTON (Reuters) - Federal Reserve Chairman Alan  Greenspan on Friday said he was very worried Congress would try  to thwart efforts by the Financial Accounting Standards Board  to require expensing of stock options.", "true_label": "Business", "predicted_label": "World", "max_prob": 0.498896}
+{"text": "NASA Leader Cites Finances and Submits His Resignation Sean O #39;Keefe resigned as NASA administrator on Monday, saying he is leaving the position he has held for three years to pursue better economic opportunity for his family.", "true_label": "Sci/Tech", "predicted_label": "Sports", "max_prob": 0.604052}
+{"text": "Bad Day for Drug Companies - and Patients  LONDON (Reuters) - Investors in pharmaceuticals were dealt  a triple whammy on Friday as Pfizer Inc, AstraZeneca Plc and  Eli Lilly and Co all shocked the market with bad news about key  products.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.495296}
+{"text": "Why Putin is backing Kyoto again Why did Russian President Vladimir Putin decide to ratify the Kyoto Protocol on climate change last week, only six months after his top adviser, Andrei Illarionov, called it a  quot;death treaty?", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.648978}
+{"text": "An Afghan 'hanging chad' dispute An independent inquiry is helping to defuse a controversy over ink used in Saturday's election.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.571217}
+{"text": "Calif. Mental Health Services May Expand (AP) AP - As pressures increase on California's mental health system, its workers and advocates say they are forced to do more with a supply of money that seems to shrink each year.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.470756}
+{"text": "Stability Control Systems Can Save Lives (AP) AP - Stability control systems could save up to 7,000 lives each year if they were standard equipment on all vehicles, according to a study by the insurance industry.", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.44314}
+{"text": "Two hurricanes = two deductibles Many homeowners in the Orlando area suffered a double blow when hurricanes Charley and Frances struck in quick succession. Now, they #39;re smarting from a financial one-two punch - two insurance deductibles.", "true_label": "Business", "predicted_label": "Sports", "max_prob": 0.532307}
+{"text": "Stocks Seen Flat as Earnings Pour In US stock futures pointed to a flat market open Thursday as a rush of quarterly earnings reports painted a mixed picture for corporate profits amid lingering worries over the high price of oil.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.482357}
+{"text": "Thaksin in the Firing Line After Massacre BANGKOK/JEDDAH, 29 October 2004 - A bomb ripped through two bars in southern Thailand yesterday, killing two people and wounding about 20, in what could be the first reaction to the deaths of 78 Muslims in police custody this week.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.517503}
+{"text": "Fannie Mae agrees to accounting changes Fannie Mae, facing questions about its accounting similar to those that shook up Freddie Mac last year, has agreed to changes that will bring it in compliance with accounting standards.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.487456}
+{"text": "PeopleSoft gives in to Oracle, takes bid PeopleSoft Inc. capitulated to Oracle Corp., accepting a sweetened \\$10.3 billion takeover offer to end an 18-month battle that pitted PeopleSoft against its investors and led to the ouster of its chief executive.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.495211}
+{"text": "Secretary Rumsfeld Remarks En Route to El Salvador SEC. RUMSFELD: As you know, were going to be stopping in El Salvador and Nicaragua and Panama and Ecuador. The first stops will be visits to the places, particularly El Salvador and Nicaragua - countries ", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.522648}
+{"text": "Virus writers look for work THE WRITERS of the MyDoom viruses are encoding job applications into the latest variants of the bug. According to Sophos the plea for work was found when its boffins were stripping the code of the MyDoom-U and MyDoom-V variants.", "true_label": "Sci/Tech", "predicted_label": "Sports", "max_prob": 0.461721}
+{"text": "Key Martha Witness Gets Wrist Slap (CBS/AP) A former brokerage assistant who helped Martha Stewart make her fateful stock trade and later emerged as a key government witness was spared both prison and probation Friday for accepting a payoff during the government #39;s investigation.", "true_label": "Business", "predicted_label": "World", "max_prob": 0.509705}
+{"text": "Australia turns down plea for more troops to protect UN staff in Iraq (AFP) AFP - Australia has turned down a diplomatic plea for a contribution to a military force to protect United Nations (UN) personnel in Iraq.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.477345}
+{"text": "Oracle #39;s sweet on PeopleSoft Oracle sweetened its hostile bid for rival business software maker PeopleSoft to \\$9.2 billion, a 14 increase aimed at resolving the long-running takeover battle between the bitter foes.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.501009}
+{"text": "Chirac #39;s tour of China magnifies partnership Dialogue between China and France, two countries which highly value cultural diversity and pluralism in international politics, is no doubt conducive to world peace.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.597438}
+{"text": "Philippine Rebels Free Troops, Talks in Doubt  PRESENTACION, Philippines (Reuters) - Philippine communist  rebels freed Wednesday two soldiers they had held as \"prisoners  of war\" for more than five months, saying they wanted to  rebuild confidence in peace talks with the government.", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.553101}
+{"text": "September Retail Sales Up by 1.5 Percent WASHINGTON - Shoppers got their buying groove back last month, propelling sales at the nation's retailers by a strong 1.5 percent. It was the best showing since March...", "true_label": "World", "predicted_label": "Sci/Tech", "max_prob": 0.47643}
+{"text": "Parmalat Investors Flock to Court Seeking Damages (Update2) Hundreds of Italians who lost savings in the collapse of food company Parmalat Finanziaria SpA flocked to Milan #39;s courthouse, seeking to recover damages as part of the criminal investigation of Italy #39; biggest bankruptcy.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.490151}
+{"text": "Eight children stabbed to death as they sleep A man broke into a school dormitory and stabbed eight sleeping children to death before fleeing. The murders at the Ruzhou No2 Senior Middle School in Pingdingshan, in the central province of Henan, was the ", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.639374}
+{"text": "Eurostocks Nudge Up on Ericsson  PARIS (Reuters) - European shares nosed up on Wednesday as  Ericsson gained on news it had won part of \\$4-billion Cingular  deal and with Glaxo buoyed after Pfizer affirmed its outlook.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.479547}
+{"text": "Siemens Says Cellphone Flaw May Hurt Users and Its Profit Siemens, the world #39;s fourth-largest maker of mobile phones, said Friday that a software flaw that can create a piercing ring in its newest phone models might hurt earnings in its handset division.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.503006}
+{"text": "Strong Earthquake Strikes Central Calif. PARKFIELD, Calif. - A strong earthquake struck Central California on Tuesday that was felt from San Francisco to the Los Angeles area...", "true_label": "World", "predicted_label": "Sports", "max_prob": 0.599114}
+{"text": "Update 8: Crude Oil Futures Sink to \\$46 Per Barrel December delivery crude on the New York Mercantile Exchange dropped 76 cents to \\$46.11 per barrel. The benchmark light, sweet crude remained about \\$8 a barrel cheaper than its closing record of \\$55.17 recorded Oct. 22 and Oct. 26.", "true_label": "Business", "predicted_label": "Sci/Tech", "max_prob": 0.490082}

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d8d77dc2a0ece8e5732ed13a9e0fe0c157619a98db34146b5c3dd0bc35012578
 size 5361600

 version https://git-lfs.github.com/spec/v1
+oid sha256:85c65e6ac208c96098a76133e4b2ba6d32f81f36b7be6e46a069c7f855bd1fef
 size 5361600

tokenizer.json CHANGED Viewed

@@ -10,7 +10,7 @@
   "added_tokens": [
     {
       "id": 0,
-      "content": "[PAD]",
       "single_word": false,
       "lstrip": false,
       "rstrip": false,
@@ -19,7 +19,7 @@
     },
     {
       "id": 1,
-      "content": "[UNK]",
       "single_word": false,
       "lstrip": false,
       "rstrip": false,
@@ -149,12 +149,12 @@
   "decoder": null,
   "model": {
     "type": "WordPiece",
-    "unk_token": "[UNK]",
     "continuing_subword_prefix": "##",
     "max_input_chars_per_word": 100,
     "vocab": {
-      "[PAD]": 0,
-      "[UNK]": 1,
       "[CLS]": 2,
       "[SEP]": 3,
       "[MASK]": 4,
@@ -214,56 +214,56 @@
       "y": 58,
       "z": 59,
       "##e": 60,
-      "##t": 61,
-      "##l": 62,
-      "##s": 63,
-      "##7": 64,
-      "##9": 65,
-      "##r": 66,
-      "##o": 67,
-      "##b": 68,
-      "##m": 69,
-      "##u": 70,
-      "##a": 71,
-      "##i": 72,
-      "##y": 73,
-      "##w": 74,
-      "##k": 75,
-      "##n": 76,
       "##h": 77,
-      "##p": 78,
-      "##c": 79,
-      "##d": 80,
       "##f": 81,
-      "##g": 82,
-      "##v": 83,
-      "##j": 84,
-      "##z": 85,
-      "##6": 86,
-      "##1": 87,
-      "##x": 88,
-      "##4": 89,
-      "##3": 90,
-      "##q": 91,
-      "##2": 92,
-      "##0": 93,
-      "##5": 94,
-      "##-": 95,
-      "##\\": 96,
-      "##8": 97,
-      "##\"": 98,
-      "##$": 99,
       "##:": 100,
-      "##.": 101,
-      "##&": 102,
       "##,": 103,
-      "##'": 104,
-      "##(": 105,
-      "##)": 106,
-      "##/": 107,
-      "##_": 108,
       "###": 109,
-      "##*": 110,
       "##er": 111,
       "th": 112,
       "##in": 113,
@@ -690,11 +690,11 @@
       "##ood": 534,
       "li": 535,
       "tech": 536,
-      "##les": 537,
-      "##az": 538,
       "##yst": 539,
-      "##duc": 540,
-      "##ged": 541,
       "week": 542,
       "##ason": 543,
       "##ounc": 544,
@@ -761,8 +761,8 @@
       "##ely": 605,
       "ass": 606,
       "news": 607,
-      "##land": 608,
-      "##gr": 609,
       "##orts": 610,
       "afp": 611,
       "milit": 612,
@@ -782,8 +782,8 @@
       "stocks": 626,
       "red": 627,
       "ter": 628,
-      "##line": 629,
-      "##co": 630,
       "fed": 631,
       "par": 632,
       "shar": 633,
@@ -890,10 +890,10 @@
       "##ards": 734,
       "years": 735,
       "help": 736,
-      "##ual": 737,
-      "##ph": 738,
-      "##ct": 739,
-      "##ger": 740,
       "near": 741,
       "launch": 742,
       "##int": 743,
@@ -914,8 +914,8 @@
       "col": 758,
       "day": 759,
       "jap": 760,
-      "##ms": 761,
-      "##ied": 762,
       "##ony": 763,
       "champ": 764,
       "aw": 765,
@@ -988,8 +988,8 @@
       "killed": 832,
       "home": 833,
       "job": 834,
-      "##ious": 835,
-      "##cer": 836,
       "##inian": 837,
       "##orn": 838,
       "##ures": 839,
@@ -998,8 +998,8 @@
       "ca": 842,
       "fall": 843,
       "tra": 844,
-      "##bs": 845,
-      "##uts": 846,
       "##ants": 847,
       "ins": 848,
       "plans": 849,
@@ -1111,8 +1111,8 @@
       "ed": 955,
       "loss": 956,
       "publ": 957,
-      "##be": 958,
-      "##ium": 959,
       "##ving": 960,
       "stre": 961,
       "##use": 962,
@@ -1136,9 +1136,9 @@
       "19": 980,
       "11": 981,
       "san": 982,
-      "##lud": 983,
-      "##bm": 984,
-      "##ges": 985,
       "##ons": 986,
       "##ities": 987,
       "##eld": 988,
@@ -1150,8 +1150,8 @@
       "pakistan": 994,
       "her": 995,
       "way": 996,
-      "##ling": 997,
-      "##ned": 998,
       "##pr": 999,
       "stock": 1000,
       "leader": 1001,
@@ -1188,10 +1188,10 @@
       "tick": 1032,
       "##eb": 1033,
       "##eep": 1034,
-      "##ted": 1035,
-      "##lin": 1036,
-      "##let": 1037,
-      "##ces": 1038,
       "##ank": 1039,
       "##round": 1040,
       "##isco": 1041,
@@ -1225,8 +1225,8 @@
       "baghdad": 1069,
       "cust": 1070,
       "nas": 1071,
-      "##ty": 1072,
-      "##ok": 1073,
       "there": 1074,
       "##ects": 1075,
       "being": 1076,
@@ -1267,8 +1267,8 @@
       "gre": 1111,
       "make": 1112,
       "##eks": 1113,
-      "##fore": 1114,
-      "##gy": 1115,
       "##ility": 1116,
       "before": 1117,
       "dev": 1118,
@@ -1327,9 +1327,9 @@
       "mid": 1171,
       "riv": 1172,
       "sold": 1173,
-      "##se": 1174,
-      "##ov": 1175,
-      "##over": 1176,
       "stud": 1177,
       "##ution": 1178,
       "prime": 1179,
@@ -1355,11 +1355,11 @@
       "many": 1199,
       "round": 1200,
       "ww": 1201,
-      "##loy": 1202,
-      "##son": 1203,
-      "##ux": 1204,
       "##aw": 1205,
-      "##cl": 1206,
       "##fat": 1207,
       "##ene": 1208,
       "##ication": 1209,
@@ -1460,8 +1460,8 @@
       "poss": 1304,
       "six": 1305,
       "tur": 1306,
-      "##uel": 1307,
-      "##com": 1308,
       "##ject": 1309,
       "##ind": 1310,
       "##elf": 1311,
@@ -1475,9 +1475,9 @@
       "num": 1319,
       "tim": 1320,
       "your": 1321,
-      "##ave": 1322,
       "##ior": 1323,
-      "##ne": 1324,
       "##ham": 1325,
       "##pite": 1326,
       "##erry": 1327,
@@ -1523,8 +1523,8 @@
       "lif": 1367,
       "past": 1368,
       "rul": 1369,
-      "##med": 1370,
-      "##iers": 1371,
       "##info": 1372,
       "##iting": 1373,
       "##ised": 1374,
@@ -1657,13 +1657,13 @@
       "sent": 1501,
       "soc": 1502,
       "##ev": 1503,
-      "##so": 1504,
-      "##oid": 1505,
-      "##back": 1506,
-      "##min": 1507,
-      "##ik": 1508,
-      "##ws": 1509,
-      "##ners": 1510,
       "##ator": 1511,
       "##ida": 1512,
       "chr": 1513,
@@ -1718,9 +1718,9 @@
       "men": 1562,
       "match": 1563,
       "val": 1564,
-      "##port": 1565,
-      "##da": 1566,
-      "##gan": 1567,
       "##van": 1568,
       "inf": 1569,
       "##ally": 1570,
@@ -1755,14 +1755,14 @@
       "lo": 1599,
       "mal": 1600,
       "ur": 1601,
-      "##ler": 1602,
-      "##sh": 1603,
-      "##ai": 1604,
-      "##ns": 1605,
-      "##ness": 1606,
-      "##pe": 1607,
-      "##fact": 1608,
-      "##ging": 1609,
       "##ered": 1610,
       "##ench": 1611,
       "##row": 1612,
@@ -1797,11 +1797,11 @@
       "vo": 1641,
       "want": 1642,
       "##ta": 1643,
-      "##lo": 1644,
-      "##lier": 1645,
-      "##mber": 1646,
-      "##ause": 1647,
-      "##ward": 1648,
       "toky": 1649,
       "stop": 1650,
       "reven": 1651,
@@ -1839,10 +1839,10 @@
       "mail": 1683,
       "much": 1684,
       "outs": 1685,
-      "##uk": 1686,
-      "##uj": 1687,
-      "##pl": 1688,
-      "##gers": 1689,
       "##att": 1690,
       "anal": 1691,
       "##chester": 1692,
@@ -1884,13 +1884,13 @@
       "pur": 1728,
       "sn": 1729,
       "tv": 1730,
-      "##red": 1731,
-      "##off": 1732,
-      "##by": 1733,
-      "##ming": 1734,
-      "##af": 1735,
-      "##ael": 1736,
-      "##ked": 1737,
       "##atic": 1738,
       "##ative": 1739,
       "inform": 1740,
@@ -1940,14 +1940,14 @@
       "rise": 1784,
       "##ex": 1785,
       "##eal": 1786,
-      "##ll": 1787,
-      "##ress": 1788,
-      "##aign": 1789,
-      "##iver": 1790,
-      "##go": 1791,
       "##esh": 1792,
-      "##ena": 1793,
-      "##eng": 1794,
       "forces": 1795,
       "univers": 1796,
       "##rial": 1797,
@@ -1979,10 +1979,10 @@
       "rates": 1823,
       "rank": 1824,
       "xp": 1825,
-      "##ny": 1826,
-      "##ping": 1827,
-      "##cy": 1828,
-      "##cri": 1829,
       "##ink": 1830,
       "##ests": 1831,
       "##ater": 1832,
@@ -2103,9 +2103,9 @@
       "sil": 1947,
       "sim": 1948,
       "sum": 1949,
-      "##lant": 1950,
-      "##lines": 1951,
-      "##de": 1952,
       "##enal": 1953,
       "##ale": 1954,
       "prison": 1955,
@@ -2158,14 +2158,14 @@
       "wound": 2002,
       "wild": 2003,
       "##eat": 2004,
-      "##til": 2005,
-      "##bur": 2006,
-      "##mir": 2007,
       "##ij": 2008,
       "##ience": 2009,
       "##ket": 2010,
-      "##ney": 2011,
-      "##cc": 2012,
       "##ining": 2013,
       "##ately": 2014,
       "infl": 2015,
@@ -2275,10 +2275,10 @@
       "yards": 2119,
       "##ef": 2120,
       "##ees": 2121,
-      "##lam": 2122,
-      "##less": 2123,
-      "##bit": 2124,
-      "##gin": 2125,
       "##ert": 2126,
       "though": 2127,
       "##inal": 2128,
@@ -2332,12 +2332,12 @@
       "sir": 2176,
       "youn": 2177,
       "yasser": 2178,
-      "##lad": 2179,
-      "##bc": 2180,
-      "##band": 2181,
       "##ways": 2182,
-      "##hic": 2183,
-      "##zil": 2184,
       "those": 2185,
       "##ined": 2186,
       "##orks": 2187,
@@ -2472,8 +2472,8 @@
       "rout": 2316,
       "vol": 2317,
       "wants": 2318,
-      "##ban": 2319,
-      "##dle": 2320,
       "##arm": 2321,
       "##ared": 2322,
       "##itness": 2323,
@@ -2530,16 +2530,16 @@
       "roy": 2374,
       "rising": 2375,
       "yah": 2376,
-      "##res": 2377,
-      "##ren": 2378,
-      "##oint": 2379,
-      "##br": 2380,
-      "##mo": 2381,
-      "##mar": 2382,
-      "##iest": 2383,
-      "##wh": 2384,
-      "##han": 2385,
-      "##den": 2386,
       "##ona": 2387,
       "town": 2388,
       "##icip": 2389,
@@ -2605,13 +2605,13 @@
       "penn": 2449,
       "rap": 2450,
       "witness": 2451,
-      "##set": 2452,
-      "##sue": 2453,
-      "##bi": 2454,
-      "##bed": 2455,
-      "##no": 2456,
-      "##ds": 2457,
-      "##gage": 2458,
       "##val": 2459,
       "##vet": 2460,
       "##50": 2461,
@@ -2697,16 +2697,16 @@
       "same": 2541,
       "vill": 2542,
       "##ric": 2543,
-      "##bo": 2544,
-      "##mas": 2545,
-      "##iw": 2546,
-      "##yl": 2547,
-      "##yle": 2548,
-      "##we": 2549,
-      "##wards": 2550,
-      "##kins": 2551,
-      "##pal": 2552,
-      "##ga": 2553,
       "##xic": 2554,
       "##ino": 2555,
       "##ories": 2556,
@@ -2804,12 +2804,12 @@
       "vent": 2648,
       "vote": 2649,
       "western": 2650,
-      "##te": 2651,
-      "##rac": 2652,
-      "##oot": 2653,
-      "##mit": 2654,
-      "##ues": 2655,
-      "##aff": 2656,
       "##inc": 2657,
       "##ana": 2658,
       "##atory": 2659,
@@ -2897,17 +2897,17 @@
       "wid": 2741,
       "wood": 2742,
       "yuk": 2743,
-      "##lt": 2744,
-      "##liam": 2745,
-      "##rich": 2746,
-      "##ba": 2747,
-      "##bes": 2748,
-      "##ufact": 2749,
-      "##wide": 2750,
-      "##nes": 2751,
-      "##come": 2752,
-      "##ford": 2753,
-      "##get": 2754,
       "thin": 2755,
       "##ann": 2756,
       "##ening": 2757,
@@ -3015,20 +3015,20 @@
       "size": 2859,
       "sites": 2860,
       "tem": 2861,
-      "##to": 2862,
-      "##la": 2863,
-      "##ss": 2864,
-      "##men": 2865,
-      "##mun": 2866,
-      "##uation": 2867,
-      "##uff": 2868,
       "##aa": 2869,
       "##ait": 2870,
-      "##ya": 2871,
-      "##yn": 2872,
       "##wood": 2873,
-      "##ps": 2874,
-      "##ced": 2875,
       "##ery": 2876,
       "thail": 2877,
       "##onment": 2878,
@@ -3139,17 +3139,17 @@
       "view": 2983,
       "water": 2984,
       "##eg": 2985,
-      "##ring": 2986,
-      "##bn": 2987,
-      "##bor": 2988,
-      "##aj": 2989,
-      "##wal": 2990,
-      "##nel": 2991,
       "##pon": 2992,
-      "##cel": 2993,
-      "##ded": 2994,
-      "##duce": 2995,
-      "##fi": 2996,
       "##ville": 2997,
       "##erent": 2998,
       "then": 2999,
@@ -3288,18 +3288,18 @@
       "wel": 3132,
       "went": 3133,
       "##egr": 3134,
-      "##tors": 3135,
-      "##tles": 3136,
-      "##lish": 3137,
-      "##ris": 3138,
-      "##uit": 3139,
-      "##uary": 3140,
-      "##ax": 3141,
-      "##yc": 3142,
-      "##ken": 3143,
-      "##cal": 3144,
-      "##cow": 3145,
-      "##cial": 3146,
       "##inth": 3147,
       "theft": 3148,
       "##itz": 3149,
@@ -3425,8 +3425,8 @@
       ".'": 3269,
       "70": 3270,
       "ba": 3271,
-      "cp": 3272,
-      "cz": 3273,
       "cat": 3274,
       "cro": 3275,
       "catch": 3276,
@@ -3459,22 +3459,22 @@
       "wrap": 3303,
       "zone": 3304,
       "zeal": 3305,
-      "##ties": 3306,
-      "##rill": 3307,
-      "##rict": 3308,
-      "##makers": 3309,
-      "##na": 3310,
-      "##ha": 3311,
-      "##har": 3312,
-      "##pers": 3313,
-      "##post": 3314,
-      "##do": 3315,
-      "##dam": 3316,
-      "##ders": 3317,
-      "##dog": 3318,
-      "##vent": 3319,
-      "##zn": 3320,
-      "##zz": 3321,
       "threw": 3322,
       "##ored": 3323,
       "##ati": 3324,
@@ -3641,22 +3641,22 @@
       "ways": 3485,
       "wond": 3486,
       "zar": 3487,
-      "##tions": 3488,
-      "##lies": 3489,
-      "##lah": 3490,
-      "##sa": 3491,
       "##rone": 3492,
-      "##ox": 3493,
-      "##mal": 3494,
-      "##mond": 3495,
-      "##west": 3496,
       "##kes": 3497,
-      "##night": 3498,
-      "##car": 3499,
-      "##cape": 3500,
       "##field": 3501,
-      "##gu": 3502,
-      "##gent": 3503,
       "##xx": 3504,
       "##inos": 3505,
       "##anch": 3506,
@@ -3860,8 +3860,8 @@
       "dial": 3704,
       "done": 3705,
       "donald": 3706,
-      "ep": 3707,
-      "ec": 3708,
       "eth": 3709,
       "egy": 3710,
       "eastern": 3711,
@@ -3891,21 +3891,21 @@
       "##eas": 3735,
       "##east": 3736,
       "##ease": 3737,
-      "##ten": 3738,
-      "##lan": 3739,
-      "##lom": 3740,
-      "##rad": 3741,
-      "##rum": 3742,
-      "##ma": 3743,
-      "##aver": 3744,
-      "##wer": 3745,
-      "##wich": 3746,
-      "##nn": 3747,
-      "##hi": 3748,
-      "##hib": 3749,
-      "##haps": 3750,
-      "##ption": 3751,
-      "##die": 3752,
       "##va": 3753,
       "thing": 3754,
       "think": 3755,
@@ -4110,8 +4110,8 @@
       "33": 3954,
       "49": 3955,
       "46": 3956,
-      "54": 3957,
-      "52": 3958,
       "800": 3959,
       "96": 3960,
       "auction": 3961,
@@ -4154,28 +4154,28 @@
       "vod": 3998,
       "wm": 3999,
       "##ead": 4000,
-      "##lig": 4001,
-      "##oes": 4002,
-      "##bod": 4003,
-      "##mens": 4004,
-      "##ua": 4005,
-      "##ift": 4006,
-      "##ique": 4007,
-      "##ym": 4008,
-      "##yond": 4009,
-      "##held": 4010,
-      "##py": 4011,
-      "##pts": 4012,
-      "##cat": 4013,
-      "##cle": 4014,
-      "##ction": 4015,
-      "##cats": 4016,
-      "##craft": 4017,
-      "##fort": 4018,
-      "##gs": 4019,
-      "##gn": 4020,
-      "##ze": 4021,
-      "##zona": 4022,
       "##0s": 4023,
       "thom": 4024,
       "##oring": 4025,
@@ -4455,29 +4455,29 @@
       "wizards": 4299,
       "##ein": 4300,
       "##ever": 4301,
-      "##tom": 4302,
-      "##lins": 4303,
-      "##rd": 4304,
-      "##ray": 4305,
-      "##right": 4306,
-      "##ml": 4307,
-      "##my": 4308,
-      "##mos": 4309,
-      "##au": 4310,
-      "##ien": 4311,
-      "##iro": 4312,
-      "##iod": 4313,
-      "##nam": 4314,
-      "##hran": 4315,
-      "##pm": 4316,
-      "##pool": 4317,
-      "##cing": 4318,
-      "##col": 4319,
-      "##dick": 4320,
-      "##dden": 4321,
-      "##gar": 4322,
-      "##jay": 4323,
-      "##za": 4324,
       "##ern": 4325,
       "##iny": 4326,
       "##inter": 4327,
@@ -4783,34 +4783,34 @@
       "wide": 4627,
       "yud": 4628,
       "##ew": 4629,
-      "##sf": 4630,
-      "##sler": 4631,
-      "##syl": 4632,
-      "##ril": 4633,
-      "##ruit": 4634,
-      "##oon": 4635,
-      "##oom": 4636,
-      "##burn": 4637,
-      "##backs": 4638,
-      "##mers": 4639,
-      "##ule": 4640,
-      "##ae": 4641,
-      "##aur": 4642,
-      "##iet": 4643,
-      "##wart": 4644,
-      "##work": 4645,
-      "##wait": 4646,
-      "##ko": 4647,
-      "##kn": 4648,
-      "##ht": 4649,
-      "##hy": 4650,
-      "##hoy": 4651,
-      "##cus": 4652,
-      "##cop": 4653,
-      "##dney": 4654,
-      "##fe": 4655,
-      "##gether": 4656,
-      "##giving": 4657,
       "##ero": 4658,
       "##inet": 4659,
       "##ono": 4660,
@@ -5164,33 +5164,33 @@
       "yard": 5008,
       "zimb": 5009,
       "##eer": 5010,
-      "##tv": 5011,
-      "##lers": 5012,
-      "##light": 5013,
-      "##liest": 5014,
-      "##of": 5015,
-      "##bury": 5016,
-      "##me": 5017,
-      "##mad": 5018,
-      "##mand": 5019,
-      "##uan": 5020,
-      "##ued": 5021,
-      "##uite": 5022,
-      "##iation": 5023,
-      "##web": 5024,
-      "##ki": 5025,
-      "##kel": 5026,
-      "##hard": 5027,
-      "##horn": 5028,
-      "##pp": 5029,
-      "##cos": 5030,
-      "##cell": 5031,
-      "##care": 5032,
       "##fc": 5033,
-      "##ground": 5034,
-      "##jects": 5035,
-      "##zen": 5036,
-      "##zai": 5037,
       "##inct": 5038,
       "##esco": 5039,
       "##orary": 5040,
@@ -5209,8 +5209,8 @@
       "anton": 5053,
       "##iled": 5054,
       "##ayan": 5055,
-      "##oum": 5056,
-      "##oud": 5057,
       "##outh": 5058,
       "##eline": 5059,
       "##omm": 5060,
@@ -5259,8 +5259,8 @@
       "altern": 5103,
       "##eman": 5104,
       "##emony": 5105,
-      "##umi": 5106,
-      "##umn": 5107,
       "##iana": 5108,
       "hass": 5109,
       "upset": 5110,
@@ -5572,8 +5572,8 @@
       "\"\\": 5416,
       "47": 5417,
       "43": 5418,
-      "56": 5419,
-      "53": 5420,
       "550": 5421,
       "64": 5422,
       "79": 5423,
@@ -5664,37 +5664,37 @@
       "zur": 5508,
       "##eim": 5509,
       "##eals": 5510,
-      "##lon": 5511,
-      "##ror": 5512,
-      "##rel": 5513,
-      "##rate": 5514,
-      "##rick": 5515,
-      "##rating": 5516,
-      "##ois": 5517,
-      "##bus": 5518,
-      "##break": 5519,
-      "##maker": 5520,
-      "##aq": 5521,
-      "##aire": 5522,
-      "##iop": 5523,
-      "##iem": 5524,
-      "##yg": 5525,
-      "##yan": 5526,
-      "##wr": 5527,
-      "##wom": 5528,
-      "##well": 5529,
-      "##netw": 5530,
-      "##hen": 5531,
-      "##hol": 5532,
-      "##hood": 5533,
-      "##house": 5534,
-      "##ple": 5535,
-      "##pose": 5536,
-      "##pread": 5537,
-      "##cr": 5538,
-      "##cling": 5539,
-      "##dy": 5540,
-      "##gas": 5541,
       "##vic": 5542,
       "##40": 5543,
       "##erv": 5544,
@@ -5767,9 +5767,9 @@
       "##ocol": 5611,
       "concess": 5612,
       "conrad": 5613,
-      "unw": 5614,
-      "unc": 5615,
-      "und": 5616,
       "units": 5617,
       "que": 5618,
       "quality": 5619,
@@ -6181,9 +6181,9 @@
       "37": 6025,
       "3rd": 6026,
       "59": 6027,
-      "66": 6028,
-      "63": 6029,
-      "65": 6030,
       "71": 6031,
       "81": 6032,
       "88": 6033,
@@ -6213,9 +6213,9 @@
       "ele": 6057,
       "ech": 6058,
       "erupted": 6059,
-      "ft": 6060,
-      "fm": 6061,
-      "fd": 6062,
       "fan": 6063,
       "fash": 6064,
       "fals": 6065,
@@ -6232,8 +6232,8 @@
       "hart": 6076,
       "hok": 6077,
       "hiring": 6078,
-      "ii": 6079,
-      "ic": 6080,
       "ioc": 6081,
       "ill": 6082,
       "ji": 6083,
@@ -6256,8 +6256,8 @@
       "lacks": 6100,
       "lux": 6101,
       "lley": 6102,
-      "ml": 6103,
-      "mh": 6104,
       "mates": 6105,
       "moss": 6106,
       "mey": 6107,
@@ -6302,696 +6302,696 @@
       "wave": 6146,
       "woes": 6147,
       "##eason": 6148,
-      "##tar": 6149,
-      "##town": 6150,
-      "##tada": 6151,
-      "##lor": 6152,
-      "##lison": 6153,
-      "##lana": 6154,
-      "##sin": 6155,
-      "##rim": 6156,
-      "##rors": 6157,
-      "##oit": 6158,
-      "##bon": 6159,
-      "##bel": 6160,
-      "##bos": 6161,
-      "##bour": 6162,
-      "##base": 6163,
-      "##bek": 6164,
-      "##boy": 6165,
-      "##bank": 6166,
-      "##body": 6167,
-      "##mes": 6168,
-      "##mine": 6169,
-      "##ao": 6170,
-      "##iar": 6171,
-      "##ias": 6172,
-      "##ios": 6173,
-      "##yad": 6174,
-      "##wag": 6175,
-      "##western": 6176,
-      "##wau": 6177,
-      "##kin": 6178,
-      "##ni": 6179,
-      "##nic": 6180,
-      "##hill": 6181,
-      "##hman": 6182,
-      "##po": 6183,
-      "##pes": 6184,
-      "##pass": 6185,
-      "##pped": 6186,
-      "##ca": 6187,
-      "##case": 6188,
-      "##ding": 6189,
-      "##fall": 6190,
-      "##vin": 6191,
-      "##jected": 6192,
-      "##joy": 6193,
-      "##zed": 6194,
-      "##zie": 6195,
-      "##zone": 6196,
-      "##zuk": 6197,
-      "##qtada": 6198,
       "##--": 6199,
-      "##\\\\": 6200,
-      "##erate": 6201,
-      "##erie": 6202,
-      "these": 6203,
-      "##iners": 6204,
-      "##inters": 6205,
       "##oni": 6206,
-      "##onz": 6207,
-      "##oned": 6208,
-      "##orox": 6209,
-      "##anges": 6210,
-      "##anova": 6211,
-      "##arr": 6212,
-      "##arus": 6213,
-      "##enque": 6214,
-      "##attered": 6215,
-      "##atively": 6216,
-      "##alo": 6217,
-      "##alore": 6218,
-      "##iture": 6219,
-      "##itting": 6220,
-      "##icts": 6221,
-      "##icut": 6222,
-      "##icate": 6223,
-      "##ician": 6224,
-      "##icits": 6225,
-      "##stal": 6226,
-      "##stream": 6227,
-      "##roc": 6228,
-      "##roll": 6229,
-      "##ilo": 6230,
-      "##ilit": 6231,
-      "##ayed": 6232,
-      "##ayers": 6233,
-      "##asy": 6234,
-      "##eling": 6235,
-      "##elps": 6236,
-      "##isk": 6237,
-      "##isan": 6238,
-      "##isign": 6239,
-      "##eced": 6240,
-      "##ecially": 6241,
-      "##eton": 6242,
-      "##idia": 6243,
-      "##urers": 6244,
-      "andrew": 6245,
-      "##entication": 6246,
-      "stab": 6247,
-      "stays": 6248,
-      "stations": 6249,
-      "forest": 6250,
-      "##rek": 6251,
-      "revel": 6252,
-      "##ading": 6253,
-      "##irts": 6254,
-      "##irut": 6255,
-      "##iristan": 6256,
-      "##ols": 6257,
-      "##ola": 6258,
-      "##olph": 6259,
-      "##aming": 6260,
-      "##ambi": 6261,
-      "##utt": 6262,
-      "##uty": 6263,
-      "##utor": 6264,
-      "##vement": 6265,
-      "##chrysler": 6266,
-      "##owled": 6267,
-      "chec": 6268,
-      "chess": 6269,
-      "chave": 6270,
-      "chances": 6271,
-      "##rah": 6272,
-      "##raib": 6273,
-      "##ravel": 6274,
-      "##imler": 6275,
-      "##imming": 6276,
-      "bear": 6277,
-      "beirut": 6278,
-      "##osh": 6279,
-      "##osed": 6280,
-      "##osys": 6281,
-      "dement": 6282,
-      "deuts": 6283,
-      "##aka": 6284,
-      "##earer": 6285,
-      "newest": 6286,
-      "##agers": 6287,
-      "##ande": 6288,
-      "##essary": 6289,
-      "shy": 6290,
-      "sher": 6291,
-      "shake": 6292,
-      "shape": 6293,
-      "shrek": 6294,
-      "shearer": 6295,
-      "apart": 6296,
-      "thatcher": 6297,
-      "##ully": 6298,
-      "##ocating": 6299,
-      "conc": 6300,
-      "constr": 6301,
-      "unbeat": 6302,
-      "unesco": 6303,
-      "sab": 6304,
-      "saving": 6305,
-      "queen": 6306,
-      "quiet": 6307,
-      "##aine": 6308,
-      "seym": 6309,
-      "lew": 6310,
-      "lehman": 6311,
-      "placed": 6312,
-      "itunes": 6313,
-      "itanium": 6314,
-      "ali": 6315,
-      "##emn": 6316,
-      "enable": 6317,
-      "enjoy": 6318,
-      "##umed": 6319,
-      "##iant": 6320,
-      "hasn": 6321,
-      "sue": 6322,
-      "suzuk": 6323,
-      "upon": 6324,
-      "upgrade": 6325,
-      "worry": 6326,
-      "clorox": 6327,
-      "ackn": 6328,
-      "wealth": 6329,
-      "franch": 6330,
-      "overwh": 6331,
-      "##artoum": 6332,
-      "##inea": 6333,
-      "havan": 6334,
-      "##ellar": 6335,
-      "amster": 6336,
-      "##ensions": 6337,
-      "expensive": 6338,
-      "opin": 6339,
-      "opel": 6340,
-      "opec": 6341,
-      "fires": 6342,
-      "income": 6343,
-      "reporter": 6344,
-      "marks": 6345,
-      "marqu": 6346,
-      "martha": 6347,
-      "##ifts": 6348,
-      "##abled": 6349,
-      "##abeth": 6350,
-      "resist": 6351,
-      "intense": 6352,
-      "twins": 6353,
-      "serve": 6354,
-      "serena": 6355,
-      "outcome": 6356,
-      "##anson": 6357,
-      "winless": 6358,
-      "sco": 6359,
-      "scen": 6360,
-      "scam": 6361,
-      "scare": 6362,
-      "scale": 6363,
-      "orland": 6364,
-      "blu": 6365,
-      "blocks": 6366,
-      "bloom": 6367,
-      "##onduct": 6368,
-      "aren": 6369,
-      "arth": 6370,
-      "argent": 6371,
-      "coverage": 6372,
-      "##achi": 6373,
-      "##aching": 6374,
-      "adam": 6375,
-      "adap": 6376,
-      "brid": 6377,
-      "brings": 6378,
-      "brink": 6379,
-      "##kee": 6380,
-      "##keys": 6381,
-      "recruit": 6382,
-      "trend": 6383,
-      "true": 6384,
-      "laur": 6385,
-      "##olds": 6386,
-      "##ograph": 6387,
-      "button": 6388,
-      "peru": 6389,
-      "perform": 6390,
-      "disk": 6391,
-      "disaster": 6392,
-      "##iction": 6393,
-      "teens": 6394,
-      "manned": 6395,
-      "present": 6396,
-      "presented": 6397,
-      "preseason": 6398,
-      "cancel": 6399,
-      "appar": 6400,
-      "appear": 6401,
-      "appointment": 6402,
-      "corr": 6403,
-      "##ampton": 6404,
-      "##active": 6405,
-      "invit": 6406,
-      "ellison": 6407,
-      "accel": 6408,
-      "relatives": 6409,
-      "sunni": 6410,
-      "##ashing": 6411,
-      "##under": 6412,
-      "phelps": 6413,
-      "securing": 6414,
-      "marketw": 6415,
-      "milan": 6416,
-      "milwau": 6417,
-      "warplanes": 6418,
-      "defence": 6419,
-      "defends": 6420,
-      "desper": 6421,
-      "soars": 6422,
-      "##azi": 6423,
-      "##azin": 6424,
-      "teaming": 6425,
-      "carried": 6426,
-      "considers": 6427,
-      "joy": 6428,
-      "slug": 6429,
-      "surger": 6430,
-      "assur": 6431,
-      "assess": 6432,
-      "militia": 6433,
-      "quarterf": 6434,
-      "caps": 6435,
-      "capability": 6436,
-      "redmond": 6437,
-      "genoc": 6438,
-      "regain": 6439,
-      "101": 6440,
-      "recording": 6441,
-      "governments": 6442,
-      "norway": 6443,
-      "swing": 6444,
-      "dealers": 6445,
-      "briton": 6446,
-      "searches": 6447,
-      "producers": 6448,
-      "talent": 6449,
-      "impress": 6450,
-      "barrier": 6451,
-      "barrage": 6452,
-      "billionair": 6453,
-      "billionaire": 6454,
-      "ministers": 6455,
-      "seasons": 6456,
-      "helping": 6457,
-      "demonstr": 6458,
-      "cola": 6459,
-      "guinea": 6460,
-      "youth": 6461,
-      "smoot": 6462,
-      "supplier": 6463,
-      "remember": 6464,
-      "##ront": 6465,
-      "raf": 6466,
-      "raids": 6467,
-      "##ennial": 6468,
-      "confusion": 6469,
-      "bigley": 6470,
-      "poised": 6471,
-      "steady": 6472,
-      "courts": 6473,
-      "execution": 6474,
-      "fallen": 6475,
-      "trapped": 6476,
-      "inside": 6477,
-      "managers": 6478,
-      "mobiles": 6479,
-      "belarus": 6480,
-      "israelis": 6481,
-      "antitr": 6482,
-      "evil": 6483,
-      "##yss": 6484,
-      "largely": 6485,
-      "missile": 6486,
-      "thrown": 6487,
-      "throws": 6488,
-      "targeted": 6489,
-      "targeting": 6490,
-      "developers": 6491,
-      "industries": 6492,
-      "older": 6493,
-      "vienna": 6494,
-      "depot": 6495,
-      "filip": 6496,
-      "coaching": 6497,
-      "eddie": 6498,
-      "streets": 6499,
-      "batting": 6500,
-      "196": 6501,
-      "113": 6502,
-      "##gests": 6503,
       "update1": 6504,
-      "update2": 6505,
-      "hernn": 6506,
-      "beaten": 6507,
-      "providing": 6508,
-      "imm": 6509,
-      "immun": 6510,
-      "efficient": 6511,
-      "verisign": 6512,
-      "exploit": 6513,
-      "loses": 6514,
-      "approve": 6515,
-      "approves": 6516,
-      "grew": 6517,
-      "write": 6518,
-      "writing": 6519,
-      "camps": 6520,
-      "driving": 6521,
-      "harring": 6522,
-      "virenque": 6523,
-      "legend": 6524,
-      "legacy": 6525,
-      "peacekeep": 6526,
-      "draws": 6527,
-      "midwest": 6528,
-      "syn": 6529,
-      "sybase": 6530,
-      "acquired": 6531,
-      "stronger": 6532,
-      "stronghold": 6533,
-      "radical": 6534,
-      "visits": 6535,
-      "followed": 6536,
-      "idg": 6537,
-      "videot": 6538,
-      "actions": 6539,
-      "scientist": 6540,
-      "scientific": 6541,
-      "lowered": 6542,
-      "solve": 6543,
-      "threatening": 6544,
-      "severe": 6545,
-      "homers": 6546,
-      "speculation": 6547,
-      "programme": 6548,
-      "insurer": 6549,
-      "hardware": 6550,
-      "persist": 6551,
-      "promising": 6552,
-      "install": 6553,
-      "instant": 6554,
-      "250": 6555,
-      "famous": 6556,
-      "labs": 6557,
-      "labour": 6558,
-      "commissioner": 6559,
-      "expert": 6560,
-      "watching": 6561,
-      "foreigners": 6562,
-      "safer": 6563,
-      "cypr": 6564,
-      "equatorial": 6565,
-      "demanding": 6566,
-      "democrats": 6567,
-      "maintain": 6568,
-      "wells": 6569,
-      "strained": 6570,
-      "matchup": 6571,
-      "infring": 6572,
-      "exchanges": 6573,
-      "crashes": 6574,
-      "hamid": 6575,
-      "humans": 6576,
-      "loans": 6577,
-      "malicious": 6578,
-      "financing": 6579,
-      "clears": 6580,
-      "investigated": 6581,
-      "investigators": 6582,
-      "curb": 6583,
-      "privat": 6584,
-      "privacy": 6585,
-      "authentication": 6586,
-      "rivalry": 6587,
-      "employers": 6588,
-      "employee": 6589,
-      "speedway": 6590,
-      "meth": 6591,
-      "mosul": 6592,
-      "pilot": 6593,
-      "communication": 6594,
-      "pollution": 6595,
-      "richest": 6596,
-      "doubles": 6597,
-      "clean": 6598,
-      "cuba": 6599,
-      "jumps": 6600,
-      "trees": 6601,
-      "prosp": 6602,
-      "fifa": 6603,
-      "martial": 6604,
-      "multi": 6605,
-      "silver": 6606,
-      "summit": 6607,
-      "##dez": 6608,
-      "estim": 6609,
-      "estimate": 6610,
-      "establ": 6611,
-      "factory": 6612,
-      "maced": 6613,
-      "tourists": 6614,
-      "eurozone": 6615,
-      "negotiators": 6616,
-      "cheese": 6617,
-      "respond": 6618,
-      "chipset": 6619,
-      "qaida": 6620,
-      "blockade": 6621,
-      "grounds": 6622,
-      "hunt": 6623,
-      "shoppers": 6624,
-      "disputed": 6625,
-      "feature": 6626,
-      "featuring": 6627,
-      "brazilian": 6628,
-      "landed": 6629,
-      "landing": 6630,
-      "bangalore": 6631,
-      "civilian": 6632,
-      "facing": 6633,
-      "happen": 6634,
-      "locked": 6635,
-      "volume": 6636,
-      "suggests": 6637,
-      "awarded": 6638,
-      "calc": 6639,
-      "calpers": 6640,
-      "gained": 6641,
-      "choose": 6642,
-      "captors": 6643,
-      "purchased": 6644,
-      "purchases": 6645,
-      "combined": 6646,
-      "conditions": 6647,
-      "climbing": 6648,
-      "aboard": 6649,
-      "professionals": 6650,
-      "automakers": 6651,
-      "dominated": 6652,
-      "laboratory": 6653,
-      "casino": 6654,
-      "casinos": 6655,
-      "gasol": 6656,
-      "villanova": 6657,
-      "reveal": 6658,
-      "connection": 6659,
-      "attempts": 6660,
-      "travelers": 6661,
-      "currency": 6662,
-      "escaped": 6663,
-      "biotech": 6664,
-      "daimler": 6665,
-      "initi": 6666,
-      "responsibility": 6667,
-      "tsb": 6668,
-      "clients": 6669,
-      "screens": 6670,
-      "manufacturers": 6671,
-      "knocked": 6672,
-      "burns": 6673,
-      "bullet": 6674,
-      "faith": 6675,
-      "fault": 6676,
-      "figure": 6677,
-      "khat": 6678,
-      "khartoum": 6679,
-      "occur": 6680,
-      "punish": 6681,
-      "staying": 6682,
-      "clashes": 6683,
-      "pleaded": 6684,
-      "crowd": 6685,
-      "overseas": 6686,
-      "rubble": 6687,
-      "##fielder": 6688,
-      "plunge": 6689,
-      "residents": 6690,
-      "survival": 6691,
-      "charlie": 6692,
-      "introduces": 6693,
-      "alexander": 6694,
-      "learning": 6695,
-      "learned": 6696,
-      "quitting": 6697,
-      "pledges": 6698,
-      "cracking": 6699,
-      "detainees": 6700,
-      "belgium": 6701,
-      "subscriber": 6702,
-      "momentum": 6703,
-      "destr": 6704,
-      "integration": 6705,
-      "celebrate": 6706,
-      "celebrated": 6707,
-      "brewer": 6708,
-      "madison": 6709,
-      "pacers": 6710,
-      "spurrier": 6711,
-      "lebanon": 6712,
-      "mozambi": 6713,
-      "described": 6714,
-      "carrying": 6715,
-      "swedish": 6716,
-      "broadcasting": 6717,
-      "roadside": 6718,
-      "##bridge": 6719,
-      "counterfe": 6720,
-      "observ": 6721,
-      "venezuela": 6722,
-      "venezuelans": 6723,
-      "bryant": 6724,
-      "mystery": 6725,
-      "randolph": 6726,
-      "vaugh": 6727,
-      "reasons": 6728,
-      "split": 6729,
-      "washed": 6730,
-      "brokerage": 6731,
-      "feeling": 6732,
-      "granted": 6733,
-      "astrona": 6734,
-      "diplomats": 6735,
-      "wmc": 6736,
-      "beheaded": 6737,
-      "property": 6738,
-      "inventories": 6739,
-      "slumping": 6740,
-      "delivery": 6741,
-      "guerrill": 6742,
-      "wrongdo": 6743,
-      "espn": 6744,
-      "especially": 6745,
-      "achie": 6746,
-      "coca": 6747,
-      "pieces": 6748,
-      "penalty": 6749,
-      "tackle": 6750,
-      "tossed": 6751,
-      "coalition": 6752,
-      "phillips": 6753,
-      "website": 6754,
-      "removed": 6755,
-      "throwing": 6756,
-      "refugee": 6757,
-      "citizens": 6758,
-      "coughlin": 6759,
-      "culture": 6760,
-      "grants": 6761,
-      "pedro": 6762,
-      "sgi": 6763,
-      "tipped": 6764,
-      "degree": 6765,
-      "admitted": 6766,
-      "floods": 6767,
-      "mohammad": 6768,
-      "guidant": 6769,
-      "trouble": 6770,
-      "batteries": 6771,
-      "copyright": 6772,
-      "connecticut": 6773,
-      "temporary": 6774,
-      "favorite": 6775,
-      "craig": 6776,
-      "gibbs": 6777,
-      "honda": 6778,
-      "nvidia": 6779,
-      "nedbank": 6780,
-      "rwanda": 6781,
-      "roster": 6782,
-      "##mandu": 6783,
-      "primary": 6784,
-      "exclusive": 6785,
-      "sponsored": 6786,
-      "alternative": 6787,
-      "clothing": 6788,
-      "database": 6789,
-      "telephony": 6790,
-      "hammered": 6791,
-      "ferrari": 6792,
-      "pharmaceutical": 6793,
-      "ghraib": 6794,
-      "hughes": 6795,
-      "nortel": 6796,
-      "rossi": 6797,
-      "veteran": 6798,
-      "##standing": 6799,
-      "anthony": 6800,
-      "stroke": 6801,
-      "proxy": 6802,
-      "concessions": 6803,
-      "queens": 6804,
-      "enforcement": 6805,
-      "freddie": 6806,
-      "overcame": 6807,
-      "serbia": 6808,
-      "elsewhere": 6809,
-      "cafes": 6810,
-      "rebuild": 6811,
-      "himalayan": 6812,
-      "distraction": 6813,
-      "experimental": 6814,
-      "friendly": 6815,
-      "overshadow": 6816,
-      "cavaliers": 6817,
-      "dallas": 6818,
-      "daily": 6819,
-      "fashion": 6820,
-      "giambi": 6821,
-      "hockey": 6822,
-      "hartford": 6823,
-      "hokies": 6824,
-      "kazaa": 6825,
-      "kathmandu": 6826,
-      "karmazin": 6827,
-      "loyal": 6828,
-      "lleyton": 6829,
-      "mhz": 6830,
-      "meyer": 6831,
-      "pfiz": 6832,
-      "p2p": 6833,
-      "ponting": 6834,
-      "rfid": 6835,
-      "svetlana": 6836,
-      "waziristan": 6837,
-      "wysiw": 6838,
       "##owledged": 6839,
       "chavez": 6840,
       "deutsche": 6841,
@@ -7020,12 +7020,12 @@
       "650": 6864,
       "7e": 6865,
       "77": 6866,
-      "86": 6867,
-      "84": 6868,
       "93": 6869,
       "950": 6870,
-      "a9": 6871,
-      "ak": 6872,
       "aet": 6873,
       "aven": 6874,
       "bug": 6875,
@@ -7107,8 +7107,8 @@
       "ol": 6951,
       "oce": 6952,
       "ori": 6953,
-      "pk": 6954,
-      "pd": 6955,
       "pad": 6956,
       "pest": 6957,
       "pist": 6958,
@@ -7139,8 +7139,8 @@
       "tying": 6983,
       "tumb": 6984,
       "tumble": 6985,
-      "um": 6986,
-      "ug": 6987,
       "u2": 6988,
       "van": 6989,
       "vanc": 6990,
@@ -7160,64 +7160,64 @@
       "zid": 7004,
       "ziff": 7005,
       "##ean": 7006,
-      "##sm": 7007,
-      "##sburg": 7008,
-      "##ression": 7009,
-      "##render": 7010,
-      "##oin": 7011,
-      "##oke": 7012,
-      "##bar": 7013,
-      "##bits": 7014,
-      "##buster": 7015,
-      "##bara": 7016,
-      "##mu": 7017,
-      "##mare": 7018,
-      "##mail": 7019,
-      "##uals": 7020,
-      "##uana": 7021,
-      "##aug": 7022,
-      "##aware": 7023,
-      "##iate": 7024,
-      "##iber": 7025,
-      "##iece": 7026,
-      "##yton": 7027,
-      "##wl": 7028,
-      "##wol": 7029,
-      "##wire": 7030,
-      "##water": 7031,
-      "##waves": 7032,
-      "##kok": 7033,
-      "##killer": 7034,
-      "##net": 7035,
-      "##nom": 7036,
-      "##nants": 7037,
-      "##nology": 7038,
-      "##hes": 7039,
-      "##had": 7040,
-      "##hocks": 7041,
-      "##pin": 7042,
-      "##pen": 7043,
-      "##part": 7044,
-      "##ports": 7045,
-      "##pson": 7046,
-      "##cu": 7047,
-      "##cers": 7048,
-      "##cair": 7049,
-      "##crew": 7050,
-      "##due": 7051,
-      "##dale": 7052,
-      "##fa": 7053,
-      "##function": 7054,
-      "##gb": 7055,
-      "##gom": 7056,
-      "##gus": 7057,
-      "##gard": 7058,
-      "##gment": 7059,
-      "##gart": 7060,
-      "##greg": 7061,
-      "##voc": 7062,
-      "##zer": 7063,
-      "##zard": 7064,
       "##xious": 7065,
       "##35": 7066,
       "##erious": 7067,
@@ -7393,8 +7393,8 @@
       "monetary": 7237,
       "haven": 7238,
       "haiti": 7239,
-      "havoc": 7240,
-      "hazard": 7241,
       "##ella": 7242,
       "amr": 7243,
       "amelie": 7244,
@@ -7648,9 +7648,9 @@
       "really": 7492,
       "realnetw": 7493,
       "147": 7494,
-      "146": 7495,
       "140": 7496,
-      "148": 7497,
       "household": 7498,
       "mcken": 7499,
       "visa": 7500,
@@ -7811,8 +7811,8 @@
       "focusing": 7655,
       "copper": 7656,
       "fixes": 7657,
-      "##market": 7658,
-      "##while": 7659,
       "towns": 7660,
       "suffering": 7661,
       "declares": 7662,
@@ -8045,8 +8045,8 @@
       "astronomers": 7889,
       "neighborhood": 7890,
       "dhaka": 7891,
-      "ftc": 7892,
-      "fdx": 7893,
       "false": 7894,
       "goran": 7895,
       "gurunet": 7896,
@@ -8145,8 +8145,8 @@
       "1b": 7989,
       "1st": 7990,
       "166": 7991,
-      "76": 7992,
-      "72": 7993,
       "92": 7994,
       "95": 7995,
       "900": 7996,

   "added_tokens": [
     {
       "id": 0,
+      "content": "<redacted_PAD>",
       "single_word": false,
       "lstrip": false,
       "rstrip": false,
     },
     {
       "id": 1,
+      "content": "<redacted_UNK>",
       "single_word": false,
       "lstrip": false,
       "rstrip": false,
   "decoder": null,
   "model": {
     "type": "WordPiece",
+    "unk_token": "<redacted_UNK>",
     "continuing_subword_prefix": "##",
     "max_input_chars_per_word": 100,
     "vocab": {
+      "<redacted_PAD>": 0,
+      "<redacted_UNK>": 1,
       "[CLS]": 2,
       "[SEP]": 3,
       "[MASK]": 4,
       "y": 58,
       "z": 59,
       "##e": 60,
+      "##c": 61,
+      "##q": 62,
+      "##u": 63,
+      "##z": 64,
+      "##o": 65,
+      "##n": 66,
+      "##d": 67,
+      "##r": 68,
+      "##t": 69,
+      "##i": 70,
+      "##s": 71,
+      "##j": 72,
+      "##g": 73,
+      "##y": 74,
+      "##a": 75,
+      "##k": 76,
       "##h": 77,
+      "##l": 78,
+      "##p": 79,
+      "##w": 80,
       "##f": 81,
+      "##\\": 82,
+      "##&": 83,
+      "##m": 84,
+      "##b": 85,
+      "##7": 86,
+      "##v": 87,
+      "##2": 88,
+      "##9": 89,
+      "##1": 90,
+      "##x": 91,
+      "##3": 92,
+      "##5": 93,
+      "##8": 94,
+      "##4": 95,
+      "##0": 96,
+      "##6": 97,
+      "##.": 98,
+      "##/": 99,
       "##:": 100,
+      "##'": 101,
+      "##\"": 102,
       "##,": 103,
+      "##-": 104,
+      "##*": 105,
+      "##$": 106,
+      "##)": 107,
+      "##(": 108,
       "###": 109,
+      "##_": 110,
       "##er": 111,
       "th": 112,
       "##in": 113,
       "##ood": 534,
       "li": 535,
       "tech": 536,
+      "##duc": 537,
+      "##ged": 538,
       "##yst": 539,
+      "##az": 540,
+      "##les": 541,
       "week": 542,
       "##ason": 543,
       "##ounc": 544,
       "##ely": 605,
       "ass": 606,
       "news": 607,
+      "##gr": 608,
+      "##land": 609,
       "##orts": 610,
       "afp": 611,
       "milit": 612,
       "stocks": 626,
       "red": 627,
       "ter": 628,
+      "##co": 629,
+      "##line": 630,
       "fed": 631,
       "par": 632,
       "shar": 633,
       "##ards": 734,
       "years": 735,
       "help": 736,
+      "##ct": 737,
+      "##ual": 738,
+      "##ger": 739,
+      "##ph": 740,
       "near": 741,
       "launch": 742,
       "##int": 743,
       "col": 758,
       "day": 759,
       "jap": 760,
+      "##ied": 761,
+      "##ms": 762,
       "##ony": 763,
       "champ": 764,
       "aw": 765,
       "killed": 832,
       "home": 833,
       "job": 834,
+      "##cer": 835,
+      "##ious": 836,
       "##inian": 837,
       "##orn": 838,
       "##ures": 839,
       "ca": 842,
       "fall": 843,
       "tra": 844,
+      "##uts": 845,
+      "##bs": 846,
       "##ants": 847,
       "ins": 848,
       "plans": 849,
       "ed": 955,
       "loss": 956,
       "publ": 957,
+      "##ium": 958,
+      "##be": 959,
       "##ving": 960,
       "stre": 961,
       "##use": 962,
       "19": 980,
       "11": 981,
       "san": 982,
+      "##ges": 983,
+      "##lud": 984,
+      "##bm": 985,
       "##ons": 986,
       "##ities": 987,
       "##eld": 988,
       "pakistan": 994,
       "her": 995,
       "way": 996,
+      "##ned": 997,
+      "##ling": 998,
       "##pr": 999,
       "stock": 1000,
       "leader": 1001,
       "tick": 1032,
       "##eb": 1033,
       "##eep": 1034,
+      "##ces": 1035,
+      "##ted": 1036,
+      "##lin": 1037,
+      "##let": 1038,
       "##ank": 1039,
       "##round": 1040,
       "##isco": 1041,
       "baghdad": 1069,
       "cust": 1070,
       "nas": 1071,
+      "##ok": 1072,
+      "##ty": 1073,
       "there": 1074,
       "##ects": 1075,
       "being": 1076,
       "gre": 1111,
       "make": 1112,
       "##eks": 1113,
+      "##gy": 1114,
+      "##fore": 1115,
       "##ility": 1116,
       "before": 1117,
       "dev": 1118,
       "mid": 1171,
       "riv": 1172,
       "sold": 1173,
+      "##ov": 1174,
+      "##over": 1175,
+      "##se": 1176,
       "stud": 1177,
       "##ution": 1178,
       "prime": 1179,
       "many": 1199,
       "round": 1200,
       "ww": 1201,
+      "##cl": 1202,
+      "##ux": 1203,
+      "##son": 1204,
       "##aw": 1205,
+      "##loy": 1206,
       "##fat": 1207,
       "##ene": 1208,
       "##ication": 1209,
       "poss": 1304,
       "six": 1305,
       "tur": 1306,
+      "##com": 1307,
+      "##uel": 1308,
       "##ject": 1309,
       "##ind": 1310,
       "##elf": 1311,
       "num": 1319,
       "tim": 1320,
       "your": 1321,
+      "##ne": 1322,
       "##ior": 1323,
+      "##ave": 1324,
       "##ham": 1325,
       "##pite": 1326,
       "##erry": 1327,
       "lif": 1367,
       "past": 1368,
       "rul": 1369,
+      "##iers": 1370,
+      "##med": 1371,
       "##info": 1372,
       "##iting": 1373,
       "##ised": 1374,
       "sent": 1501,
       "soc": 1502,
       "##ev": 1503,
+      "##oid": 1504,
+      "##ners": 1505,
+      "##ik": 1506,
+      "##so": 1507,
+      "##ws": 1508,
+      "##min": 1509,
+      "##back": 1510,
       "##ator": 1511,
       "##ida": 1512,
       "chr": 1513,
       "men": 1562,
       "match": 1563,
       "val": 1564,
+      "##da": 1565,
+      "##gan": 1566,
+      "##port": 1567,
       "##van": 1568,
       "inf": 1569,
       "##ally": 1570,
       "lo": 1599,
       "mal": 1600,
       "ur": 1601,
+      "##ns": 1602,
+      "##ness": 1603,
+      "##sh": 1604,
+      "##ging": 1605,
+      "##ai": 1606,
+      "##ler": 1607,
+      "##pe": 1608,
+      "##fact": 1609,
       "##ered": 1610,
       "##ench": 1611,
       "##row": 1612,
       "vo": 1641,
       "want": 1642,
       "##ta": 1643,
+      "##ause": 1644,
+      "##lo": 1645,
+      "##lier": 1646,
+      "##ward": 1647,
+      "##mber": 1648,
       "toky": 1649,
       "stop": 1650,
       "reven": 1651,
       "mail": 1683,
       "much": 1684,
       "outs": 1685,
+      "##uj": 1686,
+      "##uk": 1687,
+      "##gers": 1688,
+      "##pl": 1689,
       "##att": 1690,
       "anal": 1691,
       "##chester": 1692,
       "pur": 1728,
       "sn": 1729,
       "tv": 1730,
+      "##off": 1731,
+      "##red": 1732,
+      "##af": 1733,
+      "##ael": 1734,
+      "##ked": 1735,
+      "##ming": 1736,
+      "##by": 1737,
       "##atic": 1738,
       "##ative": 1739,
       "inform": 1740,
       "rise": 1784,
       "##ex": 1785,
       "##eal": 1786,
+      "##ress": 1787,
+      "##iver": 1788,
+      "##go": 1789,
+      "##aign": 1790,
+      "##ll": 1791,
       "##esh": 1792,
+      "##eng": 1793,
+      "##ena": 1794,
       "forces": 1795,
       "univers": 1796,
       "##rial": 1797,
       "rates": 1823,
       "rank": 1824,
       "xp": 1825,
+      "##cy": 1826,
+      "##cri": 1827,
+      "##ny": 1828,
+      "##ping": 1829,
       "##ink": 1830,
       "##ests": 1831,
       "##ater": 1832,
       "sil": 1947,
       "sim": 1948,
       "sum": 1949,
+      "##de": 1950,
+      "##lant": 1951,
+      "##lines": 1952,
       "##enal": 1953,
       "##ale": 1954,
       "prison": 1955,
       "wound": 2002,
       "wild": 2003,
       "##eat": 2004,
+      "##cc": 2005,
+      "##ney": 2006,
+      "##til": 2007,
       "##ij": 2008,
       "##ience": 2009,
       "##ket": 2010,
+      "##mir": 2011,
+      "##bur": 2012,
       "##ining": 2013,
       "##ately": 2014,
       "infl": 2015,
       "yards": 2119,
       "##ef": 2120,
       "##ees": 2121,
+      "##gin": 2122,
+      "##lam": 2123,
+      "##less": 2124,
+      "##bit": 2125,
       "##ert": 2126,
       "though": 2127,
       "##inal": 2128,
       "sir": 2176,
       "youn": 2177,
       "yasser": 2178,
+      "##zil": 2179,
+      "##hic": 2180,
+      "##lad": 2181,
       "##ways": 2182,
+      "##bc": 2183,
+      "##band": 2184,
       "those": 2185,
       "##ined": 2186,
       "##orks": 2187,
       "rout": 2316,
       "vol": 2317,
       "wants": 2318,
+      "##dle": 2319,
+      "##ban": 2320,
       "##arm": 2321,
       "##ared": 2322,
       "##itness": 2323,
       "roy": 2374,
       "rising": 2375,
       "yah": 2376,
+      "##oint": 2377,
+      "##den": 2378,
+      "##res": 2379,
+      "##ren": 2380,
+      "##iest": 2381,
+      "##han": 2382,
+      "##wh": 2383,
+      "##mo": 2384,
+      "##mar": 2385,
+      "##br": 2386,
       "##ona": 2387,
       "town": 2388,
       "##icip": 2389,
       "penn": 2449,
       "rap": 2450,
       "witness": 2451,
+      "##no": 2452,
+      "##ds": 2453,
+      "##set": 2454,
+      "##sue": 2455,
+      "##gage": 2456,
+      "##bi": 2457,
+      "##bed": 2458,
       "##val": 2459,
       "##vet": 2460,
       "##50": 2461,
       "same": 2541,
       "vill": 2542,
       "##ric": 2543,
+      "##iw": 2544,
+      "##ga": 2545,
+      "##yl": 2546,
+      "##yle": 2547,
+      "##kins": 2548,
+      "##pal": 2549,
+      "##we": 2550,
+      "##wards": 2551,
+      "##mas": 2552,
+      "##bo": 2553,
       "##xic": 2554,
       "##ino": 2555,
       "##ories": 2556,
       "vent": 2648,
       "vote": 2649,
       "western": 2650,
+      "##ues": 2651,
+      "##oot": 2652,
+      "##rac": 2653,
+      "##te": 2654,
+      "##aff": 2655,
+      "##mit": 2656,
       "##inc": 2657,
       "##ana": 2658,
       "##atory": 2659,
       "wid": 2741,
       "wood": 2742,
       "yuk": 2743,
+      "##come": 2744,
+      "##ufact": 2745,
+      "##nes": 2746,
+      "##rich": 2747,
+      "##get": 2748,
+      "##lt": 2749,
+      "##liam": 2750,
+      "##wide": 2751,
+      "##ford": 2752,
+      "##ba": 2753,
+      "##bes": 2754,
       "thin": 2755,
       "##ann": 2756,
       "##ening": 2757,
       "size": 2859,
       "sites": 2860,
       "tem": 2861,
+      "##ced": 2862,
+      "##uation": 2863,
+      "##uff": 2864,
+      "##to": 2865,
+      "##ss": 2866,
+      "##yn": 2867,
+      "##ya": 2868,
       "##aa": 2869,
       "##ait": 2870,
+      "##la": 2871,
+      "##ps": 2872,
       "##wood": 2873,
+      "##men": 2874,
+      "##mun": 2875,
       "##ery": 2876,
       "thail": 2877,
       "##onment": 2878,
       "view": 2983,
       "water": 2984,
       "##eg": 2985,
+      "##cel": 2986,
+      "##nel": 2987,
+      "##ded": 2988,
+      "##duce": 2989,
+      "##ring": 2990,
+      "##aj": 2991,
       "##pon": 2992,
+      "##wal": 2993,
+      "##fi": 2994,
+      "##bn": 2995,
+      "##bor": 2996,
       "##ville": 2997,
       "##erent": 2998,
       "then": 2999,
       "wel": 3132,
       "went": 3133,
       "##egr": 3134,
+      "##cal": 3135,
+      "##cow": 3136,
+      "##cial": 3137,
+      "##uit": 3138,
+      "##uary": 3139,
+      "##ris": 3140,
+      "##tors": 3141,
+      "##tles": 3142,
+      "##yc": 3143,
+      "##ax": 3144,
+      "##ken": 3145,
+      "##lish": 3146,
       "##inth": 3147,
       "theft": 3148,
       "##itz": 3149,
       ".'": 3269,
       "70": 3270,
       "ba": 3271,
+      "cz": 3272,
+      "cp": 3273,
       "cat": 3274,
       "cro": 3275,
       "catch": 3276,
       "wrap": 3303,
       "zone": 3304,
       "zeal": 3305,
+      "##zz": 3306,
+      "##zn": 3307,
+      "##na": 3308,
+      "##do": 3309,
+      "##dam": 3310,
+      "##ders": 3311,
+      "##dog": 3312,
+      "##rill": 3313,
+      "##rict": 3314,
+      "##ties": 3315,
+      "##ha": 3316,
+      "##har": 3317,
+      "##pers": 3318,
+      "##post": 3319,
+      "##makers": 3320,
+      "##vent": 3321,
       "threw": 3322,
       "##ored": 3323,
       "##ati": 3324,
       "ways": 3485,
       "wond": 3486,
       "zar": 3487,
+      "##car": 3488,
+      "##cape": 3489,
+      "##ox": 3490,
+      "##night": 3491,
       "##rone": 3492,
+      "##tions": 3493,
+      "##sa": 3494,
+      "##gu": 3495,
+      "##gent": 3496,
       "##kes": 3497,
+      "##lies": 3498,
+      "##lah": 3499,
+      "##west": 3500,
       "##field": 3501,
+      "##mal": 3502,
+      "##mond": 3503,
       "##xx": 3504,
       "##inos": 3505,
       "##anch": 3506,
       "dial": 3704,
       "done": 3705,
       "donald": 3706,
+      "ec": 3707,
+      "ep": 3708,
       "eth": 3709,
       "egy": 3710,
       "eastern": 3711,
       "##eas": 3735,
       "##east": 3736,
       "##ease": 3737,
+      "##nn": 3738,
+      "##die": 3739,
+      "##rad": 3740,
+      "##rum": 3741,
+      "##ten": 3742,
+      "##aver": 3743,
+      "##hi": 3744,
+      "##hib": 3745,
+      "##haps": 3746,
+      "##lan": 3747,
+      "##lom": 3748,
+      "##ption": 3749,
+      "##wer": 3750,
+      "##wich": 3751,
+      "##ma": 3752,
       "##va": 3753,
       "thing": 3754,
       "think": 3755,
       "33": 3954,
       "49": 3955,
       "46": 3956,
+      "52": 3957,
+      "54": 3958,
       "800": 3959,
       "96": 3960,
       "auction": 3961,
       "vod": 3998,
       "wm": 3999,
       "##ead": 4000,
+      "##cat": 4001,
+      "##cle": 4002,
+      "##ction": 4003,
+      "##cats": 4004,
+      "##craft": 4005,
+      "##ua": 4006,
+      "##ze": 4007,
+      "##zona": 4008,
+      "##oes": 4009,
+      "##ift": 4010,
+      "##ique": 4011,
+      "##gn": 4012,
+      "##gs": 4013,
+      "##ym": 4014,
+      "##yond": 4015,
+      "##held": 4016,
+      "##lig": 4017,
+      "##py": 4018,
+      "##pts": 4019,
+      "##fort": 4020,
+      "##mens": 4021,
+      "##bod": 4022,
       "##0s": 4023,
       "thom": 4024,
       "##oring": 4025,
       "wizards": 4299,
       "##ein": 4300,
       "##ever": 4301,
+      "##cing": 4302,
+      "##col": 4303,
+      "##za": 4304,
+      "##nam": 4305,
+      "##dick": 4306,
+      "##dden": 4307,
+      "##rd": 4308,
+      "##ray": 4309,
+      "##right": 4310,
+      "##tom": 4311,
+      "##ien": 4312,
+      "##iro": 4313,
+      "##iod": 4314,
+      "##jay": 4315,
+      "##gar": 4316,
+      "##au": 4317,
+      "##hran": 4318,
+      "##lins": 4319,
+      "##pm": 4320,
+      "##pool": 4321,
+      "##my": 4322,
+      "##ml": 4323,
+      "##mos": 4324,
       "##ern": 4325,
       "##iny": 4326,
       "##inter": 4327,
       "wide": 4627,
       "yud": 4628,
       "##ew": 4629,
+      "##cus": 4630,
+      "##cop": 4631,
+      "##ule": 4632,
+      "##oon": 4633,
+      "##oom": 4634,
+      "##dney": 4635,
+      "##ril": 4636,
+      "##ruit": 4637,
+      "##iet": 4638,
+      "##sf": 4639,
+      "##sler": 4640,
+      "##syl": 4641,
+      "##gether": 4642,
+      "##giving": 4643,
+      "##ae": 4644,
+      "##aur": 4645,
+      "##ko": 4646,
+      "##kn": 4647,
+      "##ht": 4648,
+      "##hy": 4649,
+      "##hoy": 4650,
+      "##wart": 4651,
+      "##work": 4652,
+      "##wait": 4653,
+      "##fe": 4654,
+      "##mers": 4655,
+      "##burn": 4656,
+      "##backs": 4657,
       "##ero": 4658,
       "##inet": 4659,
       "##ono": 4660,
       "yard": 5008,
       "zimb": 5009,
       "##eer": 5010,
+      "##cos": 5011,
+      "##cell": 5012,
+      "##care": 5013,
+      "##uan": 5014,
+      "##ued": 5015,
+      "##uite": 5016,
+      "##zen": 5017,
+      "##zai": 5018,
+      "##of": 5019,
+      "##tv": 5020,
+      "##iation": 5021,
+      "##jects": 5022,
+      "##ground": 5023,
+      "##ki": 5024,
+      "##kel": 5025,
+      "##hard": 5026,
+      "##horn": 5027,
+      "##lers": 5028,
+      "##light": 5029,
+      "##liest": 5030,
+      "##pp": 5031,
+      "##web": 5032,
       "##fc": 5033,
+      "##me": 5034,
+      "##mad": 5035,
+      "##mand": 5036,
+      "##bury": 5037,
       "##inct": 5038,
       "##esco": 5039,
       "##orary": 5040,
       "anton": 5053,
       "##iled": 5054,
       "##ayan": 5055,
+      "##oud": 5056,
+      "##oum": 5057,
       "##outh": 5058,
       "##eline": 5059,
       "##omm": 5060,
       "altern": 5103,
       "##eman": 5104,
       "##emony": 5105,
+      "##umn": 5106,
+      "##umi": 5107,
       "##iana": 5108,
       "hass": 5109,
       "upset": 5110,
       "\"\\": 5416,
       "47": 5417,
       "43": 5418,
+      "53": 5419,
+      "56": 5420,
       "550": 5421,
       "64": 5422,
       "79": 5423,
       "zur": 5508,
       "##eim": 5509,
       "##eals": 5510,
+      "##cr": 5511,
+      "##cling": 5512,
+      "##ois": 5513,
+      "##netw": 5514,
+      "##dy": 5515,
+      "##ror": 5516,
+      "##rel": 5517,
+      "##rate": 5518,
+      "##rick": 5519,
+      "##rating": 5520,
+      "##iop": 5521,
+      "##iem": 5522,
+      "##gas": 5523,
+      "##yg": 5524,
+      "##yan": 5525,
+      "##aq": 5526,
+      "##aire": 5527,
+      "##hen": 5528,
+      "##hol": 5529,
+      "##hood": 5530,
+      "##house": 5531,
+      "##lon": 5532,
+      "##ple": 5533,
+      "##pose": 5534,
+      "##pread": 5535,
+      "##wr": 5536,
+      "##wom": 5537,
+      "##well": 5538,
+      "##maker": 5539,
+      "##bus": 5540,
+      "##break": 5541,
       "##vic": 5542,
       "##40": 5543,
       "##erv": 5544,
       "##ocol": 5611,
       "concess": 5612,
       "conrad": 5613,
+      "unc": 5614,
+      "und": 5615,
+      "unw": 5616,
       "units": 5617,
       "que": 5618,
       "quality": 5619,
       "37": 6025,
       "3rd": 6026,
       "59": 6027,
+      "63": 6028,
+      "65": 6029,
+      "66": 6030,
       "71": 6031,
       "81": 6032,
       "88": 6033,
       "ele": 6057,
       "ech": 6058,
       "erupted": 6059,
+      "fd": 6060,
+      "ft": 6061,
+      "fm": 6062,
       "fan": 6063,
       "fash": 6064,
       "fals": 6065,
       "hart": 6076,
       "hok": 6077,
       "hiring": 6078,
+      "ic": 6079,
+      "ii": 6080,
       "ioc": 6081,
       "ill": 6082,
       "ji": 6083,
       "lacks": 6100,
       "lux": 6101,
       "lley": 6102,
+      "mh": 6103,
+      "ml": 6104,
       "mates": 6105,
       "moss": 6106,
       "mey": 6107,
       "wave": 6146,
       "woes": 6147,
       "##eason": 6148,
+      "##ca": 6149,
+      "##case": 6150,
+      "##qt": 6151,
+      "##zed": 6152,
+      "##zie": 6153,
+      "##zone": 6154,
+      "##zuk": 6155,
+      "##oit": 6156,
+      "##ni": 6157,
+      "##nic": 6158,
+      "##ding": 6159,
+      "##rim": 6160,
+      "##rors": 6161,
+      "##tar": 6162,
+      "##town": 6163,
+      "##iar": 6164,
+      "##ias": 6165,
+      "##ios": 6166,
+      "##sin": 6167,
+      "##jected": 6168,
+      "##joy": 6169,
+      "##yad": 6170,
+      "##ao": 6171,
+      "##kin": 6172,
+      "##hill": 6173,
+      "##hman": 6174,
+      "##lor": 6175,
+      "##lison": 6176,
+      "##lana": 6177,
+      "##po": 6178,
+      "##pes": 6179,
+      "##pass": 6180,
+      "##pped": 6181,
+      "##wag": 6182,
+      "##western": 6183,
+      "##wau": 6184,
+      "##fall": 6185,
+      "##\\\\": 6186,
+      "##mes": 6187,
+      "##mine": 6188,
+      "##bon": 6189,
+      "##bel": 6190,
+      "##bos": 6191,
+      "##bour": 6192,
+      "##base": 6193,
+      "##bek": 6194,
+      "##boy": 6195,
+      "##bank": 6196,
+      "##body": 6197,
+      "##vin": 6198,
       "##--": 6199,
+      "##erate": 6200,
+      "##erie": 6201,
+      "these": 6202,
+      "##iners": 6203,
+      "##inters": 6204,
+      "##onz": 6205,
       "##oni": 6206,
+      "##oned": 6207,
+      "##orox": 6208,
+      "##anges": 6209,
+      "##anova": 6210,
+      "##arr": 6211,
+      "##arus": 6212,
+      "##enque": 6213,
+      "##attered": 6214,
+      "##atively": 6215,
+      "##alo": 6216,
+      "##alore": 6217,
+      "##iture": 6218,
+      "##itting": 6219,
+      "##icts": 6220,
+      "##icut": 6221,
+      "##icate": 6222,
+      "##ician": 6223,
+      "##icits": 6224,
+      "##stal": 6225,
+      "##stream": 6226,
+      "##roc": 6227,
+      "##roll": 6228,
+      "##ilo": 6229,
+      "##ilit": 6230,
+      "##ayed": 6231,
+      "##ayers": 6232,
+      "##asy": 6233,
+      "##eling": 6234,
+      "##elps": 6235,
+      "##isk": 6236,
+      "##isan": 6237,
+      "##isign": 6238,
+      "##eced": 6239,
+      "##ecially": 6240,
+      "##eton": 6241,
+      "##idia": 6242,
+      "##urers": 6243,
+      "andrew": 6244,
+      "##entication": 6245,
+      "stab": 6246,
+      "stays": 6247,
+      "stations": 6248,
+      "forest": 6249,
+      "##rek": 6250,
+      "revel": 6251,
+      "##ading": 6252,
+      "##irts": 6253,
+      "##irut": 6254,
+      "##iristan": 6255,
+      "##ols": 6256,
+      "##ola": 6257,
+      "##olph": 6258,
+      "##aming": 6259,
+      "##ambi": 6260,
+      "##utt": 6261,
+      "##uty": 6262,
+      "##utor": 6263,
+      "##vement": 6264,
+      "##chrysler": 6265,
+      "##owled": 6266,
+      "chec": 6267,
+      "chess": 6268,
+      "chave": 6269,
+      "chances": 6270,
+      "##rah": 6271,
+      "##raib": 6272,
+      "##ravel": 6273,
+      "##imler": 6274,
+      "##imming": 6275,
+      "bear": 6276,
+      "beirut": 6277,
+      "##osh": 6278,
+      "##osed": 6279,
+      "##osys": 6280,
+      "dement": 6281,
+      "deuts": 6282,
+      "##aka": 6283,
+      "##earer": 6284,
+      "newest": 6285,
+      "##agers": 6286,
+      "##ande": 6287,
+      "##essary": 6288,
+      "shy": 6289,
+      "sher": 6290,
+      "shake": 6291,
+      "shape": 6292,
+      "shrek": 6293,
+      "shearer": 6294,
+      "apart": 6295,
+      "thatcher": 6296,
+      "##ully": 6297,
+      "##ocating": 6298,
+      "conc": 6299,
+      "constr": 6300,
+      "unbeat": 6301,
+      "unesco": 6302,
+      "sab": 6303,
+      "saving": 6304,
+      "queen": 6305,
+      "quiet": 6306,
+      "##aine": 6307,
+      "seym": 6308,
+      "lew": 6309,
+      "lehman": 6310,
+      "placed": 6311,
+      "itunes": 6312,
+      "itanium": 6313,
+      "ali": 6314,
+      "##emn": 6315,
+      "enable": 6316,
+      "enjoy": 6317,
+      "##umed": 6318,
+      "##iant": 6319,
+      "hasn": 6320,
+      "sue": 6321,
+      "suzuk": 6322,
+      "upon": 6323,
+      "upgrade": 6324,
+      "worry": 6325,
+      "clorox": 6326,
+      "ackn": 6327,
+      "wealth": 6328,
+      "franch": 6329,
+      "overwh": 6330,
+      "##artoum": 6331,
+      "##inea": 6332,
+      "havan": 6333,
+      "##ellar": 6334,
+      "amster": 6335,
+      "##ensions": 6336,
+      "expensive": 6337,
+      "opin": 6338,
+      "opel": 6339,
+      "opec": 6340,
+      "fires": 6341,
+      "income": 6342,
+      "reporter": 6343,
+      "marks": 6344,
+      "marqu": 6345,
+      "martha": 6346,
+      "##ifts": 6347,
+      "##abled": 6348,
+      "##abeth": 6349,
+      "resist": 6350,
+      "intense": 6351,
+      "twins": 6352,
+      "serve": 6353,
+      "serena": 6354,
+      "outcome": 6355,
+      "##anson": 6356,
+      "winless": 6357,
+      "sco": 6358,
+      "scen": 6359,
+      "scam": 6360,
+      "scare": 6361,
+      "scale": 6362,
+      "orland": 6363,
+      "blu": 6364,
+      "blocks": 6365,
+      "bloom": 6366,
+      "##onduct": 6367,
+      "aren": 6368,
+      "arth": 6369,
+      "argent": 6370,
+      "coverage": 6371,
+      "##achi": 6372,
+      "##aching": 6373,
+      "adam": 6374,
+      "adap": 6375,
+      "brid": 6376,
+      "brings": 6377,
+      "brink": 6378,
+      "##kee": 6379,
+      "##keys": 6380,
+      "recruit": 6381,
+      "trend": 6382,
+      "true": 6383,
+      "laur": 6384,
+      "##olds": 6385,
+      "##ograph": 6386,
+      "button": 6387,
+      "peru": 6388,
+      "perform": 6389,
+      "disk": 6390,
+      "disaster": 6391,
+      "##iction": 6392,
+      "teens": 6393,
+      "manned": 6394,
+      "present": 6395,
+      "presented": 6396,
+      "preseason": 6397,
+      "cancel": 6398,
+      "appar": 6399,
+      "appear": 6400,
+      "appointment": 6401,
+      "corr": 6402,
+      "##ampton": 6403,
+      "##active": 6404,
+      "invit": 6405,
+      "ellison": 6406,
+      "accel": 6407,
+      "relatives": 6408,
+      "sunni": 6409,
+      "##ashing": 6410,
+      "##under": 6411,
+      "phelps": 6412,
+      "securing": 6413,
+      "marketw": 6414,
+      "milan": 6415,
+      "milwau": 6416,
+      "warplanes": 6417,
+      "defence": 6418,
+      "defends": 6419,
+      "desper": 6420,
+      "soars": 6421,
+      "##azi": 6422,
+      "##azin": 6423,
+      "teaming": 6424,
+      "carried": 6425,
+      "considers": 6426,
+      "joy": 6427,
+      "slug": 6428,
+      "surger": 6429,
+      "assur": 6430,
+      "assess": 6431,
+      "militia": 6432,
+      "quarterf": 6433,
+      "caps": 6434,
+      "capability": 6435,
+      "redmond": 6436,
+      "genoc": 6437,
+      "regain": 6438,
+      "101": 6439,
+      "recording": 6440,
+      "governments": 6441,
+      "norway": 6442,
+      "swing": 6443,
+      "dealers": 6444,
+      "briton": 6445,
+      "searches": 6446,
+      "producers": 6447,
+      "talent": 6448,
+      "impress": 6449,
+      "barrier": 6450,
+      "barrage": 6451,
+      "billionair": 6452,
+      "billionaire": 6453,
+      "ministers": 6454,
+      "seasons": 6455,
+      "helping": 6456,
+      "demonstr": 6457,
+      "cola": 6458,
+      "guinea": 6459,
+      "youth": 6460,
+      "smoot": 6461,
+      "supplier": 6462,
+      "remember": 6463,
+      "##ront": 6464,
+      "raf": 6465,
+      "raids": 6466,
+      "##ennial": 6467,
+      "confusion": 6468,
+      "bigley": 6469,
+      "poised": 6470,
+      "steady": 6471,
+      "courts": 6472,
+      "execution": 6473,
+      "fallen": 6474,
+      "trapped": 6475,
+      "inside": 6476,
+      "managers": 6477,
+      "mobiles": 6478,
+      "belarus": 6479,
+      "israelis": 6480,
+      "antitr": 6481,
+      "evil": 6482,
+      "##yss": 6483,
+      "largely": 6484,
+      "missile": 6485,
+      "thrown": 6486,
+      "throws": 6487,
+      "targeted": 6488,
+      "targeting": 6489,
+      "developers": 6490,
+      "industries": 6491,
+      "older": 6492,
+      "vienna": 6493,
+      "depot": 6494,
+      "filip": 6495,
+      "coaching": 6496,
+      "eddie": 6497,
+      "streets": 6498,
+      "batting": 6499,
+      "196": 6500,
+      "113": 6501,
+      "##gests": 6502,
+      "update2": 6503,
       "update1": 6504,
+      "hernn": 6505,
+      "beaten": 6506,
+      "providing": 6507,
+      "imm": 6508,
+      "immun": 6509,
+      "efficient": 6510,
+      "verisign": 6511,
+      "exploit": 6512,
+      "loses": 6513,
+      "approve": 6514,
+      "approves": 6515,
+      "grew": 6516,
+      "write": 6517,
+      "writing": 6518,
+      "camps": 6519,
+      "driving": 6520,
+      "harring": 6521,
+      "virenque": 6522,
+      "legend": 6523,
+      "legacy": 6524,
+      "peacekeep": 6525,
+      "draws": 6526,
+      "midwest": 6527,
+      "syn": 6528,
+      "sybase": 6529,
+      "acquired": 6530,
+      "stronger": 6531,
+      "stronghold": 6532,
+      "radical": 6533,
+      "visits": 6534,
+      "followed": 6535,
+      "idg": 6536,
+      "videot": 6537,
+      "actions": 6538,
+      "scientist": 6539,
+      "scientific": 6540,
+      "lowered": 6541,
+      "solve": 6542,
+      "threatening": 6543,
+      "severe": 6544,
+      "homers": 6545,
+      "speculation": 6546,
+      "programme": 6547,
+      "insurer": 6548,
+      "hardware": 6549,
+      "persist": 6550,
+      "promising": 6551,
+      "install": 6552,
+      "instant": 6553,
+      "250": 6554,
+      "famous": 6555,
+      "labs": 6556,
+      "labour": 6557,
+      "commissioner": 6558,
+      "expert": 6559,
+      "watching": 6560,
+      "foreigners": 6561,
+      "safer": 6562,
+      "cypr": 6563,
+      "equatorial": 6564,
+      "demanding": 6565,
+      "democrats": 6566,
+      "maintain": 6567,
+      "wells": 6568,
+      "strained": 6569,
+      "matchup": 6570,
+      "infring": 6571,
+      "exchanges": 6572,
+      "crashes": 6573,
+      "hamid": 6574,
+      "humans": 6575,
+      "loans": 6576,
+      "malicious": 6577,
+      "financing": 6578,
+      "clears": 6579,
+      "investigated": 6580,
+      "investigators": 6581,
+      "curb": 6582,
+      "privat": 6583,
+      "privacy": 6584,
+      "authentication": 6585,
+      "rivalry": 6586,
+      "employers": 6587,
+      "employee": 6588,
+      "speedway": 6589,
+      "meth": 6590,
+      "mosul": 6591,
+      "pilot": 6592,
+      "communication": 6593,
+      "pollution": 6594,
+      "richest": 6595,
+      "doubles": 6596,
+      "clean": 6597,
+      "cuba": 6598,
+      "jumps": 6599,
+      "trees": 6600,
+      "prosp": 6601,
+      "fifa": 6602,
+      "martial": 6603,
+      "multi": 6604,
+      "silver": 6605,
+      "summit": 6606,
+      "##dez": 6607,
+      "estim": 6608,
+      "estimate": 6609,
+      "establ": 6610,
+      "factory": 6611,
+      "maced": 6612,
+      "tourists": 6613,
+      "eurozone": 6614,
+      "negotiators": 6615,
+      "cheese": 6616,
+      "respond": 6617,
+      "chipset": 6618,
+      "qaida": 6619,
+      "blockade": 6620,
+      "grounds": 6621,
+      "hunt": 6622,
+      "shoppers": 6623,
+      "disputed": 6624,
+      "feature": 6625,
+      "featuring": 6626,
+      "brazilian": 6627,
+      "landed": 6628,
+      "landing": 6629,
+      "bangalore": 6630,
+      "civilian": 6631,
+      "facing": 6632,
+      "happen": 6633,
+      "locked": 6634,
+      "volume": 6635,
+      "suggests": 6636,
+      "awarded": 6637,
+      "calc": 6638,
+      "calpers": 6639,
+      "gained": 6640,
+      "choose": 6641,
+      "captors": 6642,
+      "purchased": 6643,
+      "purchases": 6644,
+      "combined": 6645,
+      "conditions": 6646,
+      "climbing": 6647,
+      "aboard": 6648,
+      "professionals": 6649,
+      "automakers": 6650,
+      "dominated": 6651,
+      "laboratory": 6652,
+      "casino": 6653,
+      "casinos": 6654,
+      "gasol": 6655,
+      "villanova": 6656,
+      "reveal": 6657,
+      "connection": 6658,
+      "attempts": 6659,
+      "travelers": 6660,
+      "currency": 6661,
+      "escaped": 6662,
+      "biotech": 6663,
+      "daimler": 6664,
+      "initi": 6665,
+      "responsibility": 6666,
+      "tsb": 6667,
+      "clients": 6668,
+      "screens": 6669,
+      "manufacturers": 6670,
+      "knocked": 6671,
+      "burns": 6672,
+      "bullet": 6673,
+      "faith": 6674,
+      "fault": 6675,
+      "figure": 6676,
+      "khat": 6677,
+      "khartoum": 6678,
+      "occur": 6679,
+      "punish": 6680,
+      "staying": 6681,
+      "clashes": 6682,
+      "pleaded": 6683,
+      "crowd": 6684,
+      "overseas": 6685,
+      "rubble": 6686,
+      "##fielder": 6687,
+      "plunge": 6688,
+      "residents": 6689,
+      "survival": 6690,
+      "charlie": 6691,
+      "introduces": 6692,
+      "alexander": 6693,
+      "learning": 6694,
+      "learned": 6695,
+      "quitting": 6696,
+      "pledges": 6697,
+      "cracking": 6698,
+      "detainees": 6699,
+      "belgium": 6700,
+      "subscriber": 6701,
+      "momentum": 6702,
+      "destr": 6703,
+      "integration": 6704,
+      "celebrate": 6705,
+      "celebrated": 6706,
+      "brewer": 6707,
+      "madison": 6708,
+      "pacers": 6709,
+      "spurrier": 6710,
+      "lebanon": 6711,
+      "mozambi": 6712,
+      "described": 6713,
+      "carrying": 6714,
+      "swedish": 6715,
+      "broadcasting": 6716,
+      "roadside": 6717,
+      "##bridge": 6718,
+      "counterfe": 6719,
+      "observ": 6720,
+      "venezuela": 6721,
+      "venezuelans": 6722,
+      "bryant": 6723,
+      "mystery": 6724,
+      "randolph": 6725,
+      "vaugh": 6726,
+      "reasons": 6727,
+      "split": 6728,
+      "washed": 6729,
+      "brokerage": 6730,
+      "feeling": 6731,
+      "granted": 6732,
+      "astrona": 6733,
+      "diplomats": 6734,
+      "wmc": 6735,
+      "beheaded": 6736,
+      "property": 6737,
+      "inventories": 6738,
+      "slumping": 6739,
+      "delivery": 6740,
+      "guerrill": 6741,
+      "wrongdo": 6742,
+      "espn": 6743,
+      "especially": 6744,
+      "achie": 6745,
+      "coca": 6746,
+      "pieces": 6747,
+      "penalty": 6748,
+      "tackle": 6749,
+      "tossed": 6750,
+      "coalition": 6751,
+      "phillips": 6752,
+      "website": 6753,
+      "removed": 6754,
+      "throwing": 6755,
+      "refugee": 6756,
+      "citizens": 6757,
+      "coughlin": 6758,
+      "culture": 6759,
+      "grants": 6760,
+      "pedro": 6761,
+      "sgi": 6762,
+      "tipped": 6763,
+      "degree": 6764,
+      "admitted": 6765,
+      "floods": 6766,
+      "mohammad": 6767,
+      "guidant": 6768,
+      "trouble": 6769,
+      "batteries": 6770,
+      "copyright": 6771,
+      "connecticut": 6772,
+      "temporary": 6773,
+      "favorite": 6774,
+      "craig": 6775,
+      "gibbs": 6776,
+      "honda": 6777,
+      "nvidia": 6778,
+      "nedbank": 6779,
+      "rwanda": 6780,
+      "roster": 6781,
+      "##mandu": 6782,
+      "primary": 6783,
+      "exclusive": 6784,
+      "sponsored": 6785,
+      "alternative": 6786,
+      "clothing": 6787,
+      "database": 6788,
+      "telephony": 6789,
+      "hammered": 6790,
+      "ferrari": 6791,
+      "pharmaceutical": 6792,
+      "ghraib": 6793,
+      "hughes": 6794,
+      "nortel": 6795,
+      "rossi": 6796,
+      "veteran": 6797,
+      "##standing": 6798,
+      "anthony": 6799,
+      "stroke": 6800,
+      "proxy": 6801,
+      "concessions": 6802,
+      "queens": 6803,
+      "enforcement": 6804,
+      "freddie": 6805,
+      "overcame": 6806,
+      "serbia": 6807,
+      "elsewhere": 6808,
+      "cafes": 6809,
+      "rebuild": 6810,
+      "himalayan": 6811,
+      "distraction": 6812,
+      "experimental": 6813,
+      "friendly": 6814,
+      "overshadow": 6815,
+      "cavaliers": 6816,
+      "dallas": 6817,
+      "daily": 6818,
+      "fashion": 6819,
+      "giambi": 6820,
+      "hockey": 6821,
+      "hartford": 6822,
+      "hokies": 6823,
+      "kazaa": 6824,
+      "kathmandu": 6825,
+      "karmazin": 6826,
+      "loyal": 6827,
+      "lleyton": 6828,
+      "mhz": 6829,
+      "meyer": 6830,
+      "pfiz": 6831,
+      "p2p": 6832,
+      "ponting": 6833,
+      "rfid": 6834,
+      "svetlana": 6835,
+      "waziristan": 6836,
+      "wysiw": 6837,
+      "##qtada": 6838,
       "##owledged": 6839,
       "chavez": 6840,
       "deutsche": 6841,
       "650": 6864,
       "7e": 6865,
       "77": 6866,
+      "84": 6867,
+      "86": 6868,
       "93": 6869,
       "950": 6870,
+      "ak": 6871,
+      "a9": 6872,
       "aet": 6873,
       "aven": 6874,
       "bug": 6875,
       "ol": 6951,
       "oce": 6952,
       "ori": 6953,
+      "pd": 6954,
+      "pk": 6955,
       "pad": 6956,
       "pest": 6957,
       "pist": 6958,
       "tying": 6983,
       "tumb": 6984,
       "tumble": 6985,
+      "ug": 6986,
+      "um": 6987,
       "u2": 6988,
       "van": 6989,
       "vanc": 6990,
       "zid": 7004,
       "ziff": 7005,
       "##ean": 7006,
+      "##cu": 7007,
+      "##cers": 7008,
+      "##cair": 7009,
+      "##crew": 7010,
+      "##uals": 7011,
+      "##uana": 7012,
+      "##zer": 7013,
+      "##zard": 7014,
+      "##oin": 7015,
+      "##oke": 7016,
+      "##net": 7017,
+      "##nom": 7018,
+      "##nants": 7019,
+      "##nology": 7020,
+      "##due": 7021,
+      "##dale": 7022,
+      "##ression": 7023,
+      "##render": 7024,
+      "##iate": 7025,
+      "##iber": 7026,
+      "##iece": 7027,
+      "##sm": 7028,
+      "##sburg": 7029,
+      "##gb": 7030,
+      "##gom": 7031,
+      "##gus": 7032,
+      "##gard": 7033,
+      "##gment": 7034,
+      "##gart": 7035,
+      "##greg": 7036,
+      "##yton": 7037,
+      "##aug": 7038,
+      "##aware": 7039,
+      "##kok": 7040,
+      "##killer": 7041,
+      "##hes": 7042,
+      "##had": 7043,
+      "##hocks": 7044,
+      "##pin": 7045,
+      "##pen": 7046,
+      "##part": 7047,
+      "##ports": 7048,
+      "##pson": 7049,
+      "##wl": 7050,
+      "##wol": 7051,
+      "##wire": 7052,
+      "##water": 7053,
+      "##waves": 7054,
+      "##fa": 7055,
+      "##function": 7056,
+      "##mu": 7057,
+      "##mare": 7058,
+      "##mail": 7059,
+      "##bar": 7060,
+      "##bits": 7061,
+      "##buster": 7062,
+      "##bara": 7063,
+      "##voc": 7064,
       "##xious": 7065,
       "##35": 7066,
       "##erious": 7067,
       "monetary": 7237,
       "haven": 7238,
       "haiti": 7239,
+      "hazard": 7240,
+      "havoc": 7241,
       "##ella": 7242,
       "amr": 7243,
       "amelie": 7244,
       "really": 7492,
       "realnetw": 7493,
       "147": 7494,
+      "148": 7495,
       "140": 7496,
+      "146": 7497,
       "household": 7498,
       "mcken": 7499,
       "visa": 7500,
       "focusing": 7655,
       "copper": 7656,
       "fixes": 7657,
+      "##while": 7658,
+      "##market": 7659,
       "towns": 7660,
       "suffering": 7661,
       "declares": 7662,
       "astronomers": 7889,
       "neighborhood": 7890,
       "dhaka": 7891,
+      "fdx": 7892,
+      "ftc": 7893,
       "false": 7894,
       "goran": 7895,
       "gurunet": 7896,
       "1b": 7989,
       "1st": 7990,
       "166": 7991,
+      "72": 7992,
+      "76": 7993,
       "92": 7994,
       "95": 7995,
       "900": 7996,

tokenizer_config.json CHANGED Viewed

@@ -4,10 +4,10 @@
   "do_lower_case": true,
   "mask_token": "[MASK]",
   "model_max_length": 1000000000000000019884624838656,
-  "pad_token": "[PAD]",
   "sep_token": "[SEP]",
   "strip_accents": null,
   "tokenize_chinese_chars": true,
   "tokenizer_class": "BertTokenizer",
-  "unk_token": "[UNK]"
 }

   "do_lower_case": true,
   "mask_token": "[MASK]",
   "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<redacted_PAD>",
   "sep_token": "[SEP]",
   "strip_accents": null,
   "tokenize_chinese_chars": true,
   "tokenizer_class": "BertTokenizer",
+  "unk_token": "<redacted_UNK>"
 }