fixed readme

README.md

@@ -1,9 +1,17 @@
+---
+language: english
+widget:
+- text: "They 're a young team . they have great players and amazing freshmen coming in , so think they 'll grow into themselves next year ,"
+- text: "\" We 'll talk go by now ; \" says Shucksmith ;"
+- text: "\" Warren Gatland is a professional person and it wasn 't a case of 's I 'll phone my mate Rob up to if he wants a coaching job ' , he would done a fair amount of homework about , \" Howley air said ."
+---
+
 This model can be used to more accurately detokenize the moses tokenizer (it does a better job with certain lossy quotes and things)
 
 
 batched usage: 
 
-```
+```python
 
 sentences = [
     "They 're a young team . they have great players and amazing freshmen coming in , so think they 'll grow into themselves next year ,",
@@ -23,10 +31,7 @@ def sentences_to_input_tokens(sentences):
     max_length = 0
     sents_tokens = []
     iids = tokenizer(sentences)
-    for sent_tokens in iids['input_ids']:
-#         sent_tokens = tokenizer.build_inputs_with_special_tokens(
-#             tokenizer.convert_tokens_to_ids(tokenizer.tokenize(sentence)))
-        
+    for sent_tokens in iids['input_ids']:        
         sents_tokens.append(sent_tokens)
         
         if len(sent_tokens) > max_length: