Repository: OpenNMT/OpenNMT-py

Issue:
AttributeError: 'TextMultiField' object has no attribute 'vocab'
I preprocessed the data by this command:
`python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo`

then tried to load the Glove embeddings by this command:
`python ./tools/embeddings_to_torch.py -emb_file_enc "glove_dir/glove.6B.100d.txt" -emb_file_dec "glove_dir/glove.6B.100d.txt" -dict_file "data/demo.vocab.pt" -output_file "data/demo_gloveembeddings" `

but, got the following error:
```
Traceback (most recent call last):
  File "./tools/embeddings_to_torch.py", line 125, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 83, in main
    enc_vocab, dec_vocab = get_vocabs(opt.dict_file)
  File "./tools/embeddings_to_torch.py", line 20, in get_vocabs
    enc_vocab = fields['src'][0][1].vocab
AttributeError: 'TextMultiField' object has no attribute 'vocab'
```

Did I miss something? or is it due to the compatibility of vocab files between the  current version of preprocessing.py and embeddings_to_torch.py?
--------------------------------------------------------
Looked a bit more into this ... it looks like at some point the onmt.inputters.text_dataset.TextMultiField class has changed to remove the "vocab" attribute, but only have "fields" attribute now. 

>>> import torch
>>> fields = torch.load("data/demo.vocab.pt")
>>> print (fields['src'][0][1])
<onmt.inputters.text_dataset.TextMultiField object at 0x7fb527440860>
>>> print (fields['src'][0][1].fields[0][1].vocab)
<torchtext.vocab.Vocab object at 0x7fb4c778f7f0>

AttributeError: 'TextMultiField' object has no attribute 'vocab'
I preprocessed the data by this command:
`python preprocess.py -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo`

then tried to load the Glove embeddings by this command:
`python ./tools/embeddings_to_torch.py -emb_file_enc "glove_dir/glove.6B.100d.txt" -emb_file_dec "glove_dir/glove.6B.100d.txt" -dict_file "data/demo.vocab.pt" -output_file "data/demo_gloveembeddings" `

but, got the following error:
```
Traceback (most recent call last):
  File "./tools/embeddings_to_torch.py", line 125, in <module>
    main()
  File "./tools/embeddings_to_torch.py", line 83, in main
    enc_vocab, dec_vocab = get_vocabs(opt.dict_file)
  File "./tools/embeddings_to_torch.py", line 20, in get_vocabs
    enc_vocab = fields['src'][0][1].vocab
AttributeError: 'TextMultiField' object has no attribute 'vocab'
```

Did I miss something? or is it due to the compatibility of vocab files between the  current version of preprocessing.py and embeddings_to_torch.py?
--------------------------------------------------------
Looked a bit more into this ... it looks like at some point the onmt.inputters.text_dataset.TextMultiField class has changed to remove the "vocab" attribute, but only have "fields" attribute now. 

>>> import torch
>>> fields = torch.load("data/demo.vocab.pt")
>>> print (fields['src'][0][1])
<onmt.inputters.text_dataset.TextMultiField object at 0x7fb527440860>
>>> print (fields['src'][0][1].fields[0][1].vocab)
<torchtext.vocab.Vocab object at 0x7fb4c778f7f0>



Discussion:
Probably, yes, @flauted can yo please have a look ?
Yeah ``embeddings_to_torch.py`` never got updated after #1216. Thanks for the details. I'll open a PR shortly.
Probably, yes, @flauted can yo please have a look ?
Yeah ``embeddings_to_torch.py`` never got updated after #1216. Thanks for the details. I'll open a PR shortly.

Resolving patch:
diff --git a/tools/create_vocabulary.py b/tools/create_vocabulary.py
--- a/tools/create_vocabulary.py
+++ b/tools/create_vocabulary.py
@@ -73,14 +73,25 @@ def main():
                 f.write("{0}\n".format(w))
     else:
         import torch
+        from onmt.inputters.inputter import _old_style_vocab
         print("Reading input file...")
+        if not len(opt.file) == 1:
+            raise ValueError("If using -file_type='field', only pass one "
+                             "argument for -file.")
         vocabs = torch.load(opt.file[0])
-        word_list = dict(vocabs)[opt.side].itos
+        voc = dict(vocabs)[opt.side]
+        if _old_style_vocab(voc):
+            word_list = voc.itos
+        else:
+            try:
+                word_list = voc[0][1].base_field.vocab.itos
+            except AttributeError:
+                word_list = voc[0][1].vocab.itos
 
         print("Writing vocabulary file...")
-        with open(opt.out_file, "w") as f:
+        with open(opt.out_file, "wb") as f:
             for w in word_list:
-                f.write("{0}\n".format(w))
+                f.write(u"{0}\n".format(w).encode("utf-8"))
 
 
 if __name__ == "__main__":
diff --git a/tools/embeddings_to_torch.py b/tools/embeddings_to_torch.py
--- a/tools/embeddings_to_torch.py
+++ b/tools/embeddings_to_torch.py
@@ -7,18 +7,23 @@
 import argparse
 import torch
 from onmt.utils.logging import init_logger, logger
-from onmt.inputters.inputter import old_style_vocab
+from onmt.inputters.inputter import _old_style_vocab
 
 
 def get_vocabs(dict_path):
     fields = torch.load(dict_path)
 
-    if old_style_vocab(fields):
-        enc_vocab = next((v for n, v in fields if n == 'src'), None)
-        dec_vocab = next((v for n, v in fields if n == 'tgt'), None)
-    else:
-        enc_vocab = fields['src'][0][1].vocab
-        dec_vocab = fields['tgt'][0][1].vocab
+    vocs = []
+    for side in ['src', 'tgt']:
+        if _old_style_vocab(fields):
+            vocab = next((v for n, v in fields if n == side), None)
+        else:
+            try:
+                vocab = fields[side][0][1].base_field.vocab
+            except AttributeError:
+                vocab = fields[side][0][1].vocab
+        vocs.append(vocab)
+    enc_vocab, dec_vocab = vocs
 
     logger.info("From: %s" % dict_path)
     logger.info("\t* source vocab: %d words" % len(enc_vocab))
diff --git a/tools/extract_embeddings.py b/tools/extract_embeddings.py
--- a/tools/extract_embeddings.py
+++ b/tools/extract_embeddings.py
@@ -48,7 +48,7 @@ def main():
         fields = onmt.inputters.load_old_vocab(vocab)
     else:
         fields = vocab
-    src_dict = fields['src'][0][1].base_field.vocab
+    src_dict = fields['src'][0][1].base_field.vocab  # assumes src is text
     tgt_dict = fields['tgt'][0][1].base_field.vocab
 
     model_opt = checkpoint['opt']


Test patch:
diff --git a/onmt/tests/pull_request_chk.sh b/onmt/tests/pull_request_chk.sh
--- a/onmt/tests/pull_request_chk.sh
+++ b/onmt/tests/pull_request_chk.sh
@@ -272,5 +272,64 @@ ${PYTHON} train.py -model_type audio \
 echo "Succeeded" | tee -a ${LOG_FILE}
 
 
+echo -n "[+] Doing create vocabulary {preprocess + create_vocabulary} test..."
+rm /tmp/src-train.txt
+rm /tmp/tgt-train.txt
+rm /tmp/src-val.txt
+rm /tmp/tgt-val.txt
+head ${DATA_DIR}/src-train.txt > /tmp/src-train.txt
+head ${DATA_DIR}/tgt-train.txt > /tmp/tgt-train.txt
+head ${DATA_DIR}/src-val.txt > /tmp/src-val.txt
+head ${DATA_DIR}/tgt-val.txt > /tmp/tgt-val.txt
+
+rm -rf /tmp/q*pt
+${PYTHON} preprocess.py -train_src /tmp/src-train.txt \
+		     -train_tgt /tmp/tgt-train.txt \
+		     -valid_src /tmp/src-val.txt \
+		     -valid_tgt /tmp/tgt-val.txt \
+		     -save_data /tmp/q >> ${LOG_FILE} 2>&1
+PYTHONPATH=${PROJECT_ROOT}:${PYTHONPATH} ${PYTHON} ./tools/create_vocabulary.py -file /tmp/q.vocab.pt \
+        -file_type field -out_file /tmp/vocab.txt -side src       >> ${LOG_FILE} 2>&1
+[ "$?" -eq 0 ] || error_exit
+if ! wc -l /tmp/vocab.txt | grep -qF  "181"; then
+    echo -n "wrong word count\n" >> ${LOG_FILE}
+    wc -l /tmp/vocab.txt >> ${LOG_FILE}
+    error_exit
+fi
+echo "Succeeded" | tee -a ${LOG_FILE}
+
+
+echo -n "[+] Doing embedding to torch {preprocess + embeddings_to_torch} test..."
+rm /tmp/src-train.txt
+rm /tmp/tgt-train.txt
+rm /tmp/src-val.txt
+rm /tmp/tgt-val.txt
+head ${DATA_DIR}/src-train.txt > /tmp/src-train.txt
+head ${DATA_DIR}/tgt-train.txt > /tmp/tgt-train.txt
+head ${DATA_DIR}/src-val.txt > /tmp/src-val.txt
+head ${DATA_DIR}/tgt-val.txt > /tmp/tgt-val.txt
+
+rm -rf /tmp/q*pt
+${PYTHON} preprocess.py -train_src /tmp/src-train.txt \
+		     -train_tgt /tmp/tgt-train.txt \
+		     -valid_src /tmp/src-val.txt \
+		     -valid_tgt /tmp/tgt-val.txt \
+		     -save_data /tmp/q >> ${LOG_FILE} 2>&1
+PYTHONPATH=${PROJECT_ROOT}:${PYTHONPATH} ${PYTHON} ./tools/embeddings_to_torch.py \
+        -emb_file_enc ${TEST_DIR}/sample_glove.txt \
+        -emb_file_dec ${TEST_DIR}/sample_glove.txt \
+        -dict_file /tmp/q.vocab.pt \
+        -output_file /tmp/q_gloveembeddings        >> ${LOG_FILE} 2>&1
+[ "$?" -eq 0 ] || error_exit
+echo "Succeeded" | tee -a ${LOG_FILE}
+
+
+echo -n "[+] Doing extract embeddings test..."
+PYTHONPATH=${PROJECT_ROOT}:${PYTHONPATH} ${PYTHON} tools/extract_embeddings.py \
+        -model onmt/tests/test_model.pt  >> ${LOG_FILE} 2>&1
+[ "$?" -eq 0 ] || error_exit
+echo "Succeeded" | tee -a ${LOG_FILE}
+
+
 # Finally, clean up
 clean_up
diff --git a/onmt/tests/sample_glove.txt b/onmt/tests/sample_glove.txt
new file mode 100644
--- /dev/null
+++ b/onmt/tests/sample_glove.txt
@@ -0,0 +1,20 @@
+the -0.038194 -0.24487 0.72812 -0.39961 0.083172 0.043953 -0.39141 0.3344 -0.57545 0.087459 0.28787 -0.06731 0.30906 -0.26384 -0.13231 -0.20757 0.33395 -0.33848 -0.31743 -0.48336 0.1464 -0.37304 0.34577 0.052041 0.44946 -0.46971 0.02628 -0.54155 -0.15518 -0.14107 -0.039722 0.28277 0.14393 0.23464 -0.31021 0.086173 0.20397 0.52624 0.17164 -0.082378 -0.71787 -0.41531 0.20335 -0.12763 0.41367 0.55187 0.57908 -0.33477 -0.36559 -0.54857 -0.062892 0.26584 0.30205 0.99775 -0.80481 -3.0243 0.01254 -0.36942 2.2167 0.72201 -0.24978 0.92136 0.034514 0.46745 1.1079 -0.19358 -0.074575 0.23353 -0.052062 -0.22044 0.057162 -0.15806 -0.30798 -0.41625 0.37972 0.15006 -0.53212 -0.2055 -1.2526 0.071624 0.70565 0.49744 -0.42063 0.26148 -1.538 -0.30223 -0.073438 -0.28312 0.37104 -0.25217 0.016215 -0.017099 -0.38984 0.87424 -0.72569 -0.51058 -0.52028 -0.1459 0.8278 0.27062
+, -0.10767 0.11053 0.59812 -0.54361 0.67396 0.10663 0.038867 0.35481 0.06351 -0.094189 0.15786 -0.81665 0.14172 0.21939 0.58505 -0.52158 0.22783 -0.16642 -0.68228 0.3587 0.42568 0.19021 0.91963 0.57555 0.46185 0.42363 -0.095399 -0.42749 -0.16567 -0.056842 -0.29595 0.26037 -0.26606 -0.070404 -0.27662 0.15821 0.69825 0.43081 0.27952 -0.45437 -0.33801 -0.58184 0.22364 -0.5778 -0.26862 -0.20425 0.56394 -0.58524 -0.14365 -0.64218 0.0054697 -0.35248 0.16162 1.1796 -0.47674 -2.7553 -0.1321 -0.047729 1.0655 1.1034 -0.2208 0.18669 0.13177 0.15117 0.7131 -0.35215 0.91348 0.61783 0.70992 0.23955 -0.14571 -0.37859 -0.045959 -0.47368 0.2385 0.20536 -0.18996 0.32507 -1.1112 -0.36341 0.98679 -0.084776 -0.54008 0.11726 -1.0194 -0.24424 0.12771 0.013884 0.080374 -0.35414 0.34951 -0.7226 0.37549 0.4441 -0.99059 0.61214 -0.35111 -0.83155 0.45293 0.082577
+. -0.33979 0.20941 0.46348 -0.64792 -0.38377 0.038034 0.17127 0.15978 0.46619 -0.019169 0.41479 -0.34349 0.26872 0.04464 0.42131 -0.41032 0.15459 0.022239 -0.64653 0.25256 0.043136 -0.19445 0.46516 0.45651 0.68588 0.091295 0.21875 -0.70351 0.16785 -0.35079 -0.12634 0.66384 -0.2582 0.036542 -0.13605 0.40253 0.14289 0.38132 -0.12283 -0.45886 -0.25282 -0.30432 -0.11215 -0.26182 -0.22482 -0.44554 0.2991 -0.85612 -0.14503 -0.49086 0.0082973 -0.17491 0.27524 1.4401 -0.21239 -2.8435 -0.27958 -0.45722 1.6386 0.78808 -0.55262 0.65 0.086426 0.39012 1.0632 -0.35379 0.48328 0.346 0.84174 0.098707 -0.24213 -0.27053 0.045287 -0.40147 0.11395 0.0062226 0.036673 0.018518 -1.0213 -0.20806 0.64072 -0.068763 -0.58635 0.33476 -1.1432 -0.1148 -0.25091 -0.45907 -0.096819 -0.17946 -0.063351 -0.67412 -0.068895 0.53604 -0.87773 0.31802 -0.39242 -0.23394 0.47298 -0.028803
+of -0.1529 -0.24279 0.89837 0.16996 0.53516 0.48784 -0.58826 -0.17982 -1.3581 0.42541 0.15377 0.24215 0.13474 0.41193 0.67043 -0.56418 0.42985 -0.012183 -0.11677 0.31781 0.054177 -0.054273 0.35516 -0.30241 0.31434 -0.33846 0.71715 -0.26855 -0.15837 -0.47467 0.051581 -0.33252 0.15003 -0.1299 -0.54617 -0.37843 0.64261 0.82187 -0.080006 0.078479 -0.96976 -0.57741 0.56491 -0.39873 -0.057099 0.19743 0.065706 -0.48092 -0.20125 -0.40834 0.39456 -0.02642 -0.11838 1.012 -0.53171 -2.7474 -0.042981 -0.74849 1.7574 0.59085 0.04885 0.78267 0.38497 0.42097 0.67882 0.10337 0.6328 -0.026595 0.58647 -0.44332 0.33057 -0.12022 -0.55645 0.073611 0.20915 0.43395 -0.012761 0.089874 -1.7991 0.084808 0.77112 0.63105 -0.90685 0.60326 -1.7515 0.18596 -0.50687 -0.70203 0.66578 -0.81304 0.18712 -0.018488 -0.26757 0.727 -0.59363 -0.34839 -0.56094 -0.591 1.0039 0.20664
+to -0.1897 0.050024 0.19084 -0.049184 -0.089737 0.21006 -0.54952 0.098377 -0.20135 0.34241 -0.092677 0.161 -0.13268 -0.2816 0.18737 -0.42959 0.96039 0.13972 -1.0781 0.40518 0.50539 -0.55064 0.4844 0.38044 -0.0029055 -0.34942 -0.099696 -0.78368 1.0363 -0.2314 -0.47121 0.57126 -0.21454 0.35958 -0.48319 1.0875 0.28524 0.12447 -0.039248 -0.076732 -0.76343 -0.32409 -0.5749 -1.0893 -0.41811 0.4512 0.12112 -0.51367 -0.13349 -1.1378 -0.28768 0.16774 0.55804 1.5387 0.018859 -2.9721 -0.24216 -0.92495 2.1992 0.28234 -0.3478 0.51621 -0.43387 0.36852 0.74573 0.072102 0.27931 0.92569 -0.050336 -0.85856 -0.1358 -0.92551 -0.33991 -1.0394 -0.067203 -0.21379 -0.4769 0.21377 -0.84008 0.052536 0.59298 0.29604 -0.67644 0.13916 -1.5504 -0.20765 0.7222 0.52056 -0.076221 -0.15194 -0.13134 0.058617 -0.31869 -0.61419 -0.62393 -0.41548 -0.038175 -0.39804 0.47647 -0.15983
+and -0.071953 0.23127 0.023731 -0.50638 0.33923 0.1959 -0.32943 0.18364 -0.18057 0.28963 0.20448 -0.5496 0.27399 0.58327 0.20468 -0.49228 0.19974 -0.070237 -0.88049 0.29485 0.14071 -0.1009 0.99449 0.36973 0.44554 0.28998 -0.1376 -0.56365 -0.029365 -0.4122 -0.25269 0.63181 -0.44767 0.24363 -0.10813 0.25164 0.46967 0.3755 -0.23613 -0.14129 -0.44537 -0.65737 -0.042421 -0.28636 -0.28811 0.063766 0.20281 -0.53542 0.41307 -0.59722 -0.38614 0.19389 -0.17809 1.6618 -0.011819 -2.3737 0.058427 -0.2698 1.2823 0.81925 -0.22322 0.72932 -0.053211 0.43507 0.85011 -0.42935 0.92664 0.39051 1.0585 -0.24561 -0.18265 -0.5328 0.059518 -0.66019 0.18991 0.28836 -0.2434 0.52784 -0.65762 -0.14081 1.0491 0.5134 -0.23816 0.69895 -1.4813 -0.2487 -0.17936 -0.059137 -0.08056 -0.48782 0.014487 -0.6259 -0.32367 0.41862 -1.0807 0.46742 -0.49931 -0.71895 0.86894 0.19539
+in 0.085703 -0.22201 0.16569 0.13373 0.38239 0.35401 0.01287 0.22461 -0.43817 0.50164 -0.35874 -0.34983 0.055156 0.69648 -0.17958 0.067926 0.39101 0.16039 -0.26635 -0.21138 0.53698 0.49379 0.9366 0.66902 0.21793 -0.46642 0.22383 -0.36204 -0.17656 0.1748 -0.20367 0.13931 0.019832 -0.10413 -0.20244 0.55003 -0.1546 0.98655 -0.26863 -0.2909 -0.32866 -0.34188 -0.16943 -0.42001 -0.046727 -0.16327 0.70824 -0.74911 -0.091559 -0.96178 -0.19747 0.10282 0.55221 1.3816 -0.65636 -3.2502 -0.31556 -1.2055 1.7709 0.4026 -0.79827 1.1597 -0.33042 0.31382 0.77386 0.22595 0.52471 -0.034053 0.32048 0.079948 0.17752 -0.49426 -0.70045 -0.44569 0.17244 0.20278 0.023292 -0.20677 -1.0158 0.18325 0.56752 0.31821 -0.65011 0.68277 -0.86585 -0.059392 -0.29264 -0.55668 -0.34705 -0.32895 0.40215 -0.12746 -0.20228 0.87368 -0.545 0.79205 -0.20695 -0.074273 0.75808 -0.34243
+a -0.27086 0.044006 -0.02026 -0.17395 0.6444 0.71213 0.3551 0.47138 -0.29637 0.54427 -0.72294 -0.0047612 0.040611 0.043236 0.29729 0.10725 0.40156 -0.53662 0.033382 0.067396 0.64556 -0.085523 0.14103 0.094539 0.74947 -0.194 -0.68739 -0.41741 -0.22807 0.12 -0.48999 0.80945 0.045138 -0.11898 0.20161 0.39276 -0.20121 0.31354 0.75304 0.25907 -0.11566 -0.029319 0.93499 -0.36067 0.5242 0.23706 0.52715 0.22869 -0.51958 -0.79349 -0.20368 -0.50187 0.18748 0.94282 -0.44834 -3.6792 0.044183 -0.26751 2.1997 0.241 -0.033425 0.69553 -0.64472 -0.0072277 0.89575 0.20015 0.46493 0.61933 -0.1066 0.08691 -0.4623 0.18262 -0.15849 0.020791 0.19373 0.063426 -0.31673 -0.48177 -1.3848 0.13669 0.96859 0.049965 -0.2738 -0.035686 -1.0577 -0.24467 0.90366 -0.12442 0.080776 -0.83401 0.57201 0.088945 -0.42532 -0.018253 -0.079995 -0.28581 -0.01089 -0.4923 0.63687 0.23642
+" -0.30457 -0.23645 0.17576 -0.72854 -0.28343 -0.2564 0.26587 0.025309 -0.074775 -0.3766 -0.057774 0.12159 0.34384 0.41928 -0.23236 -0.31547 0.60939 0.25117 -0.68667 0.70873 1.2162 -0.1824 -0.48442 -0.33445 0.30343 1.086 0.49992 -0.20198 0.27959 0.68352 -0.33566 -0.12405 0.059656 0.33617 0.37501 0.56552 0.44867 0.11284 -0.16196 -0.94346 -0.67961 0.18581 0.060653 0.43776 0.13834 -0.48207 -0.56141 -0.25422 -0.52445 0.097003 -0.48925 0.19077 0.21481 1.4969 -0.86665 -3.2846 0.56854 0.41971 1.2294 0.78522 -0.29369 0.63803 -1.5926 -0.20437 1.5306 0.13548 0.50722 0.18742 0.48552 -0.28995 0.19573 0.0046515 0.092879 -0.42444 0.64987 0.52839 0.077908 0.8263 -1.2208 -0.34955 0.49855 -0.64155 -0.72308 0.26566 -1.3643 -0.46364 -0.52048 -1.0525 0.22895 -0.3456 -0.658 -0.16735 0.35158 0.74337 0.26074 0.061104 -0.39079 -0.84557 -0.035432 0.17036
+'s 0.58854 -0.2025 0.73479 -0.68338 -0.19675 -0.1802 -0.39177 0.34172 -0.60561 0.63816 -0.26695 0.36486 -0.40379 -0.1134 -0.58718 0.2838 0.8025 -0.35303 0.30083 0.078935 0.44416 -0.45906 0.79294 0.50365 0.32805 0.28027 -0.4933 -0.38482 -0.039284 -0.2483 -0.1988 1.1469 0.13228 0.91691 -0.36739 0.89425 0.5426 0.61738 -0.62205 -0.31132 -0.50933 0.23335 1.0826 -0.044637 -0.12767 0.27628 -0.032617 -0.27397 0.77764 -0.50861 0.038307 -0.33679 0.42344 1.2271 -0.53826 -3.2411 0.42626 0.025189 1.3948 0.65085 0.03325 0.37141 0.4044 0.35558 0.98265 -0.61724 0.53901 0.76219 0.30689 0.33065 0.30956 -0.15161 -0.11313 -0.81281 0.6145 -0.44341 -0.19163 -0.089551 -1.5927 0.37405 0.85857 0.54613 -0.31928 0.52598 -1.4802 -0.97931 -0.2939 -0.14724 0.25803 -0.1817 1.0149 0.77649 0.12598 0.54779 -1.0316 0.064599 -0.37523 -0.94475 0.61802 0.39591
+for -0.14401 0.32554 0.14257 -0.099227 0.72536 0.19321 -0.24188 0.20223 -0.89599 0.15215 0.035963 -0.59513 -0.051635 -0.014428 0.35475 -0.31859 0.76984 -0.087369 -0.24762 0.65059 -0.15138 -0.42703 0.18813 0.091562 0.15192 0.11303 -0.15222 -0.62786 -0.23923 0.096009 -0.46147 0.41526 -0.30475 0.1371 0.16758 0.53301 -0.043658 0.85924 -0.41192 -0.21394 -0.51228 -0.31945 0.12662 -0.3151 0.0031429 0.27129 0.17328 -1.3159 -0.42414 -0.69126 0.019017 -0.13375 -0.096057 1.7069 -0.65291 -2.6111 0.26518 -0.61178 2.095 0.38148 -0.55823 0.2036 -0.33704 0.37354 0.6951 -0.001637 0.81885 0.51793 0.27746 -0.37177 -0.43345 -0.42732 -0.54912 -0.30715 0.18101 0.2709 -0.29266 0.30834 -1.4624 -0.18999 0.92277 -0.099217 -0.25165 0.49197 -1.525 0.15326 0.2827 0.12102 -0.36766 -0.61275 -0.18884 0.10907 0.12315 0.090066 -0.65447 -0.17252 2.6336e-05 0.25398 1.1078 -0.073074
+- -1.2557 0.61036 0.56793 -0.96596 -0.45249 -0.071696 0.57122 -0.31292 -0.43814 0.90622 0.06961 -0.053104 0.25029 0.27841 0.77724 0.26329 0.56874 -1.1171 -0.078268 -0.51317 0.8071 0.99214 0.22753 1.0847 0.88292 0.17221 -0.68686 -0.86467 -0.80003 -0.34738 -0.044074 -0.30444 0.23406 0.28592 0.060548 -0.65477 -0.039738 0.74878 -0.46471 0.063023 -0.16519 -1.2217 -0.089479 -0.8125 0.27615 -0.13841 -0.76667 -0.96974 0.83123 -0.77639 -1.3327 -0.28732 -0.053684 1.1735 -1.1795 -2.7519 0.45359 1.1984 2.8203 0.060114 0.32296 0.19097 0.3459 -0.41503 0.1515 0.38148 1.619 0.9929 -0.82549 -0.098692 0.74449 -0.38602 -1.0004 -1.305 -0.31269 -0.57625 0.14095 -0.80269 -1.4714 -0.48014 1.1993 -0.48561 0.40496 -0.032867 -2.051 0.18284 -0.2723 0.043287 0.066801 -0.62832 -0.05854 0.28253 -0.083276 -0.022234 -0.55914 0.24586 0.36052 -1.5877 0.76984 -0.64998
+that -0.093337 0.19043 0.68457 -0.41548 -0.22777 -0.11803 -0.095434 0.19613 0.17785 -0.020244 -0.055409 0.33867 0.79396 -0.047126 0.44281 -0.061266 0.20796 0.034094 -0.64751 0.35874 0.13936 -0.6831 0.25596 -0.12911 0.2608 -0.11674 0.024925 -0.60259 -0.41474 -0.51104 0.14936 0.79977 -0.12716 0.40474 -0.21435 0.47031 0.49 0.48886 -0.17772 -0.18861 -0.78391 -0.14158 0.22169 -0.22078 -0.30509 -0.10837 0.57168 -0.7832 -0.16328 -0.76131 0.080873 0.00067217 0.44713 1.3434 -0.20014 -2.868 -0.002647 -0.39858 1.8379 1.2211 -0.16066 0.65853 0.26946 0.27212 0.94735 0.24372 0.8194 0.6774 0.063485 -0.55934 0.45541 -0.64684 -0.034702 -0.45566 0.21847 -0.051689 0.32299 -0.022961 -1.7955 0.31217 0.76227 -0.23191 -1.0133 -0.0064374 -1.8135 -0.75221 0.28362 -0.30815 -0.43853 -0.62654 0.13213 -0.54725 -0.47478 -0.0079727 -0.15112 -0.29326 -0.35118 -0.68175 0.28804 0.54893
+on -0.21863 -0.42664 0.5196 0.0043103 0.58045 -0.10873 -0.37726 0.4566 -0.60627 -0.075773 0.11306 0.17703 0.1605 0.074514 0.63649 -0.078852 0.75268 -0.24962 -0.51628 -0.33348 0.66754 -0.34183 0.61316 0.31668 0.64846 -0.079312 -0.065219 -0.17718 -0.32439 0.51868 -0.23424 0.34381 0.046851 0.74025 -0.47005 0.53685 -0.35549 0.40737 -0.093421 -0.13439 -0.41969 -0.30041 0.28646 0.37419 -0.46054 -0.307 -0.3858 -0.69317 -0.00092461 -0.61984 0.11978 0.1495 0.17833 1.5313 -0.92445 -3.0428 0.030761 -0.64359 2.3824 0.56219 -0.56021 1.0264 -0.45143 0.14117 0.65944 0.37572 0.098334 0.38304 -0.076882 -0.21781 -0.29892 -0.49458 0.095239 -0.63059 -0.061311 0.17767 -0.14051 0.47182 -0.95891 0.045334 0.808 -0.026867 -0.27483 0.35541 -0.82896 -0.78838 -0.079732 0.22941 -0.45013 -0.3004 -0.52716 0.11358 -0.49906 0.827 -0.56991 0.25143 -0.40266 -0.29146 1.3816 0.18084
+is -0.54264 0.41476 1.0322 -0.40244 0.46691 0.21816 -0.074864 0.47332 0.080996 -0.22079 -0.12808 -0.1144 0.50891 0.11568 0.028211 -0.3628 0.43823 0.047511 0.20282 0.49857 -0.10068 0.13269 0.16972 0.11653 0.31355 0.25713 0.092783 -0.56826 -0.52975 -0.051456 -0.67326 0.92533 0.2693 0.22734 0.66365 0.26221 0.19719 0.2609 0.18774 -0.3454 -0.42635 0.13975 0.56338 -0.56907 0.12398 -0.12894 0.72484 -0.26105 -0.26314 -0.43605 0.078908 -0.84146 0.51595 1.3997 -0.7646 -3.1453 -0.29202 -0.31247 1.5129 0.52435 0.21456 0.42452 -0.088411 -0.17805 1.1876 0.10579 0.76571 0.21914 0.35824 -0.11636 0.093261 -0.62483 -0.21898 0.21796 0.74056 -0.43735 0.14343 0.14719 -1.1605 -0.050508 0.12677 -0.014395 -0.98676 -0.091297 -1.2054 -0.11974 0.047847 -0.54001 0.52457 -0.70963 -0.32528 -0.1346 -0.41314 0.33435 -0.0072412 0.32253 -0.044219 -1.2969 0.76217 0.46349
+was 0.13717 -0.54287 0.19419 -0.29953 0.17545 0.084672 0.67752 0.098295 -0.035611 0.21334 0.51663 0.20687 0.44082 -0.33655 0.56025 -0.6879 0.51957 -0.21258 -0.52708 -0.12249 0.33099 0.026448 0.59007 0.0065469 0.45405 -0.33884 -0.28261 -0.24633 0.10847 0.3164 -0.15368 0.73503 0.11858 0.70842 0.075081 0.29738 -0.11395 0.40807 -0.042531 -0.21301 -0.79849 -0.12703 0.752 -0.41746 0.46615 -0.039097 0.65961 -0.32336 0.442 -0.94137 -0.23125 -0.30604 0.79912 1.4581 -0.88199 -3.0041 -0.75243 -0.20503 1.1998 0.94881 0.30649 0.48411 -0.7572 0.65856 0.70107 -0.93141 0.52928 0.23323 0.18857 0.38691 0.011489 -0.31937 0.011858 0.22944 0.17764 0.16868 0.14003 0.58647 -1.5447 -0.064425 -0.00064711 0.13606 -0.32695 0.10043 -1.546 -0.5476 0.21027 -0.67195 -0.1597 -0.68271 -0.22043 -0.87088 -0.16248 0.83086 -0.23045 0.19864 -0.051892 -0.52057 0.25434 -0.23759
+said -0.13128 -0.452 0.043399 -0.99798 -0.21053 -0.95868 -0.24609 0.48413 0.18178 0.475 -0.22305 0.30064 0.43496 -0.3605 0.20245 -0.52594 -0.34708 0.0075873 -1.0497 0.18673 0.57369 0.43814 0.098659 0.3877 -0.2258 0.41911 0.043602 -0.7352 -0.53583 0.19276 -0.21961 0.42515 -0.19082 0.47187 0.18826 0.13357 0.41839 1.3138 0.35678 -0.32172 -1.2257 -0.26635 0.36716 -0.27586 -0.53246 0.16786 -0.11253 -0.99959 -0.60706 -0.89271 0.65156 -0.88784 0.049233 0.67111 -0.27553 -2.4005 -0.36989 0.29136 1.3498 1.7353 0.27 0.021299 0.14422 0.023784 0.33643 -0.35476 1.0921 1.4845 0.4943 0.15688 0.34679 -0.57221 0.12093 -1.2616 1.0541 0.064335 -0.002732 0.19038 -1.7643 0.055068 1.4737 -0.41782 -0.57342 -0.12129 -1.3169 -0.73883 0.17682 -0.019991 -0.49176 -0.55247 1.0623 -0.62879 0.29098 0.13238 -0.70414 0.67128 -0.085462 -0.30526 -0.045495 0.56509
+with -0.43608 0.39104 0.51657 -0.13861 0.2029 0.50723 -0.012544 0.22948 -0.6316 0.21199 -0.018043 -0.39364 0.74164 0.30221 0.51792 -0.25191 0.25373 -0.65184 -0.42963 0.0093622 0.023334 -0.39245 0.34948 0.21217 0.7346 -0.21962 -0.028611 -0.34641 -0.20934 -0.27091 -0.17637 0.82396 -0.082339 -0.034869 0.079722 0.34841 0.60887 0.22811 -0.29633 0.18633 0.234 -0.70966 0.16312 -0.20857 0.092369 -0.075435 -0.13905 -0.35121 -0.19972 -0.41687 -0.31485 0.16123 0.038882 1.6654 -0.12401 -3.3419 0.10929 -0.026199 1.244 0.84374 -0.15679 0.79041 -0.042433 0.18884 0.064345 -0.11683 1.0467 0.71813 0.57834 0.27014 -0.50908 -0.083995 -0.1437 -0.76408 0.27418 0.56814 -0.39375 -0.32558 -0.92854 -0.13098 1.3277 0.11851 -0.15551 0.5972 -1.084 -0.058137 0.23886 0.14558 -0.59303 -0.47511 -0.22064 -0.37591 -0.79649 0.013465 -0.44595 -0.34623 -0.75398 -0.3517 0.99456 0.088196
+he 0.1225 -0.058833 0.23658 -0.28877 -0.028181 0.31524 0.070229 0.16447 -0.027623 0.25214 0.21174 -0.059674 0.36133 0.13607 0.18755 -0.1487 0.31315 0.13368 -0.59703 -0.030161 0.080656 0.26162 -0.055924 -0.35351 0.34722 -0.0055801 -0.57935 -0.88007 0.42931 -0.15695 -0.51256 1.2684 -0.25228 0.35265 -0.46419 0.55648 -0.57556 0.32574 -0.21893 -0.13178 -1.1027 -0.039591 0.89643 -0.9845 -0.47393 -0.12855 0.63506 -0.94888 0.40088 -0.77542 -0.35153 -0.27788 0.68747 1.458 -0.38474 -2.8937 -0.29523 -0.38836 0.94881 1.3891 0.054591 0.70486 -0.65699 0.075648 0.7655 -0.63365 0.86556 0.42441 0.14796 0.4156 0.29354 -0.51295 0.19635 -0.45568 0.0080246 0.14528 -0.15395 0.11406 -1.2167 -0.1111 0.8264 0.21738 -0.63776 -0.074874 -1.713 -0.8827 -0.0073058 -0.37623 -0.50209 -0.58844 -0.24943 -1.0425 0.27678 0.64142 -0.64605 0.43559 -0.37276 -0.0032068 0.18744 0.30702
+as -0.32721 0.096446 0.34244 -0.44327 0.30535 -0.042016 -0.071235 -0.31036 -0.22557 -0.181 -0.29088 -0.61542 0.29751 0.030491 0.41504 -0.51489 0.68628 -0.020302 -0.18486 0.31605 0.59472 -0.2147 0.29256 0.43262 0.35466 -0.29659 -0.27086 -0.48953 -0.047391 0.24521 -0.15783 0.59742 -0.41664 0.057632 0.1233 0.62326 -0.08844 0.3077 -0.15742 -0.28381 -0.58058 -0.022824 0.26689 -0.22565 0.47548 0.11134 0.37263 -0.14554 -0.16775 -0.79377 -0.30593 -0.10671 0.44199 1.5698 -0.73062 -2.7314 -0.19366 -0.32983 1.2881 0.62126 -0.255 0.8416 -0.23658 0.42594 0.86589 -0.35904 0.78162 0.20396 0.82898 0.0016123 -0.24008 -0.72735 -0.053671 -0.22264 0.31034 -0.21243 -0.14335 0.317 -0.80478 -0.49311 0.88023 -0.24147 -0.3922 0.15997 -1.5854 -0.25824 0.052834 -0.11983 -0.018874 -0.77356 0.049285 -0.25332 -0.3073 0.51295 -0.56802 -0.21239 -0.39741 -0.38165 0.43994 0.24683