{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "en_st_train_from_drive.ipynb", "provenance": [], "collapsed_sections": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "ytDi5g6HsdSI", "colab_type": "text" }, "source": [ "# Train JoeyNMT from Google Drive\n", "\n", "Run this notebook to train a JoeyNMT model from train and test data that has already been stored in a google drive folder. This is to allow a model to run after lengthy data preprocessing (such as fuzzy wuzzy) has run. " ] }, { "cell_type": "markdown", "metadata": { "id": "EnE0-hcpvIvA", "colab_type": "text" }, "source": [ "## Initial Configuration" ] }, { "cell_type": "code", "metadata": { "id": "4pS_cpuksZMn", "colab_type": "code", "outputId": "96fff02b-fa39-45ee-9617-2b8a28d08153", "colab": { "base_uri": "https://localhost:8080/", "height": 130 } }, "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n", "\n", "Enter your authorization code:\n", "··········\n", "Mounted at /content/drive\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "fJXxZEoOtmOY", "colab_type": "code", "colab": {} }, "source": [ "import os\n", "\n", "# EDIT THESE:\n", "source_language = \"en\"\n", "target_language = \"st\"\n", "tag = \"baseline\"\n", "\n", "os.environ[\"src\"] = source_language\n", "os.environ[\"tgt\"] = target_language\n", "os.environ[\"tag\"] = tag\n", "\n", "# assumes the gdrive path was created by the masakhane starter notebook\n", "os.environ[\"gdrive_path\"] = \"/content/drive/My Drive/masakhane/%s-%s-%s\" % (source_language, target_language, tag)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "X-2SUGUAuczs", "colab_type": "code", "outputId": "2586aec6-3154-4244-801e-790eaf66a46c", "colab": { "base_uri": "https://localhost:8080/", "height": 36 } }, "source": [ "!echo $gdrive_path" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "/content/drive/My Drive/masakhane/en-st-baseline\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "8Q9NSfhIufdp", "colab_type": "text" }, "source": [ "## Loading Data\n", "Copy data from our drive to Colab." ] }, { "cell_type": "code", "metadata": { "id": "urUWVf2SvRjm", "colab_type": "code", "outputId": "f4a6352c-4cb6-437d-d7f4-dd23e368d1e5", "colab": { "base_uri": "https://localhost:8080/", "height": 36 } }, "source": [ "# copy train files\n", "! cp \"$gdrive_path/train.$src\" ./\n", "! cp \"$gdrive_path/train.$tgt\" ./\n", "\n", "# copy dev files\n", "! cp \"$gdrive_path/dev.$src\" ./\n", "! cp \"$gdrive_path/dev.$tgt\" ./\n", "\n", "# copy test files\n", "! cp \"$gdrive_path/test.$src\" ./\n", "! cp \"$gdrive_path/test.$tgt\" ./\n", "\n", "! ls" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "dev.en\tdev.st\tdrive sample_data test.en test.st train.en\ttrain.st\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "YjMraQF2FOne", "colab_type": "code", "outputId": "831625ee-e9c8-4d41-95fb-f9f3eaf8db6a", "colab": { "base_uri": "https://localhost:8080/", "height": 130 } }, "source": [ "! head -n 5 train.$src" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "Little did I realize that in future years I would spend the major portion of my life continuing the work of these pilgrims by serving as a traveling overseer of Jehovah’s Witnesses .\n", "These were things I never imagined I would be able to get rid of . ”\n", "Ruins of the theater at Ephesus\n", "We were arrested and taken to the police station .\n", "Today , a more far - reaching destruction is looming , one that will bring an end to this entire system of things .\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "nCxC-AvlyxYT", "colab_type": "text" }, "source": [ "## Install JoeyNMT" ] }, { "cell_type": "code", "metadata": { "id": "a4IlFdrNyzpw", "colab_type": "code", "outputId": "2a624407-c8bd-47da-84cc-7310c674de13", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Install JoeyNMT\n", "! git clone https://github.com/joeynmt/joeynmt.git\n", "! cd joeynmt; pip3 install ." ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "Cloning into 'joeynmt'...\n", "remote: Enumerating objects: 149, done.\u001b[K\n", "remote: Counting objects: 0% (1/149)\u001b[K\rremote: Counting objects: 1% (2/149)\u001b[K\rremote: Counting objects: 2% (3/149)\u001b[K\rremote: Counting objects: 3% (5/149)\u001b[K\rremote: Counting objects: 4% (6/149)\u001b[K\rremote: Counting objects: 5% (8/149)\u001b[K\rremote: Counting objects: 6% (9/149)\u001b[K\rremote: Counting objects: 7% (11/149)\u001b[K\rremote: Counting objects: 8% (12/149)\u001b[K\rremote: Counting objects: 9% (14/149)\u001b[K\rremote: Counting objects: 10% (15/149)\u001b[K\rremote: Counting objects: 11% (17/149)\u001b[K\rremote: Counting objects: 12% (18/149)\u001b[K\rremote: Counting objects: 13% (20/149)\u001b[K\rremote: Counting objects: 14% (21/149)\u001b[K\rremote: Counting objects: 15% (23/149)\u001b[K\rremote: Counting objects: 16% (24/149)\u001b[K\rremote: Counting objects: 17% (26/149)\u001b[K\rremote: Counting objects: 18% (27/149)\u001b[K\rremote: Counting objects: 19% (29/149)\u001b[K\rremote: Counting objects: 20% (30/149)\u001b[K\rremote: Counting objects: 21% (32/149)\u001b[K\rremote: Counting objects: 22% (33/149)\u001b[K\rremote: Counting objects: 23% (35/149)\u001b[K\rremote: Counting objects: 24% (36/149)\u001b[K\rremote: Counting objects: 25% (38/149)\u001b[K\rremote: Counting objects: 26% (39/149)\u001b[K\rremote: Counting objects: 27% (41/149)\u001b[K\rremote: Counting objects: 28% (42/149)\u001b[K\rremote: Counting objects: 29% (44/149)\u001b[K\rremote: Counting objects: 30% (45/149)\u001b[K\rremote: Counting objects: 31% (47/149)\u001b[K\rremote: Counting objects: 32% (48/149)\u001b[K\rremote: Counting objects: 33% (50/149)\u001b[K\rremote: Counting objects: 34% (51/149)\u001b[K\rremote: Counting objects: 35% (53/149)\u001b[K\rremote: Counting objects: 36% (54/149)\u001b[K\rremote: Counting objects: 37% (56/149)\u001b[K\rremote: Counting objects: 38% (57/149)\u001b[K\rremote: Counting objects: 39% (59/149)\u001b[K\rremote: Counting objects: 40% (60/149)\u001b[K\rremote: Counting objects: 41% (62/149)\u001b[K\rremote: Counting objects: 42% (63/149)\u001b[K\rremote: Counting objects: 43% (65/149)\u001b[K\rremote: Counting objects: 44% (66/149)\u001b[K\rremote: Counting objects: 45% (68/149)\u001b[K\rremote: Counting objects: 46% (69/149)\u001b[K\rremote: Counting objects: 47% (71/149)\u001b[K\rremote: Counting objects: 48% (72/149)\u001b[K\rremote: Counting objects: 49% (74/149)\u001b[K\rremote: Counting objects: 50% (75/149)\u001b[K\rremote: Counting objects: 51% (76/149)\u001b[K\rremote: Counting objects: 52% (78/149)\u001b[K\rremote: Counting objects: 53% (79/149)\u001b[K\rremote: Counting objects: 54% (81/149)\u001b[K\rremote: Counting objects: 55% (82/149)\u001b[K\rremote: Counting objects: 56% (84/149)\u001b[K\rremote: Counting objects: 57% (85/149)\u001b[K\rremote: Counting objects: 58% (87/149)\u001b[K\rremote: Counting objects: 59% (88/149)\u001b[K\rremote: Counting objects: 60% (90/149)\u001b[K\rremote: Counting objects: 61% (91/149)\u001b[K\rremote: Counting objects: 62% (93/149)\u001b[K\rremote: Counting objects: 63% (94/149)\u001b[K\rremote: Counting objects: 64% (96/149)\u001b[K\rremote: Counting objects: 65% (97/149)\u001b[K\rremote: Counting objects: 66% (99/149)\u001b[K\rremote: Counting objects: 67% (100/149)\rremote: Counting objects: 68% (102/149)\u001b[K\rremote: Counting objects: 69% (103/149)\u001b[K\rremote: Counting objects: 70% (105/149)\u001b[K\rremote: Counting objects: 71% (106/149)\u001b[K\rremote: Counting objects: 72% (108/149)\u001b[K\rremote: Counting objects: 73% (109/149)\u001b[K\rremote: Counting objects: 74% (111/149)\u001b[K\rremote: Counting objects: 75% (112/149)\u001b[K\rremote: Counting objects: 76% (114/149)\u001b[K\rremote: Counting objects: 77% (115/149)\u001b[K\rremote: Counting objects: 78% (117/149)\u001b[K\rremote: Counting objects: 79% (118/149)\u001b[K\rremote: Counting objects: 80% (120/149)\u001b[K\rremote: Counting objects: 81% (121/149)\u001b[K\rremote: Counting objects: 82% (123/149)\u001b[K\rremote: Counting objects: 83% (124/149)\u001b[K\rremote: Counting objects: 84% (126/149)\u001b[K\rremote: Counting objects: 85% (127/149)\u001b[K\rremote: Counting objects: 86% (129/149)\u001b[K\rremote: Counting objects: 87% (130/149)\u001b[K\rremote: Counting objects: 88% (132/149)\u001b[K\rremote: Counting objects: 89% (133/149)\u001b[K\rremote: Counting objects: 90% (135/149)\u001b[K\rremote: Counting objects: 91% (136/149)\u001b[K\rremote: Counting objects: 92% (138/149)\u001b[K\rremote: Counting objects: 93% (139/149)\u001b[K\rremote: Counting objects: 94% (141/149)\u001b[K\rremote: Counting objects: 95% (142/149)\u001b[K\rremote: Counting objects: 96% (144/149)\u001b[K\rremote: Counting objects: 97% (145/149)\u001b[K\rremote: Counting objects: 98% (147/149)\u001b[K\rremote: Counting objects: 99% (148/149)\u001b[K\rremote: Counting objects: 100% (149/149)\u001b[K\rremote: Counting objects: 100% (149/149), done.\u001b[K\n", "remote: Compressing objects: 100% (105/105), done.\u001b[K\n", "remote: Total 2333 (delta 98), reused 72 (delta 44), pack-reused 2184\u001b[K\n", "Receiving objects: 100% (2333/2333), 2.64 MiB | 5.01 MiB/s, done.\n", "Resolving deltas: 100% (1619/1619), done.\n", "Processing /content/joeynmt\n", "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.16.0)\n", "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (6.2.2)\n", "Requirement already satisfied: numpy<2.0,>=1.14.5 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.17.5)\n", "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (45.1.0)\n", "Requirement already satisfied: torch>=1.1 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.4.0)\n", "Requirement already satisfied: tensorflow>=1.14 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.15.0)\n", "Requirement already satisfied: torchtext in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.3.1)\n", "Collecting sacrebleu>=1.3.6\n", " Downloading https://files.pythonhosted.org/packages/45/31/1a135b964c169984b27fb2f7a50280fa7f8e6d9d404d8a9e596180487fd1/sacrebleu-1.4.3-py3-none-any.whl\n", "Collecting subword-nmt\n", " Downloading https://files.pythonhosted.org/packages/74/60/6600a7bc09e7ab38bc53a48a20d8cae49b837f93f5842a41fe513a694912/subword_nmt-0.3.7-py2.py3-none-any.whl\n", "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (3.1.3)\n", "Requirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (0.10.0)\n", "Collecting pyyaml>=5.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/3d/d9/ea9816aea31beeadccd03f1f8b625ecf8f645bd66744484d162d84803ce5/PyYAML-5.3.tar.gz (268kB)\n", "\u001b[K |████████████████████████████████| 276kB 27.1MB/s \n", "\u001b[?25hCollecting pylint\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/e9/59/43fc36c5ee316bb9aeb7cf5329cdbdca89e5749c34d5602753827c0aa2dc/pylint-2.4.4-py3-none-any.whl (302kB)\n", "\u001b[K |████████████████████████████████| 307kB 55.6MB/s \n", "\u001b[?25hRequirement already satisfied: six==1.12 in /usr/local/lib/python3.6/dist-packages (from joeynmt==0.0.1) (1.12.0)\n", "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.0.8)\n", "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.27.1)\n", "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.1.0)\n", "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: tensorflow-estimator==1.15.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.1)\n", "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.11.2)\n", "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.34.2)\n", "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.2.2)\n", "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.9.0)\n", "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.8.1)\n", "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (0.1.8)\n", "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (1.15.0)\n", "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow>=1.14->joeynmt==0.0.1) (3.10.0)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (4.28.1)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from torchtext->joeynmt==0.0.1) (2.21.0)\n", "Collecting portalocker\n", " Downloading https://files.pythonhosted.org/packages/91/db/7bc703c0760df726839e0699b7f78a4d8217fdc9c7fcb1b51b39c5a22a4e/portalocker-1.5.2-py2.py3-none-any.whl\n", "Requirement already satisfied: typing in /usr/local/lib/python3.6/dist-packages (from sacrebleu>=1.3.6->joeynmt==0.0.1) (3.6.6)\n", "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.4.6)\n", "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (0.10.0)\n", "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (2.6.1)\n", "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->joeynmt==0.0.1) (1.1.0)\n", "Requirement already satisfied: scipy>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (1.4.1)\n", "Requirement already satisfied: pandas>=0.22.0 in /usr/local/lib/python3.6/dist-packages (from seaborn->joeynmt==0.0.1) (0.25.3)\n", "Collecting mccabe<0.7,>=0.6\n", " Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl\n", "Collecting isort<5,>=4.2.5\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/e5/b0/c121fd1fa3419ea9bfd55c7f9c4fedfec5143208d8c7ad3ce3db6c623c21/isort-4.3.21-py2.py3-none-any.whl (42kB)\n", "\u001b[K |████████████████████████████████| 51kB 8.9MB/s \n", "\u001b[?25hCollecting astroid<2.4,>=2.3.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/ad/ae/86734823047962e7b8c8529186a1ac4a7ca19aaf1aa0c7713c022ef593fd/astroid-2.3.3-py3-none-any.whl (205kB)\n", "\u001b[K |████████████████████████████████| 215kB 57.9MB/s \n", "\u001b[?25hRequirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow>=1.14->joeynmt==0.0.1) (2.8.0)\n", "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (3.2.1)\n", "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow>=1.14->joeynmt==0.0.1) (1.0.0)\n", "Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (1.24.3)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2019.11.28)\n", "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (3.0.4)\n", "Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->torchtext->joeynmt==0.0.1) (2.8)\n", "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.22.0->seaborn->joeynmt==0.0.1) (2018.9)\n", "Collecting typed-ast<1.5,>=1.4.0; implementation_name == \"cpython\" and python_version < \"3.8\"\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/90/ed/5459080d95eb87a02fe860d447197be63b6e2b5e9ff73c2b0a85622994f4/typed_ast-1.4.1-cp36-cp36m-manylinux1_x86_64.whl (737kB)\n", "\u001b[K |████████████████████████████████| 747kB 52.3MB/s \n", "\u001b[?25hCollecting lazy-object-proxy==1.4.*\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/0b/dd/b1e3407e9e6913cf178e506cd0dee818e58694d9a5cd1984e3f6a8b9a10f/lazy_object_proxy-1.4.3-cp36-cp36m-manylinux1_x86_64.whl (55kB)\n", "\u001b[K |████████████████████████████████| 61kB 9.4MB/s \n", "\u001b[?25hBuilding wheels for collected packages: joeynmt, pyyaml\n", " Building wheel for joeynmt (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for joeynmt: filename=joeynmt-0.0.1-cp36-none-any.whl size=73467 sha256=e22678653c33e59e45afd63d631a2a714bc1d2156c7ceb28663c1a93b50710f5\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-zfq_kovv/wheels/db/01/db/751cc9f3e7f6faec127c43644ba250a3ea7ad200594aeda70a\n", " Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for pyyaml: filename=PyYAML-5.3-cp36-cp36m-linux_x86_64.whl size=44229 sha256=6797d7798bc7d9858caad5e6676b7e753f41c1b9ddf7c61aa23ac6c286fdcd72\n", " Stored in directory: /root/.cache/pip/wheels/e4/76/4d/a95b8dd7b452b69e8ed4f68b69e1b55e12c9c9624dd962b191\n", "Successfully built joeynmt pyyaml\n", "Installing collected packages: portalocker, sacrebleu, subword-nmt, pyyaml, mccabe, isort, typed-ast, lazy-object-proxy, astroid, pylint, joeynmt\n", " Found existing installation: PyYAML 3.13\n", " Uninstalling PyYAML-3.13:\n", " Successfully uninstalled PyYAML-3.13\n", "Successfully installed astroid-2.3.3 isort-4.3.21 joeynmt-0.0.1 lazy-object-proxy-1.4.3 mccabe-0.6.1 portalocker-1.5.2 pylint-2.4.4 pyyaml-5.3 sacrebleu-1.4.3 subword-nmt-0.3.7 typed-ast-1.4.1\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "65gKd0oyw82d", "colab_type": "text" }, "source": [ "# Preprocessing the Data into Subword BPE Tokens\n", "\n", "- One of the most powerful improvements for agglutinative languages (a feature of most Bantu languages) is using BPE tokenization [ (Sennrich, 2015) ](https://arxiv.org/abs/1508.07909).\n", "\n", "- It was also shown that by optimizing the umber of BPE codes we significantly improve results for low-resourced languages [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021) [(Martinus, 2019)](https://arxiv.org/abs/1906.05685)\n", "\n", "- Below we have the scripts for doing BPE tokenization of our data. We use 4000 tokens as recommended by [(Sennrich, 2019)](https://www.aclweb.org/anthology/P19-1021). You do not need to change anything. Simply running the below will be suitable. " ] }, { "cell_type": "code", "metadata": { "id": "p9yaObAyxTA-", "colab_type": "code", "outputId": "eac7f8e1-42c8-4605-96ae-2c2c1cd1e510", "colab": { "base_uri": "https://localhost:8080/", "height": 463 } }, "source": [ "# One of the huge boosts in NMT performance was to use a different method of tokenizing. \n", "# Usually, NMT would tokenize by words. However, using a method called BPE gave amazing boosts to performance\n", "\n", "# Do subword NMT\n", "from os import path\n", "\n", "# set number of bpe codes to use\n", "nb_codes = 40000\n", "os.environ[\"codes\"] = str(nb_codes)\n", "\n", "# Learn BPEs on the training data.\n", "os.environ[\"data_path\"] = path.join(\"joeynmt\", \"data\", source_language + target_language) # Herman! \n", "! subword-nmt learn-joint-bpe-and-vocab --input train.$src train.$tgt -s $codes -o bpe.codes.$codes --write-vocabulary vocab.$src vocab.$tgt\n", "\n", "# Apply BPE splits to the development and test data.\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$src < train.$src > train.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < train.$tgt > train.bpe.$tgt\n", "\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$src < dev.$src > dev.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < dev.$tgt > dev.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$src < test.$src > test.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt\n", "\n", "# Create directory, move everyone we care about to the correct location\n", "! mkdir -p $data_path\n", "! cp train.* $data_path\n", "! cp test.* $data_path\n", "! cp dev.* $data_path\n", "! cp bpe.codes.$codes $data_path\n", "! ls $data_path\n", "\n", "# Also move the bpe stuff back to the mounted location in google drive (relevant if running in colab) at gdrive_path\n", "! cp dev.* \"$gdrive_path\"\n", "! cp bpe.codes.$codes \"$gdrive_path\"\n", "! ls \"$gdrive_path\"\n", "\n", "# Create that vocab using build_vocab\n", "! sudo chmod 777 joeynmt/scripts/build_vocab.py\n", "! joeynmt/scripts/build_vocab.py joeynmt/data/$src$tgt/train.bpe.$src joeynmt/data/$src$tgt/train.bpe.$tgt --output_path joeynmt/data/$src$tgt/vocab.txt\n", "\n", "# Some output\n", "! echo \"BPE $tgt Sentences\"\n", "! tail -n 5 test.bpe.$tgt\n", "! echo \"Combined BPE Vocab\"\n", "! tail -n 10 joeynmt/data/$src$tgt/vocab.txt # Herman" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "bpe.codes.40000 dev.en test.bpe.st train.bpe.en train.st\n", "dev.bpe.en\t dev.st test.en\t train.bpe.st\n", "dev.bpe.st\t test.bpe.en test.st\t train.en\n", "bpe.codes.4000\t dev.en test.bpe.en\t test.st train.st\n", "bpe.codes.40000 dev.st test.bpe.st\t train.bpe.en\n", "dev.bpe.en\t dev.xh test.en\t train.bpe.st\n", "dev.bpe.st\t models test.en-any.en train.en\n", "BPE st Sentences\n", "Ka lebaka leo , ke ile ka tloaela ho se tšepahale .\n", "Ka mor’a hore ke ithute Bibele , ke ile ka tlohela mosebetsi oo , le hoja ke ne ke pata@@ loa hantle .\n", "Ke behetse bara ba ka ba babeli mohlala o motle , ’ me ke khona le ho sebeletsa ka phuthehong .\n", "Ho basebeletsi ba lekhetho le ho batho ba bang bao ke sebetsang le bona , ke tsebahala ke le motho ea tšepahalang . ”\n", "Ruthe o ile a fallela Iseraele moo a neng a tla rapela Molimo oa ’ nete .\n", "Combined BPE Vocab\n", "vern@@\n", "fatsing\n", "zim\n", "itamins\n", "claim@@\n", "leoat@@\n", "Jerusale@@\n", "them@@\n", "elings\n", "parishi@@\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "lHAxUS6B0puo", "colab_type": "code", "colab": {} }, "source": [ "# This creates the config file for our JoeyNMT system. It might seem overwhelming so we've provided a couple of useful parameters you'll need to update\n", "# (You can of course play with all the parameters if you'd like!)\n", "\n", "name = '%s%s' % (source_language, target_language)\n", "gdrive_path = os.environ[\"gdrive_path\"]\n", "\n", "# Create the config\n", "config = \"\"\"\n", "name: \"{name}_transformer\"\n", "\n", "data:\n", " src: \"{source_language}\"\n", " trg: \"{target_language}\"\n", " train: \"data/{name}/train.bpe\"\n", " dev: \"data/{name}/dev.bpe\"\n", " test: \"data/{name}/test.bpe\"\n", " level: \"bpe\"\n", " lowercase: False\n", " max_sent_length: 100\n", " src_vocab: \"data/{name}/vocab.txt\"\n", " trg_vocab: \"data/{name}/vocab.txt\"\n", "\n", "testing:\n", " beam_size: 5\n", " alpha: 1.0\n", "\n", "training:\n", " #load_model: \"{gdrive_path}/models/{name}_transformer/1.ckpt\" # if uncommented, load a pre-trained model from this checkpoint\n", " random_seed: 42\n", " optimizer: \"adam\"\n", " normalization: \"tokens\"\n", " adam_betas: [0.9, 0.999] \n", " scheduling: \"plateau\" # TODO: try switching from plateau to Noam scheduling\n", " patience: 5 # For plateau: decrease learning rate by decrease_factor if validation score has not improved for this many validation rounds.\n", " learning_rate_factor: 0.5 # factor for Noam scheduler (used with Transformer)\n", " learning_rate_warmup: 1000 # warmup steps for Noam scheduler (used with Transformer)\n", " decrease_factor: 0.7\n", " loss: \"crossentropy\"\n", " learning_rate: 0.0003\n", " learning_rate_min: 0.00000001\n", " weight_decay: 0.0\n", " label_smoothing: 0.1\n", " batch_size: 4096\n", " batch_type: \"token\"\n", " eval_batch_size: 3600\n", " eval_batch_type: \"token\"\n", " batch_multiplier: 1\n", " early_stopping_metric: \"ppl\"\n", " epochs: 14 # TODO: Decrease for when playing around and checking of working. Around 30 is sufficient to check if its working at all\n", " validation_freq: 1000 # TODO: Set to at least once per epoch.\n", " logging_freq: 100\n", " eval_metric: \"bleu\"\n", " model_dir: \"models/{name}_transformer\"\n", " overwrite: False # TODO: Set to True if you want to overwrite possibly existing models. \n", " shuffle: True\n", " use_cuda: True\n", " max_output_length: 100\n", " print_valid_sents: [0, 1, 2, 3]\n", " keep_last_ckpts: 3\n", "\n", "model:\n", " initializer: \"xavier\"\n", " bias_initializer: \"zeros\"\n", " init_gain: 1.0\n", " embed_initializer: \"xavier\"\n", " embed_init_gain: 1.0\n", " tied_embeddings: True\n", " tied_softmax: True\n", " encoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", " decoder:\n", " type: \"transformer\"\n", " num_layers: 6\n", " num_heads: 4 # TODO: Increase to 8 for larger data.\n", " embeddings:\n", " embedding_dim: 256 # TODO: Increase to 512 for larger data.\n", " scale: True\n", " dropout: 0.2\n", " # typically ff_size = 4 x hidden_size\n", " hidden_size: 256 # TODO: Increase to 512 for larger data.\n", " ff_size: 1024 # TODO: Increase to 2048 for larger data.\n", " dropout: 0.3\n", "\"\"\".format(name=name, gdrive_path=os.environ[\"gdrive_path\"], source_language=source_language, target_language=target_language)\n", "with open(\"joeynmt/configs/transformer_{name}.yaml\".format(name=name),'w') as f:\n", " f.write(config)" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "5kPvBiAa2HOD", "colab_type": "code", "outputId": "8b86ceea-0c15-4890-e90c-74f8e0d9f87c", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Train the model\n", "# You can press Ctrl-C to stop. And then run the next cell to save your checkpoints! \n", "!cd joeynmt; python3 -m joeynmt train configs/transformer_$src$tgt.yaml" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "2020-02-13 12:28:37,724 Hello! This is Joey-NMT.\n", "2020-02-13 12:28:38,902 Total params: 21245696\n", "2020-02-13 12:28:38,904 Trainable parameters: ['decoder.layer_norm.bias', 'decoder.layer_norm.weight', 'decoder.layers.0.dec_layer_norm.bias', 'decoder.layers.0.dec_layer_norm.weight', 'decoder.layers.0.feed_forward.layer_norm.bias', 'decoder.layers.0.feed_forward.layer_norm.weight', 'decoder.layers.0.feed_forward.pwff_layer.0.bias', 'decoder.layers.0.feed_forward.pwff_layer.0.weight', 'decoder.layers.0.feed_forward.pwff_layer.3.bias', 'decoder.layers.0.feed_forward.pwff_layer.3.weight', 'decoder.layers.0.src_trg_att.k_layer.bias', 'decoder.layers.0.src_trg_att.k_layer.weight', 'decoder.layers.0.src_trg_att.output_layer.bias', 'decoder.layers.0.src_trg_att.output_layer.weight', 'decoder.layers.0.src_trg_att.q_layer.bias', 'decoder.layers.0.src_trg_att.q_layer.weight', 'decoder.layers.0.src_trg_att.v_layer.bias', 'decoder.layers.0.src_trg_att.v_layer.weight', 'decoder.layers.0.trg_trg_att.k_layer.bias', 'decoder.layers.0.trg_trg_att.k_layer.weight', 'decoder.layers.0.trg_trg_att.output_layer.bias', 'decoder.layers.0.trg_trg_att.output_layer.weight', 'decoder.layers.0.trg_trg_att.q_layer.bias', 'decoder.layers.0.trg_trg_att.q_layer.weight', 'decoder.layers.0.trg_trg_att.v_layer.bias', 'decoder.layers.0.trg_trg_att.v_layer.weight', 'decoder.layers.0.x_layer_norm.bias', 'decoder.layers.0.x_layer_norm.weight', 'decoder.layers.1.dec_layer_norm.bias', 'decoder.layers.1.dec_layer_norm.weight', 'decoder.layers.1.feed_forward.layer_norm.bias', 'decoder.layers.1.feed_forward.layer_norm.weight', 'decoder.layers.1.feed_forward.pwff_layer.0.bias', 'decoder.layers.1.feed_forward.pwff_layer.0.weight', 'decoder.layers.1.feed_forward.pwff_layer.3.bias', 'decoder.layers.1.feed_forward.pwff_layer.3.weight', 'decoder.layers.1.src_trg_att.k_layer.bias', 'decoder.layers.1.src_trg_att.k_layer.weight', 'decoder.layers.1.src_trg_att.output_layer.bias', 'decoder.layers.1.src_trg_att.output_layer.weight', 'decoder.layers.1.src_trg_att.q_layer.bias', 'decoder.layers.1.src_trg_att.q_layer.weight', 'decoder.layers.1.src_trg_att.v_layer.bias', 'decoder.layers.1.src_trg_att.v_layer.weight', 'decoder.layers.1.trg_trg_att.k_layer.bias', 'decoder.layers.1.trg_trg_att.k_layer.weight', 'decoder.layers.1.trg_trg_att.output_layer.bias', 'decoder.layers.1.trg_trg_att.output_layer.weight', 'decoder.layers.1.trg_trg_att.q_layer.bias', 'decoder.layers.1.trg_trg_att.q_layer.weight', 'decoder.layers.1.trg_trg_att.v_layer.bias', 'decoder.layers.1.trg_trg_att.v_layer.weight', 'decoder.layers.1.x_layer_norm.bias', 'decoder.layers.1.x_layer_norm.weight', 'decoder.layers.2.dec_layer_norm.bias', 'decoder.layers.2.dec_layer_norm.weight', 'decoder.layers.2.feed_forward.layer_norm.bias', 'decoder.layers.2.feed_forward.layer_norm.weight', 'decoder.layers.2.feed_forward.pwff_layer.0.bias', 'decoder.layers.2.feed_forward.pwff_layer.0.weight', 'decoder.layers.2.feed_forward.pwff_layer.3.bias', 'decoder.layers.2.feed_forward.pwff_layer.3.weight', 'decoder.layers.2.src_trg_att.k_layer.bias', 'decoder.layers.2.src_trg_att.k_layer.weight', 'decoder.layers.2.src_trg_att.output_layer.bias', 'decoder.layers.2.src_trg_att.output_layer.weight', 'decoder.layers.2.src_trg_att.q_layer.bias', 'decoder.layers.2.src_trg_att.q_layer.weight', 'decoder.layers.2.src_trg_att.v_layer.bias', 'decoder.layers.2.src_trg_att.v_layer.weight', 'decoder.layers.2.trg_trg_att.k_layer.bias', 'decoder.layers.2.trg_trg_att.k_layer.weight', 'decoder.layers.2.trg_trg_att.output_layer.bias', 'decoder.layers.2.trg_trg_att.output_layer.weight', 'decoder.layers.2.trg_trg_att.q_layer.bias', 'decoder.layers.2.trg_trg_att.q_layer.weight', 'decoder.layers.2.trg_trg_att.v_layer.bias', 'decoder.layers.2.trg_trg_att.v_layer.weight', 'decoder.layers.2.x_layer_norm.bias', 'decoder.layers.2.x_layer_norm.weight', 'decoder.layers.3.dec_layer_norm.bias', 'decoder.layers.3.dec_layer_norm.weight', 'decoder.layers.3.feed_forward.layer_norm.bias', 'decoder.layers.3.feed_forward.layer_norm.weight', 'decoder.layers.3.feed_forward.pwff_layer.0.bias', 'decoder.layers.3.feed_forward.pwff_layer.0.weight', 'decoder.layers.3.feed_forward.pwff_layer.3.bias', 'decoder.layers.3.feed_forward.pwff_layer.3.weight', 'decoder.layers.3.src_trg_att.k_layer.bias', 'decoder.layers.3.src_trg_att.k_layer.weight', 'decoder.layers.3.src_trg_att.output_layer.bias', 'decoder.layers.3.src_trg_att.output_layer.weight', 'decoder.layers.3.src_trg_att.q_layer.bias', 'decoder.layers.3.src_trg_att.q_layer.weight', 'decoder.layers.3.src_trg_att.v_layer.bias', 'decoder.layers.3.src_trg_att.v_layer.weight', 'decoder.layers.3.trg_trg_att.k_layer.bias', 'decoder.layers.3.trg_trg_att.k_layer.weight', 'decoder.layers.3.trg_trg_att.output_layer.bias', 'decoder.layers.3.trg_trg_att.output_layer.weight', 'decoder.layers.3.trg_trg_att.q_layer.bias', 'decoder.layers.3.trg_trg_att.q_layer.weight', 'decoder.layers.3.trg_trg_att.v_layer.bias', 'decoder.layers.3.trg_trg_att.v_layer.weight', 'decoder.layers.3.x_layer_norm.bias', 'decoder.layers.3.x_layer_norm.weight', 'decoder.layers.4.dec_layer_norm.bias', 'decoder.layers.4.dec_layer_norm.weight', 'decoder.layers.4.feed_forward.layer_norm.bias', 'decoder.layers.4.feed_forward.layer_norm.weight', 'decoder.layers.4.feed_forward.pwff_layer.0.bias', 'decoder.layers.4.feed_forward.pwff_layer.0.weight', 'decoder.layers.4.feed_forward.pwff_layer.3.bias', 'decoder.layers.4.feed_forward.pwff_layer.3.weight', 'decoder.layers.4.src_trg_att.k_layer.bias', 'decoder.layers.4.src_trg_att.k_layer.weight', 'decoder.layers.4.src_trg_att.output_layer.bias', 'decoder.layers.4.src_trg_att.output_layer.weight', 'decoder.layers.4.src_trg_att.q_layer.bias', 'decoder.layers.4.src_trg_att.q_layer.weight', 'decoder.layers.4.src_trg_att.v_layer.bias', 'decoder.layers.4.src_trg_att.v_layer.weight', 'decoder.layers.4.trg_trg_att.k_layer.bias', 'decoder.layers.4.trg_trg_att.k_layer.weight', 'decoder.layers.4.trg_trg_att.output_layer.bias', 'decoder.layers.4.trg_trg_att.output_layer.weight', 'decoder.layers.4.trg_trg_att.q_layer.bias', 'decoder.layers.4.trg_trg_att.q_layer.weight', 'decoder.layers.4.trg_trg_att.v_layer.bias', 'decoder.layers.4.trg_trg_att.v_layer.weight', 'decoder.layers.4.x_layer_norm.bias', 'decoder.layers.4.x_layer_norm.weight', 'decoder.layers.5.dec_layer_norm.bias', 'decoder.layers.5.dec_layer_norm.weight', 'decoder.layers.5.feed_forward.layer_norm.bias', 'decoder.layers.5.feed_forward.layer_norm.weight', 'decoder.layers.5.feed_forward.pwff_layer.0.bias', 'decoder.layers.5.feed_forward.pwff_layer.0.weight', 'decoder.layers.5.feed_forward.pwff_layer.3.bias', 'decoder.layers.5.feed_forward.pwff_layer.3.weight', 'decoder.layers.5.src_trg_att.k_layer.bias', 'decoder.layers.5.src_trg_att.k_layer.weight', 'decoder.layers.5.src_trg_att.output_layer.bias', 'decoder.layers.5.src_trg_att.output_layer.weight', 'decoder.layers.5.src_trg_att.q_layer.bias', 'decoder.layers.5.src_trg_att.q_layer.weight', 'decoder.layers.5.src_trg_att.v_layer.bias', 'decoder.layers.5.src_trg_att.v_layer.weight', 'decoder.layers.5.trg_trg_att.k_layer.bias', 'decoder.layers.5.trg_trg_att.k_layer.weight', 'decoder.layers.5.trg_trg_att.output_layer.bias', 'decoder.layers.5.trg_trg_att.output_layer.weight', 'decoder.layers.5.trg_trg_att.q_layer.bias', 'decoder.layers.5.trg_trg_att.q_layer.weight', 'decoder.layers.5.trg_trg_att.v_layer.bias', 'decoder.layers.5.trg_trg_att.v_layer.weight', 'decoder.layers.5.x_layer_norm.bias', 'decoder.layers.5.x_layer_norm.weight', 'encoder.layer_norm.bias', 'encoder.layer_norm.weight', 'encoder.layers.0.feed_forward.layer_norm.bias', 'encoder.layers.0.feed_forward.layer_norm.weight', 'encoder.layers.0.feed_forward.pwff_layer.0.bias', 'encoder.layers.0.feed_forward.pwff_layer.0.weight', 'encoder.layers.0.feed_forward.pwff_layer.3.bias', 'encoder.layers.0.feed_forward.pwff_layer.3.weight', 'encoder.layers.0.layer_norm.bias', 'encoder.layers.0.layer_norm.weight', 'encoder.layers.0.src_src_att.k_layer.bias', 'encoder.layers.0.src_src_att.k_layer.weight', 'encoder.layers.0.src_src_att.output_layer.bias', 'encoder.layers.0.src_src_att.output_layer.weight', 'encoder.layers.0.src_src_att.q_layer.bias', 'encoder.layers.0.src_src_att.q_layer.weight', 'encoder.layers.0.src_src_att.v_layer.bias', 'encoder.layers.0.src_src_att.v_layer.weight', 'encoder.layers.1.feed_forward.layer_norm.bias', 'encoder.layers.1.feed_forward.layer_norm.weight', 'encoder.layers.1.feed_forward.pwff_layer.0.bias', 'encoder.layers.1.feed_forward.pwff_layer.0.weight', 'encoder.layers.1.feed_forward.pwff_layer.3.bias', 'encoder.layers.1.feed_forward.pwff_layer.3.weight', 'encoder.layers.1.layer_norm.bias', 'encoder.layers.1.layer_norm.weight', 'encoder.layers.1.src_src_att.k_layer.bias', 'encoder.layers.1.src_src_att.k_layer.weight', 'encoder.layers.1.src_src_att.output_layer.bias', 'encoder.layers.1.src_src_att.output_layer.weight', 'encoder.layers.1.src_src_att.q_layer.bias', 'encoder.layers.1.src_src_att.q_layer.weight', 'encoder.layers.1.src_src_att.v_layer.bias', 'encoder.layers.1.src_src_att.v_layer.weight', 'encoder.layers.2.feed_forward.layer_norm.bias', 'encoder.layers.2.feed_forward.layer_norm.weight', 'encoder.layers.2.feed_forward.pwff_layer.0.bias', 'encoder.layers.2.feed_forward.pwff_layer.0.weight', 'encoder.layers.2.feed_forward.pwff_layer.3.bias', 'encoder.layers.2.feed_forward.pwff_layer.3.weight', 'encoder.layers.2.layer_norm.bias', 'encoder.layers.2.layer_norm.weight', 'encoder.layers.2.src_src_att.k_layer.bias', 'encoder.layers.2.src_src_att.k_layer.weight', 'encoder.layers.2.src_src_att.output_layer.bias', 'encoder.layers.2.src_src_att.output_layer.weight', 'encoder.layers.2.src_src_att.q_layer.bias', 'encoder.layers.2.src_src_att.q_layer.weight', 'encoder.layers.2.src_src_att.v_layer.bias', 'encoder.layers.2.src_src_att.v_layer.weight', 'encoder.layers.3.feed_forward.layer_norm.bias', 'encoder.layers.3.feed_forward.layer_norm.weight', 'encoder.layers.3.feed_forward.pwff_layer.0.bias', 'encoder.layers.3.feed_forward.pwff_layer.0.weight', 'encoder.layers.3.feed_forward.pwff_layer.3.bias', 'encoder.layers.3.feed_forward.pwff_layer.3.weight', 'encoder.layers.3.layer_norm.bias', 'encoder.layers.3.layer_norm.weight', 'encoder.layers.3.src_src_att.k_layer.bias', 'encoder.layers.3.src_src_att.k_layer.weight', 'encoder.layers.3.src_src_att.output_layer.bias', 'encoder.layers.3.src_src_att.output_layer.weight', 'encoder.layers.3.src_src_att.q_layer.bias', 'encoder.layers.3.src_src_att.q_layer.weight', 'encoder.layers.3.src_src_att.v_layer.bias', 'encoder.layers.3.src_src_att.v_layer.weight', 'encoder.layers.4.feed_forward.layer_norm.bias', 'encoder.layers.4.feed_forward.layer_norm.weight', 'encoder.layers.4.feed_forward.pwff_layer.0.bias', 'encoder.layers.4.feed_forward.pwff_layer.0.weight', 'encoder.layers.4.feed_forward.pwff_layer.3.bias', 'encoder.layers.4.feed_forward.pwff_layer.3.weight', 'encoder.layers.4.layer_norm.bias', 'encoder.layers.4.layer_norm.weight', 'encoder.layers.4.src_src_att.k_layer.bias', 'encoder.layers.4.src_src_att.k_layer.weight', 'encoder.layers.4.src_src_att.output_layer.bias', 'encoder.layers.4.src_src_att.output_layer.weight', 'encoder.layers.4.src_src_att.q_layer.bias', 'encoder.layers.4.src_src_att.q_layer.weight', 'encoder.layers.4.src_src_att.v_layer.bias', 'encoder.layers.4.src_src_att.v_layer.weight', 'encoder.layers.5.feed_forward.layer_norm.bias', 'encoder.layers.5.feed_forward.layer_norm.weight', 'encoder.layers.5.feed_forward.pwff_layer.0.bias', 'encoder.layers.5.feed_forward.pwff_layer.0.weight', 'encoder.layers.5.feed_forward.pwff_layer.3.bias', 'encoder.layers.5.feed_forward.pwff_layer.3.weight', 'encoder.layers.5.layer_norm.bias', 'encoder.layers.5.layer_norm.weight', 'encoder.layers.5.src_src_att.k_layer.bias', 'encoder.layers.5.src_src_att.k_layer.weight', 'encoder.layers.5.src_src_att.output_layer.bias', 'encoder.layers.5.src_src_att.output_layer.weight', 'encoder.layers.5.src_src_att.q_layer.bias', 'encoder.layers.5.src_src_att.q_layer.weight', 'encoder.layers.5.src_src_att.v_layer.bias', 'encoder.layers.5.src_src_att.v_layer.weight', 'src_embed.lut.weight']\n", "2020-02-13 12:28:47,979 cfg.name : enst_transformer\n", "2020-02-13 12:28:47,979 cfg.data.src : en\n", "2020-02-13 12:28:47,980 cfg.data.trg : st\n", "2020-02-13 12:28:47,980 cfg.data.train : data/enst/train.bpe\n", "2020-02-13 12:28:47,980 cfg.data.dev : data/enst/dev.bpe\n", "2020-02-13 12:28:47,980 cfg.data.test : data/enst/test.bpe\n", "2020-02-13 12:28:47,980 cfg.data.level : bpe\n", "2020-02-13 12:28:47,980 cfg.data.lowercase : False\n", "2020-02-13 12:28:47,980 cfg.data.max_sent_length : 100\n", "2020-02-13 12:28:47,980 cfg.data.src_vocab : data/enst/vocab.txt\n", "2020-02-13 12:28:47,980 cfg.data.trg_vocab : data/enst/vocab.txt\n", "2020-02-13 12:28:47,980 cfg.testing.beam_size : 5\n", "2020-02-13 12:28:47,980 cfg.testing.alpha : 1.0\n", "2020-02-13 12:28:47,980 cfg.training.random_seed : 42\n", "2020-02-13 12:28:47,980 cfg.training.optimizer : adam\n", "2020-02-13 12:28:47,980 cfg.training.normalization : tokens\n", "2020-02-13 12:28:47,980 cfg.training.adam_betas : [0.9, 0.999]\n", "2020-02-13 12:28:47,981 cfg.training.scheduling : plateau\n", "2020-02-13 12:28:47,981 cfg.training.patience : 5\n", "2020-02-13 12:28:47,981 cfg.training.learning_rate_factor : 0.5\n", "2020-02-13 12:28:47,981 cfg.training.learning_rate_warmup : 1000\n", "2020-02-13 12:28:47,981 cfg.training.decrease_factor : 0.7\n", "2020-02-13 12:28:47,981 cfg.training.loss : crossentropy\n", "2020-02-13 12:28:47,981 cfg.training.learning_rate : 0.0003\n", "2020-02-13 12:28:47,981 cfg.training.learning_rate_min : 1e-08\n", "2020-02-13 12:28:47,981 cfg.training.weight_decay : 0.0\n", "2020-02-13 12:28:47,981 cfg.training.label_smoothing : 0.1\n", "2020-02-13 12:28:47,981 cfg.training.batch_size : 4096\n", "2020-02-13 12:28:47,981 cfg.training.batch_type : token\n", "2020-02-13 12:28:47,981 cfg.training.eval_batch_size : 3600\n", "2020-02-13 12:28:47,981 cfg.training.eval_batch_type : token\n", "2020-02-13 12:28:47,981 cfg.training.batch_multiplier : 1\n", "2020-02-13 12:28:47,981 cfg.training.early_stopping_metric : ppl\n", "2020-02-13 12:28:47,981 cfg.training.epochs : 14\n", "2020-02-13 12:28:47,982 cfg.training.validation_freq : 1000\n", "2020-02-13 12:28:47,982 cfg.training.logging_freq : 100\n", "2020-02-13 12:28:47,982 cfg.training.eval_metric : bleu\n", "2020-02-13 12:28:47,982 cfg.training.model_dir : models/enst_transformer\n", "2020-02-13 12:28:47,982 cfg.training.overwrite : False\n", "2020-02-13 12:28:47,982 cfg.training.shuffle : True\n", "2020-02-13 12:28:47,982 cfg.training.use_cuda : True\n", "2020-02-13 12:28:47,982 cfg.training.max_output_length : 100\n", "2020-02-13 12:28:47,982 cfg.training.print_valid_sents : [0, 1, 2, 3]\n", "2020-02-13 12:28:47,982 cfg.training.keep_last_ckpts : 3\n", "2020-02-13 12:28:47,982 cfg.model.initializer : xavier\n", "2020-02-13 12:28:47,982 cfg.model.bias_initializer : zeros\n", "2020-02-13 12:28:47,982 cfg.model.init_gain : 1.0\n", "2020-02-13 12:28:47,982 cfg.model.embed_initializer : xavier\n", "2020-02-13 12:28:47,982 cfg.model.embed_init_gain : 1.0\n", "2020-02-13 12:28:47,982 cfg.model.tied_embeddings : True\n", "2020-02-13 12:28:47,982 cfg.model.tied_softmax : True\n", "2020-02-13 12:28:47,983 cfg.model.encoder.type : transformer\n", "2020-02-13 12:28:47,983 cfg.model.encoder.num_layers : 6\n", "2020-02-13 12:28:47,983 cfg.model.encoder.num_heads : 4\n", "2020-02-13 12:28:47,983 cfg.model.encoder.embeddings.embedding_dim : 256\n", "2020-02-13 12:28:47,983 cfg.model.encoder.embeddings.scale : True\n", "2020-02-13 12:28:47,983 cfg.model.encoder.embeddings.dropout : 0.2\n", "2020-02-13 12:28:47,983 cfg.model.encoder.hidden_size : 256\n", "2020-02-13 12:28:47,983 cfg.model.encoder.ff_size : 1024\n", "2020-02-13 12:28:47,983 cfg.model.encoder.dropout : 0.3\n", "2020-02-13 12:28:47,983 cfg.model.decoder.type : transformer\n", "2020-02-13 12:28:47,983 cfg.model.decoder.num_layers : 6\n", "2020-02-13 12:28:47,983 cfg.model.decoder.num_heads : 4\n", "2020-02-13 12:28:47,983 cfg.model.decoder.embeddings.embedding_dim : 256\n", "2020-02-13 12:28:47,983 cfg.model.decoder.embeddings.scale : True\n", "2020-02-13 12:28:47,983 cfg.model.decoder.embeddings.dropout : 0.2\n", "2020-02-13 12:28:47,983 cfg.model.decoder.hidden_size : 256\n", "2020-02-13 12:28:47,983 cfg.model.decoder.ff_size : 1024\n", "2020-02-13 12:28:47,984 cfg.model.decoder.dropout : 0.3\n", "2020-02-13 12:28:47,984 Data set sizes: \n", "\ttrain 832811,\n", "\tvalid 1000,\n", "\ttest 2672\n", "2020-02-13 12:28:47,984 First training example:\n", "\t[SRC] Little did I realize that in future years I would spend the major portion of my life continuing the work of these pilgrims by serving as a traveling overseer of Jehovah’s Witnesses .\n", "\t[TRG] Ke ne ke sa hlokomele hore lilemong tse latelang ke ne ke tla sebelisa karolo e khōlō ea bophelo ba ka ke ntšetsa pele mosebetsi oa bana ba nkang maeto a bolumeli ka hore ke sebeletse ke le molebeli oa potoloho oa Lipaki tsa Jehova .\n", "2020-02-13 12:28:47,984 First 10 words (src): (0) (1) (2) (3) (4) . (5) , (6) a (7) ho (8) ba (9) le\n", "2020-02-13 12:28:47,984 First 10 words (trg): (0) (1) (2) (3) (4) . (5) , (6) a (7) ho (8) ba (9) le\n", "2020-02-13 12:28:47,984 Number of Src words (types): 39787\n", "2020-02-13 12:28:47,985 Number of Trg words (types): 39787\n", "2020-02-13 12:28:47,985 Model(\n", "\tencoder=TransformerEncoder(num_layers=6, num_heads=4),\n", "\tdecoder=TransformerDecoder(num_layers=6, num_heads=4),\n", "\tsrc_embed=Embeddings(embedding_dim=256, vocab_size=39787),\n", "\ttrg_embed=Embeddings(embedding_dim=256, vocab_size=39787))\n", "2020-02-13 12:28:48,014 EPOCH 1\n", "2020-02-13 12:29:26,025 Epoch 1 Step: 100 Batch Loss: 5.604019 Tokens per Sec: 6440, Lr: 0.000300\n", "2020-02-13 12:30:02,054 Epoch 1 Step: 200 Batch Loss: 5.276925 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 12:30:38,146 Epoch 1 Step: 300 Batch Loss: 5.029940 Tokens per Sec: 6795, Lr: 0.000300\n", "2020-02-13 12:31:13,948 Epoch 1 Step: 400 Batch Loss: 5.123002 Tokens per Sec: 6726, Lr: 0.000300\n", "2020-02-13 12:31:50,656 Epoch 1 Step: 500 Batch Loss: 4.771174 Tokens per Sec: 6771, Lr: 0.000300\n", "2020-02-13 12:32:27,580 Epoch 1 Step: 600 Batch Loss: 4.525446 Tokens per Sec: 6685, Lr: 0.000300\n", "2020-02-13 12:33:04,252 Epoch 1 Step: 700 Batch Loss: 4.614151 Tokens per Sec: 6615, Lr: 0.000300\n", "2020-02-13 12:33:40,925 Epoch 1 Step: 800 Batch Loss: 4.606488 Tokens per Sec: 6651, Lr: 0.000300\n", "2020-02-13 12:34:17,635 Epoch 1 Step: 900 Batch Loss: 4.372069 Tokens per Sec: 6569, Lr: 0.000300\n", "2020-02-13 12:34:54,559 Epoch 1 Step: 1000 Batch Loss: 4.198009 Tokens per Sec: 6748, Lr: 0.000300\n", "2020-02-13 12:36:16,772 Hooray! New best validation result [ppl]!\n", "2020-02-13 12:36:16,772 Saving new checkpoint.\n", "2020-02-13 12:36:17,371 Example #0\n", "2020-02-13 12:36:17,371 \tSource: Has the situation improved since then ?\n", "2020-02-13 12:36:17,371 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 12:36:17,371 \tHypothesis: Ho ne ho le joalo ?\n", "2020-02-13 12:36:17,371 Example #1\n", "2020-02-13 12:36:17,372 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 12:36:17,372 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 12:36:17,372 \tHypothesis: Ka hoja ho le joalo , ho ne a ne a le le joalo , ho ne a le ho le ’ na le ho le ho le ho le joalo .\n", "2020-02-13 12:36:17,372 Example #2\n", "2020-02-13 12:36:17,372 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 12:36:17,372 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 12:36:17,372 \tHypothesis: Ka mohlala , , ho ile a ile a re : “ Ho ne a le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le le joalo , le joalo , le joalo , le joalo , le joalo\n", "2020-02-13 12:36:17,372 Example #3\n", "2020-02-13 12:36:17,373 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 12:36:17,373 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 12:36:17,373 \tHypothesis: Ho ne ho le joalo , ho ne a le joalo , ho le ho le ho le ’ na le ho le joalo .\n", "2020-02-13 12:36:17,373 Validation result (greedy) at epoch 1, step 1000: bleu: 0.96, loss: 108547.9297, ppl: 70.9966, duration: 82.8139s\n", "2020-02-13 12:36:54,500 Epoch 1 Step: 1100 Batch Loss: 4.187375 Tokens per Sec: 6734, Lr: 0.000300\n", "2020-02-13 12:37:31,327 Epoch 1 Step: 1200 Batch Loss: 4.051980 Tokens per Sec: 6610, Lr: 0.000300\n", "2020-02-13 12:38:07,838 Epoch 1 Step: 1300 Batch Loss: 3.827433 Tokens per Sec: 6604, Lr: 0.000300\n", "2020-02-13 12:38:44,793 Epoch 1 Step: 1400 Batch Loss: 3.778332 Tokens per Sec: 6713, Lr: 0.000300\n", "2020-02-13 12:39:21,394 Epoch 1 Step: 1500 Batch Loss: 3.662771 Tokens per Sec: 6554, Lr: 0.000300\n", "2020-02-13 12:39:57,805 Epoch 1 Step: 1600 Batch Loss: 3.837494 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 12:40:34,371 Epoch 1 Step: 1700 Batch Loss: 3.434447 Tokens per Sec: 6596, Lr: 0.000300\n", "2020-02-13 12:41:11,230 Epoch 1 Step: 1800 Batch Loss: 3.536253 Tokens per Sec: 6567, Lr: 0.000300\n", "2020-02-13 12:41:47,881 Epoch 1 Step: 1900 Batch Loss: 3.572351 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 12:42:24,832 Epoch 1 Step: 2000 Batch Loss: 3.972955 Tokens per Sec: 6702, Lr: 0.000300\n", "2020-02-13 12:43:47,224 Hooray! New best validation result [ppl]!\n", "2020-02-13 12:43:47,224 Saving new checkpoint.\n", "2020-02-13 12:43:47,763 Example #0\n", "2020-02-13 12:43:47,763 \tSource: Has the situation improved since then ?\n", "2020-02-13 12:43:47,763 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 12:43:47,763 \tHypothesis: Na ho na le ho na le ho na le ho na le ho ba bang ?\n", "2020-02-13 12:43:47,764 Example #1\n", "2020-02-13 12:43:47,764 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 12:43:47,764 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 12:43:47,764 \tHypothesis: Ho feta moo , Jehova o ile a re : “ Ho ne ho le joalo , ho ile ha etsa joalo .\n", "2020-02-13 12:43:47,764 Example #2\n", "2020-02-13 12:43:47,764 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 12:43:47,764 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 12:43:47,764 \tHypothesis: Ho ne ho le joalo , ho ne ho le joalo ha ho ne ho e - na le ho e - na le ho ba le ho ba le ho ba le ho ba le ho ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba ne ba le ba le ba sa feleng .\n", "2020-02-13 12:43:47,765 Example #3\n", "2020-02-13 12:43:47,765 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 12:43:47,765 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 12:43:47,765 \tHypothesis: Ho e - na le hoo , ho ne ho le ho e - na le ho ba le ho ba le ho ba bang ba bang ba bang ba bang ba bang ba bang ba bang ba bang .\n", "2020-02-13 12:43:47,765 Validation result (greedy) at epoch 1, step 2000: bleu: 1.73, loss: 88209.5547, ppl: 31.9430, duration: 82.9332s\n", "2020-02-13 12:44:24,451 Epoch 1 Step: 2100 Batch Loss: 3.305225 Tokens per Sec: 6610, Lr: 0.000300\n", "2020-02-13 12:45:01,318 Epoch 1 Step: 2200 Batch Loss: 3.527070 Tokens per Sec: 6528, Lr: 0.000300\n", "2020-02-13 12:45:38,325 Epoch 1 Step: 2300 Batch Loss: 3.264602 Tokens per Sec: 6655, Lr: 0.000300\n", "2020-02-13 12:46:15,388 Epoch 1 Step: 2400 Batch Loss: 3.658854 Tokens per Sec: 6605, Lr: 0.000300\n", "2020-02-13 12:46:52,089 Epoch 1 Step: 2500 Batch Loss: 3.425530 Tokens per Sec: 6602, Lr: 0.000300\n", "2020-02-13 12:47:29,064 Epoch 1 Step: 2600 Batch Loss: 3.104963 Tokens per Sec: 6711, Lr: 0.000300\n", "2020-02-13 12:48:06,043 Epoch 1 Step: 2700 Batch Loss: 3.424580 Tokens per Sec: 6532, Lr: 0.000300\n", "2020-02-13 12:48:42,703 Epoch 1 Step: 2800 Batch Loss: 3.212284 Tokens per Sec: 6678, Lr: 0.000300\n", "2020-02-13 12:49:19,546 Epoch 1 Step: 2900 Batch Loss: 3.310836 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 12:49:56,658 Epoch 1 Step: 3000 Batch Loss: 3.122866 Tokens per Sec: 6592, Lr: 0.000300\n", "2020-02-13 12:51:18,911 Hooray! New best validation result [ppl]!\n", "2020-02-13 12:51:18,912 Saving new checkpoint.\n", "2020-02-13 12:51:19,351 Example #0\n", "2020-02-13 12:51:19,351 \tSource: Has the situation improved since then ?\n", "2020-02-13 12:51:19,351 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 12:51:19,351 \tHypothesis: Na ho ne ho e - na le hoo ho ne ho e - na le hoo ?\n", "2020-02-13 12:51:19,351 Example #1\n", "2020-02-13 12:51:19,451 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 12:51:19,451 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 12:51:19,451 \tHypothesis: Ho feta moo , Jehova o ile a bontša hore a be le ho sebelisa lijo tsa hae .\n", "2020-02-13 12:51:19,451 Example #2\n", "2020-02-13 12:51:19,452 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 12:51:19,452 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 12:51:19,452 \tHypothesis: Ho ile ha etsa hore ho ne ho e - na le hoo , ho ne ho e - na le hoo ho ne ho e - na le ho e - na le ho etsa hore ho be le ho se ke ha etsahala .\n", "2020-02-13 12:51:19,452 Example #3\n", "2020-02-13 12:51:19,452 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 12:51:19,452 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 12:51:19,452 \tHypothesis: Kaha ho na le hoo , ho ile ha etsahala hore ho na le ho tloha ka ho fetisisa .\n", "2020-02-13 12:51:19,452 Validation result (greedy) at epoch 1, step 3000: bleu: 4.70, loss: 76930.9219, ppl: 20.5127, duration: 82.7940s\n", "2020-02-13 12:51:56,427 Epoch 1 Step: 3100 Batch Loss: 2.955669 Tokens per Sec: 6552, Lr: 0.000300\n", "2020-02-13 12:52:33,328 Epoch 1 Step: 3200 Batch Loss: 2.990921 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 12:53:10,053 Epoch 1 Step: 3300 Batch Loss: 3.548205 Tokens per Sec: 6713, Lr: 0.000300\n", "2020-02-13 12:53:46,817 Epoch 1 Step: 3400 Batch Loss: 3.161317 Tokens per Sec: 6566, Lr: 0.000300\n", "2020-02-13 12:54:23,645 Epoch 1 Step: 3500 Batch Loss: 3.215340 Tokens per Sec: 6705, Lr: 0.000300\n", "2020-02-13 12:55:00,309 Epoch 1 Step: 3600 Batch Loss: 3.212977 Tokens per Sec: 6677, Lr: 0.000300\n", "2020-02-13 12:55:37,374 Epoch 1 Step: 3700 Batch Loss: 3.218235 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 12:56:13,875 Epoch 1 Step: 3800 Batch Loss: 3.174790 Tokens per Sec: 6599, Lr: 0.000300\n", "2020-02-13 12:56:50,754 Epoch 1 Step: 3900 Batch Loss: 2.709258 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 12:57:27,516 Epoch 1 Step: 4000 Batch Loss: 2.845440 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 12:58:47,685 Hooray! New best validation result [ppl]!\n", "2020-02-13 12:58:47,685 Saving new checkpoint.\n", "2020-02-13 12:58:48,235 Example #0\n", "2020-02-13 12:58:48,236 \tSource: Has the situation improved since then ?\n", "2020-02-13 12:58:48,236 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 12:58:48,236 \tHypothesis: Na ho na le bothata bo tšoanang le ho feta ?\n", "2020-02-13 12:58:48,236 Example #1\n", "2020-02-13 12:58:48,236 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 12:58:48,236 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 12:58:48,236 \tHypothesis: Ho feta moo , ha Jehova a ile a laela Jobo hore a etse lintho tse bonahalang .\n", "2020-02-13 12:58:48,236 Example #2\n", "2020-02-13 12:58:48,237 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 12:58:48,237 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 12:58:48,237 \tHypothesis: Monna ea bohlale o ne a le mong oa ho eena : “ Ho ne ho e - na le ho mo fa matla a ho etsa hore a se ke a ja , a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le a le\n", "2020-02-13 12:58:48,237 Example #3\n", "2020-02-13 12:58:48,237 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 12:58:48,237 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 12:58:48,237 \tHypothesis: Kaha ho ne ho e - na le nako e telele , ho na le tlaleho ea ho qetela ea ho qetela .\n", "2020-02-13 12:58:48,237 Validation result (greedy) at epoch 1, step 4000: bleu: 9.22, loss: 68434.7812, ppl: 14.6935, duration: 80.7208s\n", "2020-02-13 12:59:25,279 Epoch 1 Step: 4100 Batch Loss: 2.612687 Tokens per Sec: 6611, Lr: 0.000300\n", "2020-02-13 13:00:01,835 Epoch 1 Step: 4200 Batch Loss: 2.464484 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 13:00:38,737 Epoch 1 Step: 4300 Batch Loss: 2.777717 Tokens per Sec: 6676, Lr: 0.000300\n", "2020-02-13 13:01:15,484 Epoch 1 Step: 4400 Batch Loss: 2.460826 Tokens per Sec: 6482, Lr: 0.000300\n", "2020-02-13 13:01:52,406 Epoch 1 Step: 4500 Batch Loss: 3.154388 Tokens per Sec: 6603, Lr: 0.000300\n", "2020-02-13 13:02:28,737 Epoch 1 Step: 4600 Batch Loss: 2.580979 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 13:03:05,628 Epoch 1 Step: 4700 Batch Loss: 2.568171 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 13:03:42,464 Epoch 1 Step: 4800 Batch Loss: 3.164609 Tokens per Sec: 6720, Lr: 0.000300\n", "2020-02-13 13:04:19,528 Epoch 1 Step: 4900 Batch Loss: 2.989085 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 13:04:56,189 Epoch 1 Step: 5000 Batch Loss: 2.555056 Tokens per Sec: 6607, Lr: 0.000300\n", "2020-02-13 13:05:46,902 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:05:46,902 Saving new checkpoint.\n", "2020-02-13 13:05:47,418 Example #0\n", "2020-02-13 13:05:47,418 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:05:47,419 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:05:47,419 \tHypothesis: Na boemo bo ile ba etsa hore ho be le boemo bo itseng ?\n", "2020-02-13 13:05:47,419 Example #1\n", "2020-02-13 13:05:47,419 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:05:47,419 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:05:47,419 \tHypothesis: Ho feta moo , joalokaha Jesu a ile a bolella Jobo , a fana ka matla a hae a ho bolela matla a hae .\n", "2020-02-13 13:05:47,419 Example #2\n", "2020-02-13 13:05:47,419 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:05:47,419 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:05:47,419 \tHypothesis: Monna e mong o ne a e - na le eena a re : “ Ho ne ho e - na le ho bua ka ho etsa hore motho a se ke a e - ba le ho ba le matla a ho haha moea .\n", "2020-02-13 13:05:47,420 Example #3\n", "2020-02-13 13:05:47,420 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:05:47,420 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:05:47,420 \tHypothesis: Kaha ho ne ho e - na le nako e telele , ho na le ho latela puo e eketsehileng .\n", "2020-02-13 13:05:47,420 Validation result (greedy) at epoch 1, step 5000: bleu: 13.86, loss: 61544.3281, ppl: 11.2102, duration: 51.2307s\n", "2020-02-13 13:06:24,331 Epoch 1 Step: 5100 Batch Loss: 2.497599 Tokens per Sec: 6523, Lr: 0.000300\n", "2020-02-13 13:07:01,170 Epoch 1 Step: 5200 Batch Loss: 2.643443 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 13:07:37,802 Epoch 1 Step: 5300 Batch Loss: 2.452331 Tokens per Sec: 6674, Lr: 0.000300\n", "2020-02-13 13:08:14,466 Epoch 1 Step: 5400 Batch Loss: 2.445484 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 13:08:51,742 Epoch 1 Step: 5500 Batch Loss: 2.564070 Tokens per Sec: 6635, Lr: 0.000300\n", "2020-02-13 13:09:28,438 Epoch 1 Step: 5600 Batch Loss: 2.389540 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 13:10:05,150 Epoch 1 Step: 5700 Batch Loss: 2.574234 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 13:10:41,475 Epoch 1 Step: 5800 Batch Loss: 2.508703 Tokens per Sec: 6593, Lr: 0.000300\n", "2020-02-13 13:11:18,345 Epoch 1 Step: 5900 Batch Loss: 2.388754 Tokens per Sec: 6582, Lr: 0.000300\n", "2020-02-13 13:11:55,199 Epoch 1 Step: 6000 Batch Loss: 2.691465 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 13:12:57,119 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:12:57,120 Saving new checkpoint.\n", "2020-02-13 13:12:57,633 Example #0\n", "2020-02-13 13:12:57,633 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:12:57,633 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:12:57,633 \tHypothesis: Na boemo bo ile ba eketseha ka mor’a moo ?\n", "2020-02-13 13:12:57,633 Example #1\n", "2020-02-13 13:12:57,634 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:12:57,634 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:12:57,634 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo tse bontšang hore li ne li bontša matla a hae .\n", "2020-02-13 13:12:57,634 Example #2\n", "2020-02-13 13:12:57,634 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:12:57,634 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:12:57,634 \tHypothesis: Monna ea bohlale o ne a sa bue ka eena : “ E mong ea neng a bolelloa hore a e’o kena ka ho ea ka ho ea ka ho ea ka ho ea ka ho ea ka ho ea ka ho ea ka moea , o ile oa etsa hore a be le matla .\n", "2020-02-13 13:12:57,634 Example #3\n", "2020-02-13 13:12:57,634 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:12:57,634 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:12:57,634 \tHypothesis: Kaha nako e telele , puo ea puo e ntse e le e eketsehileng e tla ba khanya .\n", "2020-02-13 13:12:57,634 Validation result (greedy) at epoch 1, step 6000: bleu: 17.99, loss: 56446.8750, ppl: 9.1765, duration: 62.4350s\n", "2020-02-13 13:13:34,482 Epoch 1 Step: 6100 Batch Loss: 2.461023 Tokens per Sec: 6512, Lr: 0.000300\n", "2020-02-13 13:14:10,856 Epoch 1 Step: 6200 Batch Loss: 2.517843 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 13:14:47,914 Epoch 1 Step: 6300 Batch Loss: 2.309825 Tokens per Sec: 6581, Lr: 0.000300\n", "2020-02-13 13:15:25,018 Epoch 1 Step: 6400 Batch Loss: 2.193013 Tokens per Sec: 6552, Lr: 0.000300\n", "2020-02-13 13:16:01,584 Epoch 1 Step: 6500 Batch Loss: 2.396491 Tokens per Sec: 6776, Lr: 0.000300\n", "2020-02-13 13:16:38,606 Epoch 1 Step: 6600 Batch Loss: 2.857018 Tokens per Sec: 6564, Lr: 0.000300\n", "2020-02-13 13:17:15,136 Epoch 1 Step: 6700 Batch Loss: 2.129714 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 13:17:51,755 Epoch 1 Step: 6800 Batch Loss: 2.153850 Tokens per Sec: 6594, Lr: 0.000300\n", "2020-02-13 13:18:28,598 Epoch 1 Step: 6900 Batch Loss: 2.463159 Tokens per Sec: 6490, Lr: 0.000300\n", "2020-02-13 13:19:05,640 Epoch 1 Step: 7000 Batch Loss: 2.042279 Tokens per Sec: 6683, Lr: 0.000300\n", "2020-02-13 13:20:02,263 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:20:02,263 Saving new checkpoint.\n", "2020-02-13 13:20:02,774 Example #0\n", "2020-02-13 13:20:02,775 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:20:02,775 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:20:02,775 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha nakong eo ?\n", "2020-02-13 13:20:02,775 Example #1\n", "2020-02-13 13:20:02,775 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:20:02,775 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:20:02,775 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo tsa hae li bontša matla a hae .\n", "2020-02-13 13:20:02,775 Example #2\n", "2020-02-13 13:20:02,775 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:20:02,775 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:20:02,775 \tHypothesis: Monna ea bohlale o ne a kōpa hore a kōpe : “ Ho tloha ka ho ea ka ho ea ho ea ka ho ea ho ea ka ho ea ka ho ea ka ho ea ka ho ea ka ho ea ho ea ka ho ea ho ea ho ea ka moea .\n", "2020-02-13 13:20:02,776 Example #3\n", "2020-02-13 13:20:02,776 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:20:02,776 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:20:02,776 \tHypothesis: Kaha ho ne ho e - na le nako e telele , libuka tse ngotsoeng ka letsoho li tla ba le leseli .\n", "2020-02-13 13:20:02,776 Validation result (greedy) at epoch 1, step 7000: bleu: 21.54, loss: 52346.0156, ppl: 7.8116, duration: 57.1359s\n", "2020-02-13 13:20:39,408 Epoch 1 Step: 7100 Batch Loss: 2.274565 Tokens per Sec: 6644, Lr: 0.000300\n", "2020-02-13 13:21:15,917 Epoch 1 Step: 7200 Batch Loss: 1.981828 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 13:21:52,973 Epoch 1 Step: 7300 Batch Loss: 2.017133 Tokens per Sec: 6605, Lr: 0.000300\n", "2020-02-13 13:22:29,945 Epoch 1 Step: 7400 Batch Loss: 2.235154 Tokens per Sec: 6725, Lr: 0.000300\n", "2020-02-13 13:23:07,090 Epoch 1 Step: 7500 Batch Loss: 2.064981 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 13:23:43,431 Epoch 1 Step: 7600 Batch Loss: 2.191868 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 13:24:19,978 Epoch 1 Step: 7700 Batch Loss: 1.925988 Tokens per Sec: 6574, Lr: 0.000300\n", "2020-02-13 13:24:56,607 Epoch 1 Step: 7800 Batch Loss: 2.087967 Tokens per Sec: 6421, Lr: 0.000300\n", "2020-02-13 13:25:33,436 Epoch 1 Step: 7900 Batch Loss: 2.364287 Tokens per Sec: 6540, Lr: 0.000300\n", "2020-02-13 13:26:10,457 Epoch 1 Step: 8000 Batch Loss: 2.216875 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 13:26:56,234 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:26:56,234 Saving new checkpoint.\n", "2020-02-13 13:26:56,795 Example #0\n", "2020-02-13 13:26:56,796 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:26:56,796 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:26:56,796 \tHypothesis: Na boemo bo ile ba atleha ho tloha ka nako ?\n", "2020-02-13 13:26:56,796 Example #1\n", "2020-02-13 13:26:56,796 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:26:56,796 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:26:56,796 \tHypothesis: Ho feta moo , joalokaha Jehova a hopotsa Jobo , liphoofolo li paka hore li ne li paka matla a Hae .\n", "2020-02-13 13:26:56,796 Example #2\n", "2020-02-13 13:26:56,796 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:26:56,796 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:26:56,797 \tHypothesis: Monna ea bohlale o ne a lokela ho kōpa tšoarelo : “ Ho ne ho bolelloa hore motho a tsamaee le ho ea ka hlooho ea mofuta o moputsoa le ho e - na le ho e - na le ho e - na le ho beha moea .\n", "2020-02-13 13:26:56,797 Example #3\n", "2020-02-13 13:26:56,797 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:26:56,797 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:26:56,797 \tHypothesis: Kaha nako e telele ea ho buuoa ka letsoho , libuka tse ngotsoeng ka letsoho li tla fihla .\n", "2020-02-13 13:26:56,797 Validation result (greedy) at epoch 1, step 8000: bleu: 25.72, loss: 49132.6406, ppl: 6.8855, duration: 46.3399s\n", "2020-02-13 13:27:33,416 Epoch 1 Step: 8100 Batch Loss: 2.089566 Tokens per Sec: 6565, Lr: 0.000300\n", "2020-02-13 13:28:10,628 Epoch 1 Step: 8200 Batch Loss: 2.033558 Tokens per Sec: 6587, Lr: 0.000300\n", "2020-02-13 13:28:47,509 Epoch 1 Step: 8300 Batch Loss: 1.992463 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 13:29:24,308 Epoch 1 Step: 8400 Batch Loss: 1.931446 Tokens per Sec: 6561, Lr: 0.000300\n", "2020-02-13 13:30:01,196 Epoch 1 Step: 8500 Batch Loss: 2.053720 Tokens per Sec: 6694, Lr: 0.000300\n", "2020-02-13 13:30:38,013 Epoch 1 Step: 8600 Batch Loss: 2.023182 Tokens per Sec: 6632, Lr: 0.000300\n", "2020-02-13 13:31:15,113 Epoch 1 Step: 8700 Batch Loss: 2.023627 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 13:31:20,050 Epoch 1: total training loss 26545.08\n", "2020-02-13 13:31:20,050 EPOCH 2\n", "2020-02-13 13:31:52,925 Epoch 2 Step: 8800 Batch Loss: 1.915689 Tokens per Sec: 6408, Lr: 0.000300\n", "2020-02-13 13:32:29,858 Epoch 2 Step: 8900 Batch Loss: 2.054130 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 13:33:06,860 Epoch 2 Step: 9000 Batch Loss: 1.833462 Tokens per Sec: 6671, Lr: 0.000300\n", "2020-02-13 13:33:43,367 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:33:43,367 Saving new checkpoint.\n", "2020-02-13 13:33:43,827 Example #0\n", "2020-02-13 13:33:43,827 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:33:43,827 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:33:43,827 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha nakong eo ?\n", "2020-02-13 13:33:43,827 Example #1\n", "2020-02-13 13:33:43,827 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:33:43,827 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:33:43,827 \tHypothesis: Ho feta moo , joalokaha Jehova a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 13:33:43,827 Example #2\n", "2020-02-13 13:33:43,827 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:33:43,827 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:33:43,827 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Ho ne ho bolelloa hore a tsamaee le ho ntša seaparo se nang le sefahla - peloana ’ me a e - na le majoe a ho a ho a beha moea .\n", "2020-02-13 13:33:43,827 Example #3\n", "2020-02-13 13:33:43,828 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:33:43,828 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:33:43,828 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngata tse ngotsoeng ka letsoho li tla fihla leseling .\n", "2020-02-13 13:33:43,828 Validation result (greedy) at epoch 2, step 9000: bleu: 27.71, loss: 46748.5625, ppl: 6.2701, duration: 36.9671s\n", "2020-02-13 13:34:19,971 Epoch 2 Step: 9100 Batch Loss: 1.989191 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 13:34:56,465 Epoch 2 Step: 9200 Batch Loss: 1.946299 Tokens per Sec: 6459, Lr: 0.000300\n", "2020-02-13 13:35:33,439 Epoch 2 Step: 9300 Batch Loss: 1.971216 Tokens per Sec: 6538, Lr: 0.000300\n", "2020-02-13 13:36:09,959 Epoch 2 Step: 9400 Batch Loss: 1.854259 Tokens per Sec: 6505, Lr: 0.000300\n", "2020-02-13 13:36:46,296 Epoch 2 Step: 9500 Batch Loss: 1.905197 Tokens per Sec: 6531, Lr: 0.000300\n", "2020-02-13 13:37:23,468 Epoch 2 Step: 9600 Batch Loss: 2.044919 Tokens per Sec: 6700, Lr: 0.000300\n", "2020-02-13 13:37:59,911 Epoch 2 Step: 9700 Batch Loss: 1.799374 Tokens per Sec: 6666, Lr: 0.000300\n", "2020-02-13 13:38:36,640 Epoch 2 Step: 9800 Batch Loss: 1.918280 Tokens per Sec: 6619, Lr: 0.000300\n", "2020-02-13 13:39:13,316 Epoch 2 Step: 9900 Batch Loss: 2.018281 Tokens per Sec: 6607, Lr: 0.000300\n", "2020-02-13 13:39:50,307 Epoch 2 Step: 10000 Batch Loss: 1.824778 Tokens per Sec: 6738, Lr: 0.000300\n", "2020-02-13 13:40:31,619 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:40:31,619 Saving new checkpoint.\n", "2020-02-13 13:40:32,118 Example #0\n", "2020-02-13 13:40:32,118 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:40:32,118 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:40:32,118 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha nakong eo ?\n", "2020-02-13 13:40:32,119 Example #1\n", "2020-02-13 13:40:32,119 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:40:32,119 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:40:32,119 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 13:40:32,119 Example #2\n", "2020-02-13 13:40:32,119 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:40:32,119 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:40:32,119 \tHypothesis: Monna ea bohlale o ne a kōpa : Ho bua ka tsela e ’ ngoe le e ’ ngoe ho ile ha bolelloa hore a tsamaee le ho nka e le ho ba le letšoao la moea .\n", "2020-02-13 13:40:32,119 Example #3\n", "2020-02-13 13:40:32,119 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:40:32,119 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:40:32,119 \tHypothesis: Kaha ho ne ho e - na le nako ea ho fumana libuka tse eketsehileng tse ngotsoeng ka letsoho ho tla ba le leseli .\n", "2020-02-13 13:40:32,120 Validation result (greedy) at epoch 2, step 10000: bleu: 29.18, loss: 45489.0039, ppl: 5.9675, duration: 41.8125s\n", "2020-02-13 13:41:09,001 Epoch 2 Step: 10100 Batch Loss: 2.007888 Tokens per Sec: 6615, Lr: 0.000300\n", "2020-02-13 13:41:45,735 Epoch 2 Step: 10200 Batch Loss: 2.078129 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 13:42:22,018 Epoch 2 Step: 10300 Batch Loss: 2.024773 Tokens per Sec: 6627, Lr: 0.000300\n", "2020-02-13 13:42:58,918 Epoch 2 Step: 10400 Batch Loss: 1.794512 Tokens per Sec: 6580, Lr: 0.000300\n", "2020-02-13 13:43:35,326 Epoch 2 Step: 10500 Batch Loss: 1.771951 Tokens per Sec: 6607, Lr: 0.000300\n", "2020-02-13 13:44:11,643 Epoch 2 Step: 10600 Batch Loss: 2.218677 Tokens per Sec: 6521, Lr: 0.000300\n", "2020-02-13 13:44:48,366 Epoch 2 Step: 10700 Batch Loss: 1.956860 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 13:45:25,035 Epoch 2 Step: 10800 Batch Loss: 1.955985 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 13:46:01,577 Epoch 2 Step: 10900 Batch Loss: 1.779067 Tokens per Sec: 6594, Lr: 0.000300\n", "2020-02-13 13:46:38,660 Epoch 2 Step: 11000 Batch Loss: 2.235164 Tokens per Sec: 6718, Lr: 0.000300\n", "2020-02-13 13:47:18,075 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:47:18,076 Saving new checkpoint.\n", "2020-02-13 13:47:18,554 Example #0\n", "2020-02-13 13:47:18,554 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:47:18,554 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:47:18,554 \tHypothesis: Na boemo bo ntlafetse ho tloha nakong eo ?\n", "2020-02-13 13:47:18,554 Example #1\n", "2020-02-13 13:47:18,554 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:47:18,554 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:47:18,554 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka ka matla a Hae .\n", "2020-02-13 13:47:18,555 Example #2\n", "2020-02-13 13:47:18,555 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:47:18,555 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:47:18,555 \tHypothesis: Monna ea bohlale o ne a kōpa : “ Ho bua leshano ho ile ha boleloa hore a ee ’ me a e - na le letšoao le hlabang le ho e ntša , a e - na le mapheo a moea .\n", "2020-02-13 13:47:18,555 Example #3\n", "2020-02-13 13:47:18,555 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:47:18,555 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:47:18,555 \tHypothesis: Kaha nako e ntse e eketseha , libuka tse fetang tse ngotsoeng ka letsoho li tla ba teng .\n", "2020-02-13 13:47:18,555 Validation result (greedy) at epoch 2, step 11000: bleu: 30.87, loss: 42909.7266, ppl: 5.3927, duration: 39.8952s\n", "2020-02-13 13:47:55,376 Epoch 2 Step: 11100 Batch Loss: 1.843343 Tokens per Sec: 6735, Lr: 0.000300\n", "2020-02-13 13:48:31,817 Epoch 2 Step: 11200 Batch Loss: 1.716015 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 13:49:08,533 Epoch 2 Step: 11300 Batch Loss: 2.048210 Tokens per Sec: 6563, Lr: 0.000300\n", "2020-02-13 13:49:45,135 Epoch 2 Step: 11400 Batch Loss: 1.748264 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 13:50:21,864 Epoch 2 Step: 11500 Batch Loss: 1.968525 Tokens per Sec: 6529, Lr: 0.000300\n", "2020-02-13 13:50:58,792 Epoch 2 Step: 11600 Batch Loss: 1.883746 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 13:51:35,757 Epoch 2 Step: 11700 Batch Loss: 1.670517 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 13:52:12,191 Epoch 2 Step: 11800 Batch Loss: 2.346115 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 13:52:49,027 Epoch 2 Step: 11900 Batch Loss: 1.909031 Tokens per Sec: 6535, Lr: 0.000300\n", "2020-02-13 13:53:26,060 Epoch 2 Step: 12000 Batch Loss: 2.009962 Tokens per Sec: 6531, Lr: 0.000300\n", "2020-02-13 13:54:09,221 Hooray! New best validation result [ppl]!\n", "2020-02-13 13:54:09,222 Saving new checkpoint.\n", "2020-02-13 13:54:09,722 Example #0\n", "2020-02-13 13:54:09,722 \tSource: Has the situation improved since then ?\n", "2020-02-13 13:54:09,722 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 13:54:09,722 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha nakong eo ?\n", "2020-02-13 13:54:09,722 Example #1\n", "2020-02-13 13:54:09,723 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 13:54:09,723 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 13:54:09,723 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 13:54:09,723 Example #2\n", "2020-02-13 13:54:09,723 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 13:54:09,723 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 13:54:09,723 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : “ Momoetsi o ne a bolelloa hore a ee ’ me a nke mopholosi oa momomomomomomochoo , a e - na le masiba a moea .\n", "2020-02-13 13:54:09,723 Example #3\n", "2020-02-13 13:54:09,723 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 13:54:09,723 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 13:54:09,723 \tHypothesis: Kaha nako ea eona e ne e le ka holimo , libuka tse ling tse ngotsoeng ka letsoho li tla leseling .\n", "2020-02-13 13:54:09,723 Validation result (greedy) at epoch 2, step 12000: bleu: 32.45, loss: 41262.1016, ppl: 5.0548, duration: 43.6625s\n", "2020-02-13 13:54:46,348 Epoch 2 Step: 12100 Batch Loss: 1.575211 Tokens per Sec: 6652, Lr: 0.000300\n", "2020-02-13 13:55:23,230 Epoch 2 Step: 12200 Batch Loss: 1.747776 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 13:55:59,911 Epoch 2 Step: 12300 Batch Loss: 1.702763 Tokens per Sec: 6560, Lr: 0.000300\n", "2020-02-13 13:56:36,987 Epoch 2 Step: 12400 Batch Loss: 1.701364 Tokens per Sec: 6558, Lr: 0.000300\n", "2020-02-13 13:57:13,779 Epoch 2 Step: 12500 Batch Loss: 1.720034 Tokens per Sec: 6580, Lr: 0.000300\n", "2020-02-13 13:57:50,493 Epoch 2 Step: 12600 Batch Loss: 1.579598 Tokens per Sec: 6764, Lr: 0.000300\n", "2020-02-13 13:58:27,389 Epoch 2 Step: 12700 Batch Loss: 1.874493 Tokens per Sec: 6670, Lr: 0.000300\n", "2020-02-13 13:59:03,768 Epoch 2 Step: 12800 Batch Loss: 1.805818 Tokens per Sec: 6471, Lr: 0.000300\n", "2020-02-13 13:59:40,788 Epoch 2 Step: 12900 Batch Loss: 1.772384 Tokens per Sec: 6666, Lr: 0.000300\n", "2020-02-13 14:00:17,366 Epoch 2 Step: 13000 Batch Loss: 1.903513 Tokens per Sec: 6620, Lr: 0.000300\n", "2020-02-13 14:00:57,652 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:00:57,652 Saving new checkpoint.\n", "2020-02-13 14:00:58,192 Example #0\n", "2020-02-13 14:00:58,193 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:00:58,193 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:00:58,193 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha ka nako eo ?\n", "2020-02-13 14:00:58,193 Example #1\n", "2020-02-13 14:00:58,193 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:00:58,193 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:00:58,193 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:00:58,193 Example #2\n", "2020-02-13 14:00:58,193 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:00:58,193 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:00:58,194 \tHypothesis: Monna ea bohlale o ne a kōpa : Ho bua leshano ka ho toba le ho nka moloki oa moloki oa mapolanka le ho e ntša , a e - na le masiba a ho a chesa .\n", "2020-02-13 14:00:58,194 Example #3\n", "2020-02-13 14:00:58,194 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:00:58,194 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:00:58,194 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho li tsoa leseling .\n", "2020-02-13 14:00:58,194 Validation result (greedy) at epoch 2, step 13000: bleu: 33.14, loss: 40603.5898, ppl: 4.9258, duration: 40.8278s\n", "2020-02-13 14:01:35,051 Epoch 2 Step: 13100 Batch Loss: 1.555029 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 14:02:11,676 Epoch 2 Step: 13200 Batch Loss: 1.727155 Tokens per Sec: 6600, Lr: 0.000300\n", "2020-02-13 14:02:48,242 Epoch 2 Step: 13300 Batch Loss: 1.804857 Tokens per Sec: 6614, Lr: 0.000300\n", "2020-02-13 14:03:25,427 Epoch 2 Step: 13400 Batch Loss: 1.716516 Tokens per Sec: 6602, Lr: 0.000300\n", "2020-02-13 14:04:01,755 Epoch 2 Step: 13500 Batch Loss: 1.737514 Tokens per Sec: 6559, Lr: 0.000300\n", "2020-02-13 14:04:39,031 Epoch 2 Step: 13600 Batch Loss: 1.739930 Tokens per Sec: 6732, Lr: 0.000300\n", "2020-02-13 14:05:15,872 Epoch 2 Step: 13700 Batch Loss: 1.702925 Tokens per Sec: 6693, Lr: 0.000300\n", "2020-02-13 14:05:52,751 Epoch 2 Step: 13800 Batch Loss: 1.649653 Tokens per Sec: 6659, Lr: 0.000300\n", "2020-02-13 14:06:29,706 Epoch 2 Step: 13900 Batch Loss: 1.762575 Tokens per Sec: 6646, Lr: 0.000300\n", "2020-02-13 14:07:06,182 Epoch 2 Step: 14000 Batch Loss: 1.803161 Tokens per Sec: 6520, Lr: 0.000300\n", "2020-02-13 14:07:42,839 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:07:42,839 Saving new checkpoint.\n", "2020-02-13 14:07:43,380 Example #0\n", "2020-02-13 14:07:43,381 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:07:43,381 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:07:43,381 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:07:43,381 Example #1\n", "2020-02-13 14:07:43,381 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:07:43,381 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:07:43,381 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:07:43,381 Example #2\n", "2020-02-13 14:07:43,381 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:07:43,381 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:07:43,381 \tHypothesis: Monna ea bohlale o ne a kōpa : Motho ea buang ka eena o ile a bolelloa hore a ee ’ me a nke mothamo ea tlaase ’ me a e ntša ka ho mo ntša , a jala masiba a moea .\n", "2020-02-13 14:07:43,381 Example #3\n", "2020-02-13 14:07:43,382 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:07:43,382 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:07:43,382 \tHypothesis: Kaha ho ne ho e - na le nako ea ho etsa libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:07:43,382 Validation result (greedy) at epoch 2, step 14000: bleu: 34.35, loss: 39439.6914, ppl: 4.7057, duration: 37.1999s\n", "2020-02-13 14:08:20,234 Epoch 2 Step: 14100 Batch Loss: 1.852965 Tokens per Sec: 6683, Lr: 0.000300\n", "2020-02-13 14:08:57,339 Epoch 2 Step: 14200 Batch Loss: 2.208117 Tokens per Sec: 6611, Lr: 0.000300\n", "2020-02-13 14:09:33,913 Epoch 2 Step: 14300 Batch Loss: 1.788891 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 14:10:10,782 Epoch 2 Step: 14400 Batch Loss: 1.998637 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 14:10:47,552 Epoch 2 Step: 14500 Batch Loss: 1.735862 Tokens per Sec: 6705, Lr: 0.000300\n", "2020-02-13 14:11:24,471 Epoch 2 Step: 14600 Batch Loss: 1.603061 Tokens per Sec: 6690, Lr: 0.000300\n", "2020-02-13 14:12:01,247 Epoch 2 Step: 14700 Batch Loss: 1.782238 Tokens per Sec: 6579, Lr: 0.000300\n", "2020-02-13 14:12:37,939 Epoch 2 Step: 14800 Batch Loss: 1.901912 Tokens per Sec: 6600, Lr: 0.000300\n", "2020-02-13 14:13:14,810 Epoch 2 Step: 14900 Batch Loss: 1.528630 Tokens per Sec: 6719, Lr: 0.000300\n", "2020-02-13 14:13:51,983 Epoch 2 Step: 15000 Batch Loss: 1.585287 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 14:14:30,046 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:14:30,046 Saving new checkpoint.\n", "2020-02-13 14:14:30,593 Example #0\n", "2020-02-13 14:14:30,593 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:14:30,593 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:14:30,593 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:14:30,593 Example #1\n", "2020-02-13 14:14:30,593 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:14:30,593 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:14:30,593 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:14:30,593 Example #2\n", "2020-02-13 14:14:30,593 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:14:30,593 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:14:30,594 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Ho ne ho bolelloa hore ke ee ’ me a nke motšoantšisi oa motšoantšisi oa motšoantšisi oa moeki ’ me a e lokisa , a jala masiba a moea .\n", "2020-02-13 14:14:30,594 Example #3\n", "2020-02-13 14:14:30,594 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:14:30,594 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:14:30,594 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho li tla khanya .\n", "2020-02-13 14:14:30,594 Validation result (greedy) at epoch 2, step 15000: bleu: 35.67, loss: 38308.2891, ppl: 4.5012, duration: 38.6110s\n", "2020-02-13 14:15:07,308 Epoch 2 Step: 15100 Batch Loss: 2.087998 Tokens per Sec: 6646, Lr: 0.000300\n", "2020-02-13 14:15:44,052 Epoch 2 Step: 15200 Batch Loss: 1.728167 Tokens per Sec: 6619, Lr: 0.000300\n", "2020-02-13 14:16:20,782 Epoch 2 Step: 15300 Batch Loss: 1.657387 Tokens per Sec: 6578, Lr: 0.000300\n", "2020-02-13 14:16:57,273 Epoch 2 Step: 15400 Batch Loss: 1.596512 Tokens per Sec: 6684, Lr: 0.000300\n", "2020-02-13 14:17:34,304 Epoch 2 Step: 15500 Batch Loss: 2.047939 Tokens per Sec: 6565, Lr: 0.000300\n", "2020-02-13 14:18:10,780 Epoch 2 Step: 15600 Batch Loss: 1.728567 Tokens per Sec: 6598, Lr: 0.000300\n", "2020-02-13 14:18:47,450 Epoch 2 Step: 15700 Batch Loss: 1.609479 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 14:19:24,051 Epoch 2 Step: 15800 Batch Loss: 1.695057 Tokens per Sec: 6502, Lr: 0.000300\n", "2020-02-13 14:20:00,724 Epoch 2 Step: 15900 Batch Loss: 1.589164 Tokens per Sec: 6705, Lr: 0.000300\n", "2020-02-13 14:20:37,789 Epoch 2 Step: 16000 Batch Loss: 1.664685 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 14:21:10,536 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:21:10,537 Saving new checkpoint.\n", "2020-02-13 14:21:11,098 Example #0\n", "2020-02-13 14:21:11,098 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:21:11,098 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:21:11,099 \tHypothesis: Na boemo bo ntlafala ho tloha ka nako eo ?\n", "2020-02-13 14:21:11,099 Example #1\n", "2020-02-13 14:21:11,099 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:21:11,099 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:21:11,099 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:21:11,099 Example #2\n", "2020-02-13 14:21:11,099 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:21:11,099 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:21:11,099 \tHypothesis: Monna ea bohlale o ne a kōpa hore a kōpe : “ Motho e mong le e mong o ile a bolelloa hore a ee ’ me a nke letšoao le hlabang ’ me a le romile , a e jala ka majoe .\n", "2020-02-13 14:21:11,099 Example #3\n", "2020-02-13 14:21:11,099 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:21:11,100 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:21:11,100 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 14:21:11,100 Validation result (greedy) at epoch 2, step 16000: bleu: 35.99, loss: 37757.9688, ppl: 4.4050, duration: 33.3107s\n", "2020-02-13 14:21:47,723 Epoch 2 Step: 16100 Batch Loss: 1.715761 Tokens per Sec: 6597, Lr: 0.000300\n", "2020-02-13 14:22:24,676 Epoch 2 Step: 16200 Batch Loss: 1.582835 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 14:23:01,995 Epoch 2 Step: 16300 Batch Loss: 1.609347 Tokens per Sec: 6591, Lr: 0.000300\n", "2020-02-13 14:23:39,000 Epoch 2 Step: 16400 Batch Loss: 1.683865 Tokens per Sec: 6587, Lr: 0.000300\n", "2020-02-13 14:24:16,193 Epoch 2 Step: 16500 Batch Loss: 1.587121 Tokens per Sec: 6766, Lr: 0.000300\n", "2020-02-13 14:24:53,349 Epoch 2 Step: 16600 Batch Loss: 1.429418 Tokens per Sec: 6542, Lr: 0.000300\n", "2020-02-13 14:25:30,020 Epoch 2 Step: 16700 Batch Loss: 1.860909 Tokens per Sec: 6417, Lr: 0.000300\n", "2020-02-13 14:26:06,455 Epoch 2 Step: 16800 Batch Loss: 1.496606 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 14:26:43,222 Epoch 2 Step: 16900 Batch Loss: 1.536534 Tokens per Sec: 6511, Lr: 0.000300\n", "2020-02-13 14:27:19,995 Epoch 2 Step: 17000 Batch Loss: 1.620427 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 14:27:58,067 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:27:58,067 Saving new checkpoint.\n", "2020-02-13 14:27:58,609 Example #0\n", "2020-02-13 14:27:58,610 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:27:58,610 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:27:58,610 \tHypothesis: Na boemo bo ntlafetse ho tloha joale ?\n", "2020-02-13 14:27:58,610 Example #1\n", "2020-02-13 14:27:58,610 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:27:58,610 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:27:58,610 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:27:58,610 Example #2\n", "2020-02-13 14:27:58,610 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:27:58,610 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:27:58,610 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea bitsoang e mong o ile a bolelloa hore a ee ’ me a nke moeki oa setšo ’ me a e tlosa , a jala masiba a moea .\n", "2020-02-13 14:27:58,610 Example #3\n", "2020-02-13 14:27:58,610 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:27:58,610 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:27:58,611 \tHypothesis: Kaha nako ea ho fumana libuka tse ngotsoeng ka letsoho e fihlile leseling .\n", "2020-02-13 14:27:58,611 Validation result (greedy) at epoch 2, step 17000: bleu: 37.04, loss: 36910.7500, ppl: 4.2609, duration: 38.6149s\n", "2020-02-13 14:28:35,405 Epoch 2 Step: 17100 Batch Loss: 1.691413 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 14:29:12,417 Epoch 2 Step: 17200 Batch Loss: 1.433548 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 14:29:48,889 Epoch 2 Step: 17300 Batch Loss: 1.704909 Tokens per Sec: 6739, Lr: 0.000300\n", "2020-02-13 14:30:25,884 Epoch 2 Step: 17400 Batch Loss: 1.469643 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 14:30:40,976 Epoch 2: total training loss 15792.18\n", "2020-02-13 14:30:40,976 EPOCH 3\n", "2020-02-13 14:31:04,003 Epoch 3 Step: 17500 Batch Loss: 1.690038 Tokens per Sec: 6405, Lr: 0.000300\n", "2020-02-13 14:31:40,457 Epoch 3 Step: 17600 Batch Loss: 1.970367 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-13 14:32:17,350 Epoch 3 Step: 17700 Batch Loss: 1.472515 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 14:32:54,591 Epoch 3 Step: 17800 Batch Loss: 1.607201 Tokens per Sec: 6583, Lr: 0.000300\n", "2020-02-13 14:33:31,641 Epoch 3 Step: 17900 Batch Loss: 1.463607 Tokens per Sec: 6722, Lr: 0.000300\n", "2020-02-13 14:34:08,673 Epoch 3 Step: 18000 Batch Loss: 1.429625 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 14:34:43,358 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:34:43,358 Saving new checkpoint.\n", "2020-02-13 14:34:43,856 Example #0\n", "2020-02-13 14:34:43,856 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:34:43,856 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:34:43,857 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:34:43,857 Example #1\n", "2020-02-13 14:34:43,857 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:34:43,857 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:34:43,857 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:34:43,857 Example #2\n", "2020-02-13 14:34:43,857 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:34:43,857 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:34:43,857 \tHypothesis: Monna ea bohlale o ne a kōpile : Motho ea neng a mo kōpa a mo bolella hore a tsamaee le moeki o tlaase ’ me a e khaola , a e hasanya masiba a moea .\n", "2020-02-13 14:34:43,857 Example #3\n", "2020-02-13 14:34:43,858 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:34:43,858 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:34:43,858 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho tse ngotsoeng ka letsoho li tla leseling .\n", "2020-02-13 14:34:43,858 Validation result (greedy) at epoch 3, step 18000: bleu: 37.14, loss: 36366.7852, ppl: 4.1708, duration: 35.1850s\n", "2020-02-13 14:35:20,482 Epoch 3 Step: 18100 Batch Loss: 1.588225 Tokens per Sec: 6719, Lr: 0.000300\n", "2020-02-13 14:35:57,830 Epoch 3 Step: 18200 Batch Loss: 1.758353 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 14:36:34,916 Epoch 3 Step: 18300 Batch Loss: 1.627003 Tokens per Sec: 6765, Lr: 0.000300\n", "2020-02-13 14:37:11,683 Epoch 3 Step: 18400 Batch Loss: 2.237396 Tokens per Sec: 6566, Lr: 0.000300\n", "2020-02-13 14:37:48,817 Epoch 3 Step: 18500 Batch Loss: 1.362483 Tokens per Sec: 6685, Lr: 0.000300\n", "2020-02-13 14:38:26,139 Epoch 3 Step: 18600 Batch Loss: 1.603604 Tokens per Sec: 6577, Lr: 0.000300\n", "2020-02-13 14:39:02,826 Epoch 3 Step: 18700 Batch Loss: 1.512032 Tokens per Sec: 6518, Lr: 0.000300\n", "2020-02-13 14:39:39,372 Epoch 3 Step: 18800 Batch Loss: 1.557069 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 14:40:16,462 Epoch 3 Step: 18900 Batch Loss: 1.680167 Tokens per Sec: 6470, Lr: 0.000300\n", "2020-02-13 14:40:53,576 Epoch 3 Step: 19000 Batch Loss: 1.779119 Tokens per Sec: 6679, Lr: 0.000300\n", "2020-02-13 14:41:32,457 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:41:32,457 Saving new checkpoint.\n", "2020-02-13 14:41:32,988 Example #0\n", "2020-02-13 14:41:32,989 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:41:32,989 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:41:32,989 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:41:32,989 Example #1\n", "2020-02-13 14:41:32,989 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:41:32,989 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:41:32,989 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:41:32,989 Example #2\n", "2020-02-13 14:41:32,989 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:41:32,989 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:41:32,989 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea buang ka motho o ile a bolelloa hore a ee ’ me a nke letšoao le tlaase ’ me a le tlosa , a hasa masiba a moea .\n", "2020-02-13 14:41:32,989 Example #3\n", "2020-02-13 14:41:32,989 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:41:32,990 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:41:32,990 \tHypothesis: Ho tloha nakong ea nako , libuka tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 14:41:32,990 Validation result (greedy) at epoch 3, step 19000: bleu: 37.40, loss: 35914.8750, ppl: 4.0974, duration: 39.4131s\n", "2020-02-13 14:42:09,744 Epoch 3 Step: 19100 Batch Loss: 1.578127 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 14:42:46,274 Epoch 3 Step: 19200 Batch Loss: 1.595193 Tokens per Sec: 6559, Lr: 0.000300\n", "2020-02-13 14:43:23,144 Epoch 3 Step: 19300 Batch Loss: 1.554369 Tokens per Sec: 6541, Lr: 0.000300\n", "2020-02-13 14:44:00,199 Epoch 3 Step: 19400 Batch Loss: 1.522064 Tokens per Sec: 6576, Lr: 0.000300\n", "2020-02-13 14:44:36,803 Epoch 3 Step: 19500 Batch Loss: 1.390383 Tokens per Sec: 6503, Lr: 0.000300\n", "2020-02-13 14:45:13,687 Epoch 3 Step: 19600 Batch Loss: 1.698410 Tokens per Sec: 6667, Lr: 0.000300\n", "2020-02-13 14:45:50,628 Epoch 3 Step: 19700 Batch Loss: 1.548661 Tokens per Sec: 6651, Lr: 0.000300\n", "2020-02-13 14:46:27,293 Epoch 3 Step: 19800 Batch Loss: 1.494191 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 14:47:04,408 Epoch 3 Step: 19900 Batch Loss: 1.622850 Tokens per Sec: 6563, Lr: 0.000300\n", "2020-02-13 14:47:41,637 Epoch 3 Step: 20000 Batch Loss: 1.473348 Tokens per Sec: 6615, Lr: 0.000300\n", "2020-02-13 14:48:16,738 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:48:16,738 Saving new checkpoint.\n", "2020-02-13 14:48:17,185 Example #0\n", "2020-02-13 14:48:17,186 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:48:17,186 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:48:17,186 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:48:17,186 Example #1\n", "2020-02-13 14:48:17,186 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:48:17,186 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:48:17,186 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:48:17,186 Example #2\n", "2020-02-13 14:48:17,186 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:48:17,186 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:48:17,186 \tHypothesis: Monna ea bohlale o ne a kōpa hore a re : Motho ea hlekefetsang o ile a bolelloa hore a ee ’ me a nke motšoantšisi oa mofuta o tlaase ’ me a o tlosa , a jala masiba a moea .\n", "2020-02-13 14:48:17,186 Example #3\n", "2020-02-13 14:48:17,186 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:48:17,187 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:48:17,187 \tHypothesis: Kaha nako ea ho fumana libuka tse ngotsoeng ka letsoho e fihlile leseling .\n", "2020-02-13 14:48:17,187 Validation result (greedy) at epoch 3, step 20000: bleu: 38.03, loss: 35548.4492, ppl: 4.0389, duration: 35.5496s\n", "2020-02-13 14:48:54,073 Epoch 3 Step: 20100 Batch Loss: 1.467295 Tokens per Sec: 6595, Lr: 0.000300\n", "2020-02-13 14:49:31,121 Epoch 3 Step: 20200 Batch Loss: 1.535228 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 14:50:08,066 Epoch 3 Step: 20300 Batch Loss: 1.470028 Tokens per Sec: 6629, Lr: 0.000300\n", "2020-02-13 14:50:44,930 Epoch 3 Step: 20400 Batch Loss: 1.568730 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 14:51:22,060 Epoch 3 Step: 20500 Batch Loss: 1.364294 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 14:51:58,904 Epoch 3 Step: 20600 Batch Loss: 1.437001 Tokens per Sec: 6517, Lr: 0.000300\n", "2020-02-13 14:52:35,686 Epoch 3 Step: 20700 Batch Loss: 1.397117 Tokens per Sec: 6593, Lr: 0.000300\n", "2020-02-13 14:53:12,574 Epoch 3 Step: 20800 Batch Loss: 1.408593 Tokens per Sec: 6681, Lr: 0.000300\n", "2020-02-13 14:53:49,606 Epoch 3 Step: 20900 Batch Loss: 1.501876 Tokens per Sec: 6749, Lr: 0.000300\n", "2020-02-13 14:54:26,560 Epoch 3 Step: 21000 Batch Loss: 1.556690 Tokens per Sec: 6646, Lr: 0.000300\n", "2020-02-13 14:55:06,301 Hooray! New best validation result [ppl]!\n", "2020-02-13 14:55:06,301 Saving new checkpoint.\n", "2020-02-13 14:55:06,721 Example #0\n", "2020-02-13 14:55:06,722 \tSource: Has the situation improved since then ?\n", "2020-02-13 14:55:06,722 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 14:55:06,722 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 14:55:06,722 Example #1\n", "2020-02-13 14:55:06,722 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 14:55:06,722 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 14:55:06,722 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 14:55:06,722 Example #2\n", "2020-02-13 14:55:06,722 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 14:55:06,723 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 14:55:06,723 \tHypothesis: Monna ea bohlale o ne a kōpile kōpo e ’ ngoe : Motho e mong le e mong o ile a bolelloa hore a ee ’ me a nke letšoao le hlabang le le tlaase ’ me a le pshatla , a jala masiba a moea .\n", "2020-02-13 14:55:06,723 Example #3\n", "2020-02-13 14:55:06,723 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 14:55:06,723 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 14:55:06,723 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 14:55:06,723 Validation result (greedy) at epoch 3, step 21000: bleu: 38.44, loss: 34824.5703, ppl: 3.9257, duration: 40.1623s\n", "2020-02-13 14:55:43,595 Epoch 3 Step: 21100 Batch Loss: 2.878551 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 14:56:20,522 Epoch 3 Step: 21200 Batch Loss: 1.393510 Tokens per Sec: 6514, Lr: 0.000300\n", "2020-02-13 14:56:57,346 Epoch 3 Step: 21300 Batch Loss: 1.404393 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 14:57:34,657 Epoch 3 Step: 21400 Batch Loss: 1.522915 Tokens per Sec: 6703, Lr: 0.000300\n", "2020-02-13 14:58:11,788 Epoch 3 Step: 21500 Batch Loss: 1.541530 Tokens per Sec: 6568, Lr: 0.000300\n", "2020-02-13 14:58:48,613 Epoch 3 Step: 21600 Batch Loss: 1.442554 Tokens per Sec: 6623, Lr: 0.000300\n", "2020-02-13 14:59:25,599 Epoch 3 Step: 21700 Batch Loss: 1.699664 Tokens per Sec: 6609, Lr: 0.000300\n", "2020-02-13 15:00:01,760 Epoch 3 Step: 21800 Batch Loss: 1.489633 Tokens per Sec: 6521, Lr: 0.000300\n", "2020-02-13 15:00:38,683 Epoch 3 Step: 21900 Batch Loss: 1.736284 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 15:01:15,227 Epoch 3 Step: 22000 Batch Loss: 1.774607 Tokens per Sec: 6647, Lr: 0.000300\n", "2020-02-13 15:01:53,802 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:01:53,803 Saving new checkpoint.\n", "2020-02-13 15:01:54,228 Example #0\n", "2020-02-13 15:01:54,228 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:01:54,228 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:01:54,228 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:01:54,229 Example #1\n", "2020-02-13 15:01:54,229 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:01:54,229 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:01:54,229 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:01:54,229 Example #2\n", "2020-02-13 15:01:54,229 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:01:54,229 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:01:54,230 \tHypothesis: Monna ea bohlale o ne a kōpa e mong hore a re : Ho bua ka motho e mong ho ne ho bolelloa hore a ee ’ me a nke letšoao le tlaase ’ me a le khaola , a e hasa ka masiba a moea .\n", "2020-02-13 15:01:54,230 Example #3\n", "2020-02-13 15:01:54,230 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:01:54,230 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:01:54,230 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngata tse ngotsoeng ka letsoho li tla leseling .\n", "2020-02-13 15:01:54,232 Validation result (greedy) at epoch 3, step 22000: bleu: 38.65, loss: 34300.1523, ppl: 3.8457, duration: 39.0044s\n", "2020-02-13 15:02:31,315 Epoch 3 Step: 22100 Batch Loss: 1.426454 Tokens per Sec: 6687, Lr: 0.000300\n", "2020-02-13 15:03:07,779 Epoch 3 Step: 22200 Batch Loss: 1.517818 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 15:03:44,566 Epoch 3 Step: 22300 Batch Loss: 1.298852 Tokens per Sec: 6592, Lr: 0.000300\n", "2020-02-13 15:04:21,178 Epoch 3 Step: 22400 Batch Loss: 1.496734 Tokens per Sec: 6613, Lr: 0.000300\n", "2020-02-13 15:04:58,474 Epoch 3 Step: 22500 Batch Loss: 1.522785 Tokens per Sec: 6553, Lr: 0.000300\n", "2020-02-13 15:05:35,358 Epoch 3 Step: 22600 Batch Loss: 1.663153 Tokens per Sec: 6475, Lr: 0.000300\n", "2020-02-13 15:06:12,512 Epoch 3 Step: 22700 Batch Loss: 1.527857 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 15:06:48,780 Epoch 3 Step: 22800 Batch Loss: 1.643510 Tokens per Sec: 6622, Lr: 0.000300\n", "2020-02-13 15:07:26,045 Epoch 3 Step: 22900 Batch Loss: 1.548475 Tokens per Sec: 6623, Lr: 0.000300\n", "2020-02-13 15:08:02,728 Epoch 3 Step: 23000 Batch Loss: 1.591781 Tokens per Sec: 6665, Lr: 0.000300\n", "2020-02-13 15:08:40,473 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:08:40,473 Saving new checkpoint.\n", "2020-02-13 15:08:40,915 Example #0\n", "2020-02-13 15:08:40,915 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:08:40,915 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:08:40,915 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:08:40,916 Example #1\n", "2020-02-13 15:08:40,916 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:08:40,916 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:08:40,916 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:08:40,916 Example #2\n", "2020-02-13 15:08:40,916 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:08:40,916 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:08:40,916 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea buang ka motho o ile a bolelloa hore a ee ’ me a nke motšoantšisi oa mofuta o monyenyane ’ me a o khaola , a e hasa masiba a moea .\n", "2020-02-13 15:08:40,916 Example #3\n", "2020-02-13 15:08:40,916 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:08:40,916 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:08:40,917 \tHypothesis: Kaha nako ea ho fumana libuka tse ngotsoeng ka letsoho e ’ nile ea tla leseling .\n", "2020-02-13 15:08:40,917 Validation result (greedy) at epoch 3, step 23000: bleu: 39.26, loss: 33841.7539, ppl: 3.7771, duration: 38.1889s\n", "2020-02-13 15:09:17,840 Epoch 3 Step: 23100 Batch Loss: 1.418258 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 15:09:54,894 Epoch 3 Step: 23200 Batch Loss: 1.375112 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 15:10:32,005 Epoch 3 Step: 23300 Batch Loss: 1.381294 Tokens per Sec: 6647, Lr: 0.000300\n", "2020-02-13 15:11:09,052 Epoch 3 Step: 23400 Batch Loss: 1.671530 Tokens per Sec: 6719, Lr: 0.000300\n", "2020-02-13 15:11:46,258 Epoch 3 Step: 23500 Batch Loss: 1.733262 Tokens per Sec: 6592, Lr: 0.000300\n", "2020-02-13 15:12:23,305 Epoch 3 Step: 23600 Batch Loss: 1.396831 Tokens per Sec: 6593, Lr: 0.000300\n", "2020-02-13 15:13:00,097 Epoch 3 Step: 23700 Batch Loss: 1.432893 Tokens per Sec: 6598, Lr: 0.000300\n", "2020-02-13 15:13:36,578 Epoch 3 Step: 23800 Batch Loss: 1.571227 Tokens per Sec: 6598, Lr: 0.000300\n", "2020-02-13 15:14:13,536 Epoch 3 Step: 23900 Batch Loss: 1.549031 Tokens per Sec: 6566, Lr: 0.000300\n", "2020-02-13 15:14:50,204 Epoch 3 Step: 24000 Batch Loss: 1.469479 Tokens per Sec: 6542, Lr: 0.000300\n", "2020-02-13 15:15:28,401 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:15:28,401 Saving new checkpoint.\n", "2020-02-13 15:15:28,841 Example #0\n", "2020-02-13 15:15:28,841 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:15:28,841 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:15:28,841 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:15:28,841 Example #1\n", "2020-02-13 15:15:28,841 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:15:28,842 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:15:28,842 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:15:28,842 Example #2\n", "2020-02-13 15:15:28,842 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:15:28,842 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:15:28,842 \tHypothesis: Monna ea bohlale o ne a kōpile e mong : Motho ea neng a kōpile hore a ee ’ me a nke motšoantši o tlaase ’ me a o rema , a jala masiba a moea .\n", "2020-02-13 15:15:28,842 Example #3\n", "2020-02-13 15:15:28,842 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:15:28,842 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:15:28,842 \tHypothesis: Kaha nako ea ho qetela , libuka tse ngotsoeng ka letsoho li tla leseling .\n", "2020-02-13 15:15:28,842 Validation result (greedy) at epoch 3, step 24000: bleu: 39.16, loss: 33616.3906, ppl: 3.7438, duration: 38.6381s\n", "2020-02-13 15:16:05,312 Epoch 3 Step: 24100 Batch Loss: 1.362064 Tokens per Sec: 6530, Lr: 0.000300\n", "2020-02-13 15:16:42,487 Epoch 3 Step: 24200 Batch Loss: 1.497856 Tokens per Sec: 6633, Lr: 0.000300\n", "2020-02-13 15:17:18,995 Epoch 3 Step: 24300 Batch Loss: 1.382743 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 15:17:55,581 Epoch 3 Step: 24400 Batch Loss: 1.338664 Tokens per Sec: 6667, Lr: 0.000300\n", "2020-02-13 15:18:32,347 Epoch 3 Step: 24500 Batch Loss: 1.652661 Tokens per Sec: 6541, Lr: 0.000300\n", "2020-02-13 15:19:09,374 Epoch 3 Step: 24600 Batch Loss: 1.576221 Tokens per Sec: 6610, Lr: 0.000300\n", "2020-02-13 15:19:46,493 Epoch 3 Step: 24700 Batch Loss: 1.466167 Tokens per Sec: 6726, Lr: 0.000300\n", "2020-02-13 15:20:23,499 Epoch 3 Step: 24800 Batch Loss: 1.386804 Tokens per Sec: 6550, Lr: 0.000300\n", "2020-02-13 15:21:00,370 Epoch 3 Step: 24900 Batch Loss: 1.423576 Tokens per Sec: 6633, Lr: 0.000300\n", "2020-02-13 15:21:37,197 Epoch 3 Step: 25000 Batch Loss: 1.576664 Tokens per Sec: 6611, Lr: 0.000300\n", "2020-02-13 15:22:12,749 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:22:12,749 Saving new checkpoint.\n", "2020-02-13 15:22:13,224 Example #0\n", "2020-02-13 15:22:13,225 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:22:13,225 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:22:13,225 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:22:13,225 Example #1\n", "2020-02-13 15:22:13,225 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:22:13,225 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:22:13,225 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:22:13,225 Example #2\n", "2020-02-13 15:22:13,225 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:22:13,225 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:22:13,225 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea neng a boleletsoe hore a ee ’ me a nke motšoantšisi oa mophula o tlaase ’ me a e khaola , a jala masiba a moea .\n", "2020-02-13 15:22:13,225 Example #3\n", "2020-02-13 15:22:13,225 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:22:13,226 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:22:13,226 \tHypothesis: Kaha nako ea tšepe , libuka tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 15:22:13,226 Validation result (greedy) at epoch 3, step 25000: bleu: 39.16, loss: 33239.6133, ppl: 3.6888, duration: 36.0283s\n", "2020-02-13 15:22:49,936 Epoch 3 Step: 25100 Batch Loss: 1.519925 Tokens per Sec: 6659, Lr: 0.000300\n", "2020-02-13 15:23:26,571 Epoch 3 Step: 25200 Batch Loss: 1.632833 Tokens per Sec: 6583, Lr: 0.000300\n", "2020-02-13 15:24:03,614 Epoch 3 Step: 25300 Batch Loss: 1.390823 Tokens per Sec: 6579, Lr: 0.000300\n", "2020-02-13 15:24:40,468 Epoch 3 Step: 25400 Batch Loss: 1.368707 Tokens per Sec: 6576, Lr: 0.000300\n", "2020-02-13 15:25:17,201 Epoch 3 Step: 25500 Batch Loss: 1.407576 Tokens per Sec: 6569, Lr: 0.000300\n", "2020-02-13 15:25:54,238 Epoch 3 Step: 25600 Batch Loss: 1.500600 Tokens per Sec: 6670, Lr: 0.000300\n", "2020-02-13 15:26:30,969 Epoch 3 Step: 25700 Batch Loss: 1.344629 Tokens per Sec: 6720, Lr: 0.000300\n", "2020-02-13 15:27:07,551 Epoch 3 Step: 25800 Batch Loss: 1.488107 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 15:27:44,686 Epoch 3 Step: 25900 Batch Loss: 1.396235 Tokens per Sec: 6642, Lr: 0.000300\n", "2020-02-13 15:28:21,684 Epoch 3 Step: 26000 Batch Loss: 1.451662 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 15:29:01,301 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:29:01,301 Saving new checkpoint.\n", "2020-02-13 15:29:01,774 Example #0\n", "2020-02-13 15:29:01,774 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:29:01,774 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:29:01,774 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:29:01,774 Example #1\n", "2020-02-13 15:29:01,775 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:29:01,775 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:29:01,775 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:29:01,775 Example #2\n", "2020-02-13 15:29:01,775 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:29:01,775 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:29:01,775 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Ho boleloa hore motho ea betang o ne a lokela ho ea ’ me a nke letšoao le lenyenyane ’ me a le tlosa , a hasanya masiba a ho ba moea .\n", "2020-02-13 15:29:01,775 Example #3\n", "2020-02-13 15:29:01,775 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:29:01,775 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:29:01,775 \tHypothesis: Kaha nako ea tšepe , libuka tse ngotsoeng ka letsoho tse eketsehileng li tla leseling .\n", "2020-02-13 15:29:01,776 Validation result (greedy) at epoch 3, step 26000: bleu: 39.97, loss: 32730.2520, ppl: 3.6158, duration: 40.0915s\n", "2020-02-13 15:29:38,880 Epoch 3 Step: 26100 Batch Loss: 1.459555 Tokens per Sec: 6532, Lr: 0.000300\n", "2020-02-13 15:29:54,829 Epoch 3: total training loss 13311.15\n", "2020-02-13 15:29:54,829 EPOCH 4\n", "2020-02-13 15:30:16,882 Epoch 4 Step: 26200 Batch Loss: 1.448043 Tokens per Sec: 6203, Lr: 0.000300\n", "2020-02-13 15:30:53,573 Epoch 4 Step: 26300 Batch Loss: 1.340848 Tokens per Sec: 6609, Lr: 0.000300\n", "2020-02-13 15:31:30,770 Epoch 4 Step: 26400 Batch Loss: 1.555186 Tokens per Sec: 6604, Lr: 0.000300\n", "2020-02-13 15:32:07,605 Epoch 4 Step: 26500 Batch Loss: 1.340090 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 15:32:44,388 Epoch 4 Step: 26600 Batch Loss: 1.655248 Tokens per Sec: 6597, Lr: 0.000300\n", "2020-02-13 15:33:20,751 Epoch 4 Step: 26700 Batch Loss: 1.456588 Tokens per Sec: 6600, Lr: 0.000300\n", "2020-02-13 15:33:58,035 Epoch 4 Step: 26800 Batch Loss: 1.580262 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 15:34:35,037 Epoch 4 Step: 26900 Batch Loss: 1.374988 Tokens per Sec: 6593, Lr: 0.000300\n", "2020-02-13 15:35:11,614 Epoch 4 Step: 27000 Batch Loss: 1.265105 Tokens per Sec: 6553, Lr: 0.000300\n", "2020-02-13 15:35:48,268 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:35:48,268 Saving new checkpoint.\n", "2020-02-13 15:35:48,703 Example #0\n", "2020-02-13 15:35:48,704 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:35:48,704 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:35:48,704 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:35:48,704 Example #1\n", "2020-02-13 15:35:48,704 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:35:48,704 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:35:48,704 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:35:48,704 Example #2\n", "2020-02-13 15:35:48,704 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:35:48,705 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:35:48,705 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea bitsoang sebi o ile a bolelloa hore a ee ’ me a nke letšoao le kang la motšoaea - phio ’ me a le khaola , a le hasa masiba a moea .\n", "2020-02-13 15:35:48,705 Example #3\n", "2020-02-13 15:35:48,705 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:35:48,705 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:35:48,705 \tHypothesis: Kaha nako ea tšepe , libuka tse ngotsoeng ka letsoho li na le leseli .\n", "2020-02-13 15:35:48,705 Validation result (greedy) at epoch 4, step 27000: bleu: 39.62, loss: 32588.4609, ppl: 3.5957, duration: 37.0910s\n", "2020-02-13 15:36:25,814 Epoch 4 Step: 27100 Batch Loss: 1.340948 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 15:37:02,802 Epoch 4 Step: 27200 Batch Loss: 1.401423 Tokens per Sec: 6603, Lr: 0.000300\n", "2020-02-13 15:37:39,768 Epoch 4 Step: 27300 Batch Loss: 1.343436 Tokens per Sec: 6690, Lr: 0.000300\n", "2020-02-13 15:38:16,522 Epoch 4 Step: 27400 Batch Loss: 1.359252 Tokens per Sec: 6550, Lr: 0.000300\n", "2020-02-13 15:38:53,425 Epoch 4 Step: 27500 Batch Loss: 1.368717 Tokens per Sec: 6777, Lr: 0.000300\n", "2020-02-13 15:39:29,731 Epoch 4 Step: 27600 Batch Loss: 1.546735 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 15:40:06,745 Epoch 4 Step: 27700 Batch Loss: 1.357090 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 15:40:43,602 Epoch 4 Step: 27800 Batch Loss: 1.266706 Tokens per Sec: 6603, Lr: 0.000300\n", "2020-02-13 15:41:20,539 Epoch 4 Step: 27900 Batch Loss: 1.336410 Tokens per Sec: 6577, Lr: 0.000300\n", "2020-02-13 15:41:57,510 Epoch 4 Step: 28000 Batch Loss: 1.446322 Tokens per Sec: 6611, Lr: 0.000300\n", "2020-02-13 15:42:35,220 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:42:35,220 Saving new checkpoint.\n", "2020-02-13 15:42:35,735 Example #0\n", "2020-02-13 15:42:35,735 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:42:35,735 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:42:35,735 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:42:35,735 Example #1\n", "2020-02-13 15:42:35,736 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:42:35,736 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:42:35,736 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:42:35,736 Example #2\n", "2020-02-13 15:42:35,736 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:42:35,736 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:42:35,736 \tHypothesis: Monna ea bohlale o ne a kōpa : Motho ea mosa o ile a bolelloa hore a ee ’ me a nke letšoao le motsu o tlaase ’ me a le khaola , a jala masiba a moea .\n", "2020-02-13 15:42:35,736 Example #3\n", "2020-02-13 15:42:35,736 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:42:35,736 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:42:35,736 \tHypothesis: Kaha nako ea tšepe , libuka tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 15:42:35,736 Validation result (greedy) at epoch 4, step 28000: bleu: 39.99, loss: 32438.3164, ppl: 3.5746, duration: 38.2259s\n", "2020-02-13 15:43:12,393 Epoch 4 Step: 28100 Batch Loss: 1.601355 Tokens per Sec: 6559, Lr: 0.000300\n", "2020-02-13 15:43:48,917 Epoch 4 Step: 28200 Batch Loss: 1.427941 Tokens per Sec: 6549, Lr: 0.000300\n", "2020-02-13 15:44:25,637 Epoch 4 Step: 28300 Batch Loss: 1.341864 Tokens per Sec: 6701, Lr: 0.000300\n", "2020-02-13 15:45:02,732 Epoch 4 Step: 28400 Batch Loss: 1.523543 Tokens per Sec: 6559, Lr: 0.000300\n", "2020-02-13 15:45:39,434 Epoch 4 Step: 28500 Batch Loss: 1.350944 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 15:46:15,989 Epoch 4 Step: 28600 Batch Loss: 1.338097 Tokens per Sec: 6445, Lr: 0.000300\n", "2020-02-13 15:46:52,899 Epoch 4 Step: 28700 Batch Loss: 1.424861 Tokens per Sec: 6595, Lr: 0.000300\n", "2020-02-13 15:47:29,850 Epoch 4 Step: 28800 Batch Loss: 1.412049 Tokens per Sec: 6649, Lr: 0.000300\n", "2020-02-13 15:48:06,556 Epoch 4 Step: 28900 Batch Loss: 1.507148 Tokens per Sec: 6536, Lr: 0.000300\n", "2020-02-13 15:48:43,350 Epoch 4 Step: 29000 Batch Loss: 1.253726 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 15:49:20,167 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:49:20,168 Saving new checkpoint.\n", "2020-02-13 15:49:20,598 Example #0\n", "2020-02-13 15:49:20,598 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:49:20,599 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:49:20,599 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:49:20,599 Example #1\n", "2020-02-13 15:49:20,599 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:49:20,599 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:49:20,599 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:49:20,599 Example #2\n", "2020-02-13 15:49:20,599 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:49:20,599 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:49:20,600 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea bitsoang ea bitsoang ea bitsoang ea bitsoang ea neng a il’o ea ’ me a e nka e le motšoaea - phio ’ me a e khaola , a hasanya masiba a moea .\n", "2020-02-13 15:49:20,600 Example #3\n", "2020-02-13 15:49:20,600 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:49:20,600 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:49:20,600 \tHypothesis: Kaha ho tloha ka 1961 , libuka tse ngotsoeng ka letsoho tse ngotsoeng ka letsoho li na le leseli .\n", "2020-02-13 15:49:20,600 Validation result (greedy) at epoch 4, step 29000: bleu: 40.06, loss: 32244.1445, ppl: 3.5474, duration: 37.2505s\n", "2020-02-13 15:49:57,301 Epoch 4 Step: 29100 Batch Loss: 1.461745 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 15:50:34,100 Epoch 4 Step: 29200 Batch Loss: 1.513760 Tokens per Sec: 6642, Lr: 0.000300\n", "2020-02-13 15:51:11,013 Epoch 4 Step: 29300 Batch Loss: 1.279716 Tokens per Sec: 6610, Lr: 0.000300\n", "2020-02-13 15:51:47,942 Epoch 4 Step: 29400 Batch Loss: 1.606281 Tokens per Sec: 6550, Lr: 0.000300\n", "2020-02-13 15:52:24,866 Epoch 4 Step: 29500 Batch Loss: 1.596232 Tokens per Sec: 6654, Lr: 0.000300\n", "2020-02-13 15:53:01,901 Epoch 4 Step: 29600 Batch Loss: 1.330625 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 15:53:39,014 Epoch 4 Step: 29700 Batch Loss: 1.371774 Tokens per Sec: 6563, Lr: 0.000300\n", "2020-02-13 15:54:15,666 Epoch 4 Step: 29800 Batch Loss: 1.355564 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 15:54:52,655 Epoch 4 Step: 29900 Batch Loss: 1.382659 Tokens per Sec: 6604, Lr: 0.000300\n", "2020-02-13 15:55:29,353 Epoch 4 Step: 30000 Batch Loss: 1.360955 Tokens per Sec: 6549, Lr: 0.000300\n", "2020-02-13 15:56:07,065 Hooray! New best validation result [ppl]!\n", "2020-02-13 15:56:07,065 Saving new checkpoint.\n", "2020-02-13 15:56:07,528 Example #0\n", "2020-02-13 15:56:07,529 \tSource: Has the situation improved since then ?\n", "2020-02-13 15:56:07,529 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 15:56:07,529 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 15:56:07,529 Example #1\n", "2020-02-13 15:56:07,529 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 15:56:07,529 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 15:56:07,529 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 15:56:07,529 Example #2\n", "2020-02-13 15:56:07,532 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 15:56:07,532 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 15:56:07,532 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Motho ea bitsoang moetselletsi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 15:56:07,532 Example #3\n", "2020-02-13 15:56:07,532 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 15:56:07,532 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 15:56:07,532 \tHypothesis: Kaha nako ea Agricola e ne e le teng , libuka tse ngata tse ngotsoeng ka letsoho li na le leseli .\n", "2020-02-13 15:56:07,532 Validation result (greedy) at epoch 4, step 30000: bleu: 40.69, loss: 31913.4062, ppl: 3.5016, duration: 38.1793s\n", "2020-02-13 15:56:44,145 Epoch 4 Step: 30100 Batch Loss: 1.690808 Tokens per Sec: 6642, Lr: 0.000300\n", "2020-02-13 15:57:21,023 Epoch 4 Step: 30200 Batch Loss: 1.334069 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 15:57:57,425 Epoch 4 Step: 30300 Batch Loss: 1.370748 Tokens per Sec: 6660, Lr: 0.000300\n", "2020-02-13 15:58:34,222 Epoch 4 Step: 30400 Batch Loss: 1.479780 Tokens per Sec: 6544, Lr: 0.000300\n", "2020-02-13 15:59:10,759 Epoch 4 Step: 30500 Batch Loss: 1.364804 Tokens per Sec: 6695, Lr: 0.000300\n", "2020-02-13 15:59:47,255 Epoch 4 Step: 30600 Batch Loss: 1.503545 Tokens per Sec: 6699, Lr: 0.000300\n", "2020-02-13 16:00:23,727 Epoch 4 Step: 30700 Batch Loss: 1.397789 Tokens per Sec: 6720, Lr: 0.000300\n", "2020-02-13 16:01:00,086 Epoch 4 Step: 30800 Batch Loss: 1.382577 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 16:01:37,174 Epoch 4 Step: 30900 Batch Loss: 1.372447 Tokens per Sec: 6582, Lr: 0.000300\n", "2020-02-13 16:02:13,375 Epoch 4 Step: 31000 Batch Loss: 1.623108 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 16:02:53,441 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:02:53,441 Saving new checkpoint.\n", "2020-02-13 16:02:53,940 Example #0\n", "2020-02-13 16:02:53,941 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:02:53,941 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:02:53,941 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:02:53,941 Example #1\n", "2020-02-13 16:02:53,941 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:02:53,941 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:02:53,941 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:02:53,941 Example #2\n", "2020-02-13 16:02:53,941 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:02:53,941 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:02:53,941 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea bitsoang ea bitsoang ea bitsoang mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a e khaola , a jala masiba a moea .\n", "2020-02-13 16:02:53,941 Example #3\n", "2020-02-13 16:02:53,942 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:02:53,942 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:02:53,942 \tHypothesis: Kaha nako ea Estienne , libuka tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 16:02:53,942 Validation result (greedy) at epoch 4, step 31000: bleu: 41.36, loss: 31599.1465, ppl: 3.4587, duration: 40.5667s\n", "2020-02-13 16:03:30,478 Epoch 4 Step: 31100 Batch Loss: 1.178223 Tokens per Sec: 6694, Lr: 0.000300\n", "2020-02-13 16:04:06,928 Epoch 4 Step: 31200 Batch Loss: 1.353667 Tokens per Sec: 6612, Lr: 0.000300\n", "2020-02-13 16:04:43,417 Epoch 4 Step: 31300 Batch Loss: 1.235416 Tokens per Sec: 6552, Lr: 0.000300\n", "2020-02-13 16:05:20,032 Epoch 4 Step: 31400 Batch Loss: 1.338271 Tokens per Sec: 6662, Lr: 0.000300\n", "2020-02-13 16:05:56,589 Epoch 4 Step: 31500 Batch Loss: 1.477149 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 16:06:33,287 Epoch 4 Step: 31600 Batch Loss: 1.371583 Tokens per Sec: 6594, Lr: 0.000300\n", "2020-02-13 16:07:09,535 Epoch 4 Step: 31700 Batch Loss: 1.368939 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 16:07:46,209 Epoch 4 Step: 31800 Batch Loss: 1.402460 Tokens per Sec: 6767, Lr: 0.000300\n", "2020-02-13 16:08:22,823 Epoch 4 Step: 31900 Batch Loss: 1.562478 Tokens per Sec: 6730, Lr: 0.000300\n", "2020-02-13 16:08:59,404 Epoch 4 Step: 32000 Batch Loss: 1.407115 Tokens per Sec: 6551, Lr: 0.000300\n", "2020-02-13 16:09:37,586 Example #0\n", "2020-02-13 16:09:37,586 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:09:37,586 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:09:37,586 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:09:37,586 Example #1\n", "2020-02-13 16:09:37,586 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:09:37,587 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:09:37,587 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:09:37,587 Example #2\n", "2020-02-13 16:09:37,587 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:09:37,587 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:09:37,587 \tHypothesis: Monna ea bohlale o ne a kōpa : Motho ea betang o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a e hasanya masiba a moea .\n", "2020-02-13 16:09:37,587 Example #3\n", "2020-02-13 16:09:37,587 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:09:37,587 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:09:37,587 \tHypothesis: Kaha nako ea Masada , libuka tse ngotsoeng ka letsoho tse ngotsoeng ka letsoho li tla leseling .\n", "2020-02-13 16:09:37,587 Validation result (greedy) at epoch 4, step 32000: bleu: 40.89, loss: 31655.8398, ppl: 3.4664, duration: 38.1832s\n", "2020-02-13 16:10:13,994 Epoch 4 Step: 32100 Batch Loss: 1.613258 Tokens per Sec: 6693, Lr: 0.000300\n", "2020-02-13 16:10:50,889 Epoch 4 Step: 32200 Batch Loss: 1.475165 Tokens per Sec: 6614, Lr: 0.000300\n", "2020-02-13 16:11:27,125 Epoch 4 Step: 32300 Batch Loss: 1.469302 Tokens per Sec: 6524, Lr: 0.000300\n", "2020-02-13 16:12:03,926 Epoch 4 Step: 32400 Batch Loss: 1.292200 Tokens per Sec: 6724, Lr: 0.000300\n", "2020-02-13 16:12:40,312 Epoch 4 Step: 32500 Batch Loss: 1.253929 Tokens per Sec: 6570, Lr: 0.000300\n", "2020-02-13 16:13:16,893 Epoch 4 Step: 32600 Batch Loss: 1.490537 Tokens per Sec: 6764, Lr: 0.000300\n", "2020-02-13 16:13:53,645 Epoch 4 Step: 32700 Batch Loss: 1.274607 Tokens per Sec: 6527, Lr: 0.000300\n", "2020-02-13 16:14:30,142 Epoch 4 Step: 32800 Batch Loss: 1.501202 Tokens per Sec: 6771, Lr: 0.000300\n", "2020-02-13 16:15:07,032 Epoch 4 Step: 32900 Batch Loss: 1.377454 Tokens per Sec: 6667, Lr: 0.000300\n", "2020-02-13 16:15:43,865 Epoch 4 Step: 33000 Batch Loss: 1.373655 Tokens per Sec: 6686, Lr: 0.000300\n", "2020-02-13 16:16:18,720 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:16:18,720 Saving new checkpoint.\n", "2020-02-13 16:16:19,298 Example #0\n", "2020-02-13 16:16:19,298 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:16:19,298 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:16:19,298 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:16:19,298 Example #1\n", "2020-02-13 16:16:19,299 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:16:19,299 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:16:19,299 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:16:19,299 Example #2\n", "2020-02-13 16:16:19,299 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:16:19,299 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:16:19,299 \tHypothesis: Monna ea bohlale o ne a kōpa e mong ka ’ ngoe : Motho ea neng a bolelloa hore a ee ’ me a nke mosamo o mosehla ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 16:16:19,299 Example #3\n", "2020-02-13 16:16:19,299 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:16:19,299 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:16:19,300 \tHypothesis: Kaha nako ea ho phatloha , libuka tse ngata tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 16:16:19,300 Validation result (greedy) at epoch 4, step 33000: bleu: 41.50, loss: 31182.2949, ppl: 3.4025, duration: 35.4342s\n", "2020-02-13 16:16:55,573 Epoch 4 Step: 33100 Batch Loss: 1.250980 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 16:17:32,245 Epoch 4 Step: 33200 Batch Loss: 1.337412 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 16:18:09,010 Epoch 4 Step: 33300 Batch Loss: 1.294082 Tokens per Sec: 6651, Lr: 0.000300\n", "2020-02-13 16:18:45,150 Epoch 4 Step: 33400 Batch Loss: 1.291791 Tokens per Sec: 6512, Lr: 0.000300\n", "2020-02-13 16:19:21,860 Epoch 4 Step: 33500 Batch Loss: 1.335221 Tokens per Sec: 6563, Lr: 0.000300\n", "2020-02-13 16:19:58,469 Epoch 4 Step: 33600 Batch Loss: 1.303070 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 16:20:35,215 Epoch 4 Step: 33700 Batch Loss: 1.219890 Tokens per Sec: 6603, Lr: 0.000300\n", "2020-02-13 16:21:11,689 Epoch 4 Step: 33800 Batch Loss: 1.424016 Tokens per Sec: 6695, Lr: 0.000300\n", "2020-02-13 16:21:47,883 Epoch 4 Step: 33900 Batch Loss: 1.410785 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 16:22:24,027 Epoch 4 Step: 34000 Batch Loss: 1.426698 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 16:23:02,683 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:23:02,683 Saving new checkpoint.\n", "2020-02-13 16:23:03,256 Example #0\n", "2020-02-13 16:23:03,257 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:23:03,257 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:23:03,257 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:23:03,257 Example #1\n", "2020-02-13 16:23:03,257 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:23:03,257 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:23:03,257 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:23:03,257 Example #2\n", "2020-02-13 16:23:03,257 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:23:03,257 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:23:03,258 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea sebetsoang o ile a bolelloa hore a ee ’ me a nke mosamo o motšoantšisi o monyenyane ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 16:23:03,258 Example #3\n", "2020-02-13 16:23:03,258 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:23:03,258 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:23:03,258 \tHypothesis: Kaha nako ea eona e ne e le ngata , libuka tse ngotsoeng ka letsoho tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 16:23:03,258 Validation result (greedy) at epoch 4, step 34000: bleu: 41.51, loss: 31066.6387, ppl: 3.3871, duration: 39.2305s\n", "2020-02-13 16:23:39,755 Epoch 4 Step: 34100 Batch Loss: 1.629679 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 16:24:16,475 Epoch 4 Step: 34200 Batch Loss: 1.464343 Tokens per Sec: 6623, Lr: 0.000300\n", "2020-02-13 16:24:53,075 Epoch 4 Step: 34300 Batch Loss: 1.451003 Tokens per Sec: 6596, Lr: 0.000300\n", "2020-02-13 16:25:29,794 Epoch 4 Step: 34400 Batch Loss: 1.450057 Tokens per Sec: 6756, Lr: 0.000300\n", "2020-02-13 16:26:06,342 Epoch 4 Step: 34500 Batch Loss: 1.391591 Tokens per Sec: 6681, Lr: 0.000300\n", "2020-02-13 16:26:42,875 Epoch 4 Step: 34600 Batch Loss: 1.583209 Tokens per Sec: 6679, Lr: 0.000300\n", "2020-02-13 16:27:19,809 Epoch 4 Step: 34700 Batch Loss: 1.474271 Tokens per Sec: 6706, Lr: 0.000300\n", "2020-02-13 16:27:56,115 Epoch 4 Step: 34800 Batch Loss: 1.292678 Tokens per Sec: 6668, Lr: 0.000300\n", "2020-02-13 16:28:23,521 Epoch 4: total training loss 12247.75\n", "2020-02-13 16:28:23,522 EPOCH 5\n", "2020-02-13 16:28:33,628 Epoch 5 Step: 34900 Batch Loss: 1.206868 Tokens per Sec: 5948, Lr: 0.000300\n", "2020-02-13 16:29:10,233 Epoch 5 Step: 35000 Batch Loss: 1.407616 Tokens per Sec: 6538, Lr: 0.000300\n", "2020-02-13 16:29:47,286 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:29:47,286 Saving new checkpoint.\n", "2020-02-13 16:29:47,771 Example #0\n", "2020-02-13 16:29:47,771 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:29:47,772 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:29:47,772 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:29:47,772 Example #1\n", "2020-02-13 16:29:47,772 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:29:47,772 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:29:47,772 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:29:47,772 Example #2\n", "2020-02-13 16:29:47,772 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:29:47,772 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:29:47,772 \tHypothesis: Monna ea bohlale o ne a kōpile e mong : Monna e mong le e mong o ile a bolelloa hore a ee ’ me a nke mosamo o mosehla ’ me a o khaola , a hasa masiba a moea .\n", "2020-02-13 16:29:47,772 Example #3\n", "2020-02-13 16:29:47,773 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:29:47,773 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:29:47,773 \tHypothesis: Kaha nako ea balsame , libuka tse ngotsoeng ka letsoho tse ngata li fihlile leseling .\n", "2020-02-13 16:29:47,773 Validation result (greedy) at epoch 5, step 35000: bleu: 41.58, loss: 30806.5254, ppl: 3.3527, duration: 37.5392s\n", "2020-02-13 16:30:24,523 Epoch 5 Step: 35100 Batch Loss: 1.306809 Tokens per Sec: 6732, Lr: 0.000300\n", "2020-02-13 16:31:01,185 Epoch 5 Step: 35200 Batch Loss: 1.331730 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 16:31:37,838 Epoch 5 Step: 35300 Batch Loss: 1.264116 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 16:32:14,753 Epoch 5 Step: 35400 Batch Loss: 1.297029 Tokens per Sec: 6845, Lr: 0.000300\n", "2020-02-13 16:32:51,226 Epoch 5 Step: 35500 Batch Loss: 1.481341 Tokens per Sec: 6577, Lr: 0.000300\n", "2020-02-13 16:33:27,732 Epoch 5 Step: 35600 Batch Loss: 1.343331 Tokens per Sec: 6605, Lr: 0.000300\n", "2020-02-13 16:34:04,503 Epoch 5 Step: 35700 Batch Loss: 1.332832 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 16:34:41,162 Epoch 5 Step: 35800 Batch Loss: 1.294888 Tokens per Sec: 6770, Lr: 0.000300\n", "2020-02-13 16:35:17,721 Epoch 5 Step: 35900 Batch Loss: 1.633099 Tokens per Sec: 6690, Lr: 0.000300\n", "2020-02-13 16:35:54,339 Epoch 5 Step: 36000 Batch Loss: 1.230424 Tokens per Sec: 6700, Lr: 0.000300\n", "2020-02-13 16:36:29,600 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:36:29,600 Saving new checkpoint.\n", "2020-02-13 16:36:30,107 Example #0\n", "2020-02-13 16:36:30,107 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:36:30,107 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:36:30,107 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:36:30,107 Example #1\n", "2020-02-13 16:36:30,107 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:36:30,107 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:36:30,108 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:36:30,108 Example #2\n", "2020-02-13 16:36:30,108 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:36:30,108 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:36:30,108 \tHypothesis: Monna ea bohlale o ne a kōpa : Monna e mong le e mong o ile a bolelloa hore a ee ’ me a nke mosamo o bofubelu bo loileng ’ me a o rema , a hasanya masiba a moea .\n", "2020-02-13 16:36:30,108 Example #3\n", "2020-02-13 16:36:30,108 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:36:30,108 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:36:30,108 \tHypothesis: Kaha mehleng ea Masada , libuka tse ngotsoeng ka letsoho tse ngata li na le leseli .\n", "2020-02-13 16:36:30,109 Validation result (greedy) at epoch 5, step 36000: bleu: 41.34, loss: 30705.5293, ppl: 3.3394, duration: 35.7690s\n", "2020-02-13 16:37:06,181 Epoch 5 Step: 36100 Batch Loss: 1.467797 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 16:37:42,976 Epoch 5 Step: 36200 Batch Loss: 1.341994 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 16:38:19,382 Epoch 5 Step: 36300 Batch Loss: 1.265767 Tokens per Sec: 6768, Lr: 0.000300\n", "2020-02-13 16:38:56,161 Epoch 5 Step: 36400 Batch Loss: 1.165315 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 16:39:32,289 Epoch 5 Step: 36500 Batch Loss: 1.309752 Tokens per Sec: 6580, Lr: 0.000300\n", "2020-02-13 16:40:08,789 Epoch 5 Step: 36600 Batch Loss: 1.355019 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 16:40:45,277 Epoch 5 Step: 36700 Batch Loss: 1.355340 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 16:41:22,179 Epoch 5 Step: 36800 Batch Loss: 1.180676 Tokens per Sec: 6608, Lr: 0.000300\n", "2020-02-13 16:41:58,549 Epoch 5 Step: 36900 Batch Loss: 1.357721 Tokens per Sec: 6729, Lr: 0.000300\n", "2020-02-13 16:42:35,059 Epoch 5 Step: 37000 Batch Loss: 1.249025 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-13 16:43:11,413 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:43:11,413 Saving new checkpoint.\n", "2020-02-13 16:43:11,830 Example #0\n", "2020-02-13 16:43:11,831 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:43:11,831 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:43:11,831 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:43:11,831 Example #1\n", "2020-02-13 16:43:11,831 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:43:11,831 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:43:11,831 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:43:11,831 Example #2\n", "2020-02-13 16:43:11,831 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:43:11,832 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:43:11,832 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea sebang o ile a bolelloa hore a ee ’ me a nke mosamo o mopatisane ’ me a o rotha , a hasanya masiba a moea .\n", "2020-02-13 16:43:11,832 Example #3\n", "2020-02-13 16:43:11,832 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:43:11,832 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:43:11,832 \tHypothesis: Kaha nako ea ho phatloha ha seretse se chesang , libuka tse ngata tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 16:43:11,832 Validation result (greedy) at epoch 5, step 37000: bleu: 41.69, loss: 30573.3809, ppl: 3.3221, duration: 36.7731s\n", "2020-02-13 16:43:48,670 Epoch 5 Step: 37100 Batch Loss: 1.228660 Tokens per Sec: 6609, Lr: 0.000300\n", "2020-02-13 16:44:25,249 Epoch 5 Step: 37200 Batch Loss: 1.800545 Tokens per Sec: 6764, Lr: 0.000300\n", "2020-02-13 16:45:01,758 Epoch 5 Step: 37300 Batch Loss: 1.440524 Tokens per Sec: 6731, Lr: 0.000300\n", "2020-02-13 16:45:38,321 Epoch 5 Step: 37400 Batch Loss: 1.187241 Tokens per Sec: 6660, Lr: 0.000300\n", "2020-02-13 16:46:14,801 Epoch 5 Step: 37500 Batch Loss: 1.333326 Tokens per Sec: 6635, Lr: 0.000300\n", "2020-02-13 16:46:51,430 Epoch 5 Step: 37600 Batch Loss: 1.392662 Tokens per Sec: 6595, Lr: 0.000300\n", "2020-02-13 16:47:28,125 Epoch 5 Step: 37700 Batch Loss: 1.439986 Tokens per Sec: 6675, Lr: 0.000300\n", "2020-02-13 16:48:04,822 Epoch 5 Step: 37800 Batch Loss: 1.365645 Tokens per Sec: 6558, Lr: 0.000300\n", "2020-02-13 16:48:41,197 Epoch 5 Step: 37900 Batch Loss: 1.398172 Tokens per Sec: 6746, Lr: 0.000300\n", "2020-02-13 16:49:17,692 Epoch 5 Step: 38000 Batch Loss: 1.326326 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 16:49:56,721 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:49:56,721 Saving new checkpoint.\n", "2020-02-13 16:49:57,153 Example #0\n", "2020-02-13 16:49:57,153 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:49:57,153 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:49:57,153 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 16:49:57,153 Example #1\n", "2020-02-13 16:49:57,153 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:49:57,153 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:49:57,153 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:49:57,153 Example #2\n", "2020-02-13 16:49:57,153 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:49:57,154 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:49:57,154 \tHypothesis: Monna ea bohlale o ne a kōpa : Monna enoa ea bitsoang sebsebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasa masiba a moea .\n", "2020-02-13 16:49:57,154 Example #3\n", "2020-02-13 16:49:57,154 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:49:57,154 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:49:57,154 \tHypothesis: Kaha mehleng ea Masada , libuka tse ngata tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 16:49:57,154 Validation result (greedy) at epoch 5, step 38000: bleu: 41.90, loss: 30325.6465, ppl: 3.2900, duration: 39.4620s\n", "2020-02-13 16:50:33,474 Epoch 5 Step: 38100 Batch Loss: 1.331906 Tokens per Sec: 6672, Lr: 0.000300\n", "2020-02-13 16:51:10,090 Epoch 5 Step: 38200 Batch Loss: 1.280565 Tokens per Sec: 6756, Lr: 0.000300\n", "2020-02-13 16:51:46,565 Epoch 5 Step: 38300 Batch Loss: 1.306646 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 16:52:23,032 Epoch 5 Step: 38400 Batch Loss: 1.338128 Tokens per Sec: 6677, Lr: 0.000300\n", "2020-02-13 16:52:59,578 Epoch 5 Step: 38500 Batch Loss: 1.536489 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 16:53:36,507 Epoch 5 Step: 38600 Batch Loss: 1.339111 Tokens per Sec: 6634, Lr: 0.000300\n", "2020-02-13 16:54:13,023 Epoch 5 Step: 38700 Batch Loss: 1.203691 Tokens per Sec: 6713, Lr: 0.000300\n", "2020-02-13 16:54:49,638 Epoch 5 Step: 38800 Batch Loss: 1.278776 Tokens per Sec: 6757, Lr: 0.000300\n", "2020-02-13 16:55:25,785 Epoch 5 Step: 38900 Batch Loss: 1.285872 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 16:56:02,304 Epoch 5 Step: 39000 Batch Loss: 1.230106 Tokens per Sec: 6730, Lr: 0.000300\n", "2020-02-13 16:56:40,070 Hooray! New best validation result [ppl]!\n", "2020-02-13 16:56:40,070 Saving new checkpoint.\n", "2020-02-13 16:56:40,510 Example #0\n", "2020-02-13 16:56:40,511 \tSource: Has the situation improved since then ?\n", "2020-02-13 16:56:40,511 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 16:56:40,511 \tHypothesis: Na boemo bo ile ba ntlafala ho tloha ka nako eo ?\n", "2020-02-13 16:56:40,511 Example #1\n", "2020-02-13 16:56:40,511 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 16:56:40,511 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 16:56:40,511 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 16:56:40,511 Example #2\n", "2020-02-13 16:56:40,512 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 16:56:40,512 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 16:56:40,512 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Monna ea bitsoang sebi o ile a bolelloa hore a ee ’ me a nke mosamo o bofubelu bo bosehla ’ me a o khaola , a hasa masiba a moea .\n", "2020-02-13 16:56:40,512 Example #3\n", "2020-02-13 16:56:40,512 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 16:56:40,512 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 16:56:40,512 \tHypothesis: Kaha e ne e le khale ho feta matatu , libuka tse ngata tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 16:56:40,512 Validation result (greedy) at epoch 5, step 39000: bleu: 41.69, loss: 30216.7324, ppl: 3.2759, duration: 38.2076s\n", "2020-02-13 16:57:17,100 Epoch 5 Step: 39100 Batch Loss: 1.279725 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 16:57:53,615 Epoch 5 Step: 39200 Batch Loss: 1.582153 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 16:58:29,942 Epoch 5 Step: 39300 Batch Loss: 1.182257 Tokens per Sec: 6612, Lr: 0.000300\n", "2020-02-13 16:59:06,394 Epoch 5 Step: 39400 Batch Loss: 1.242912 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 16:59:42,947 Epoch 5 Step: 39500 Batch Loss: 1.248904 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 17:00:19,769 Epoch 5 Step: 39600 Batch Loss: 1.222213 Tokens per Sec: 6603, Lr: 0.000300\n", "2020-02-13 17:00:55,693 Epoch 5 Step: 39700 Batch Loss: 1.324845 Tokens per Sec: 6435, Lr: 0.000300\n", "2020-02-13 17:01:32,433 Epoch 5 Step: 39800 Batch Loss: 1.421780 Tokens per Sec: 6712, Lr: 0.000300\n", "2020-02-13 17:02:08,696 Epoch 5 Step: 39900 Batch Loss: 1.349599 Tokens per Sec: 6647, Lr: 0.000300\n", "2020-02-13 17:02:45,711 Epoch 5 Step: 40000 Batch Loss: 1.230030 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 17:03:22,944 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:03:22,944 Saving new checkpoint.\n", "2020-02-13 17:03:23,402 Example #0\n", "2020-02-13 17:03:23,403 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:03:23,403 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:03:23,403 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:03:23,403 Example #1\n", "2020-02-13 17:03:23,403 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:03:23,403 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:03:23,403 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:03:23,403 Example #2\n", "2020-02-13 17:03:23,403 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:03:23,404 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:03:23,404 \tHypothesis: Monna ea bohlale o ne a kōpa : Motho e mong le e mong o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o seha , a hasa masiba a moea .\n", "2020-02-13 17:03:23,404 Example #3\n", "2020-02-13 17:03:23,404 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:03:23,404 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:03:23,404 \tHypothesis: Kaha nako ea matatu e ne e le teng , libuka tse ngata tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 17:03:23,404 Validation result (greedy) at epoch 5, step 40000: bleu: 42.08, loss: 29857.5469, ppl: 3.2300, duration: 37.6924s\n", "2020-02-13 17:04:00,223 Epoch 5 Step: 40100 Batch Loss: 1.230216 Tokens per Sec: 6724, Lr: 0.000300\n", "2020-02-13 17:04:37,156 Epoch 5 Step: 40200 Batch Loss: 1.078543 Tokens per Sec: 6722, Lr: 0.000300\n", "2020-02-13 17:05:13,536 Epoch 5 Step: 40300 Batch Loss: 1.327086 Tokens per Sec: 6656, Lr: 0.000300\n", "2020-02-13 17:05:49,780 Epoch 5 Step: 40400 Batch Loss: 1.272704 Tokens per Sec: 6554, Lr: 0.000300\n", "2020-02-13 17:06:26,019 Epoch 5 Step: 40500 Batch Loss: 1.279785 Tokens per Sec: 6777, Lr: 0.000300\n", "2020-02-13 17:07:02,794 Epoch 5 Step: 40600 Batch Loss: 1.347249 Tokens per Sec: 6794, Lr: 0.000300\n", "2020-02-13 17:07:39,707 Epoch 5 Step: 40700 Batch Loss: 1.223965 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-13 17:08:16,276 Epoch 5 Step: 40800 Batch Loss: 1.236297 Tokens per Sec: 6600, Lr: 0.000300\n", "2020-02-13 17:08:52,401 Epoch 5 Step: 40900 Batch Loss: 1.407514 Tokens per Sec: 6642, Lr: 0.000300\n", "2020-02-13 17:09:28,508 Epoch 5 Step: 41000 Batch Loss: 1.244035 Tokens per Sec: 6655, Lr: 0.000300\n", "2020-02-13 17:10:06,726 Example #0\n", "2020-02-13 17:10:06,727 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:10:06,727 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:10:06,727 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:10:06,727 Example #1\n", "2020-02-13 17:10:06,727 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:10:06,727 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:10:06,727 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:10:06,727 Example #2\n", "2020-02-13 17:10:06,728 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:10:06,728 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:10:06,728 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Motho ea sebang o ile a bolelloa hore a tsamaee ’ me a nke mosamo o motle ’ me a o seha , a hasanya masiba a moea .\n", "2020-02-13 17:10:06,728 Example #3\n", "2020-02-13 17:10:06,728 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:10:06,728 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:10:06,728 \tHypothesis: Kaha nako ea matatu e ne e le teng , libuka tse ngata tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 17:10:06,728 Validation result (greedy) at epoch 5, step 41000: bleu: 41.75, loss: 29925.3926, ppl: 3.2387, duration: 38.2195s\n", "2020-02-13 17:10:43,399 Epoch 5 Step: 41100 Batch Loss: 1.138481 Tokens per Sec: 6665, Lr: 0.000300\n", "2020-02-13 17:11:20,082 Epoch 5 Step: 41200 Batch Loss: 1.447280 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 17:11:56,538 Epoch 5 Step: 41300 Batch Loss: 1.449292 Tokens per Sec: 6801, Lr: 0.000300\n", "2020-02-13 17:12:32,729 Epoch 5 Step: 41400 Batch Loss: 1.589097 Tokens per Sec: 6708, Lr: 0.000300\n", "2020-02-13 17:13:09,578 Epoch 5 Step: 41500 Batch Loss: 1.311125 Tokens per Sec: 6749, Lr: 0.000300\n", "2020-02-13 17:13:46,138 Epoch 5 Step: 41600 Batch Loss: 1.462926 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 17:14:22,478 Epoch 5 Step: 41700 Batch Loss: 1.287129 Tokens per Sec: 6699, Lr: 0.000300\n", "2020-02-13 17:14:59,031 Epoch 5 Step: 41800 Batch Loss: 1.430924 Tokens per Sec: 6717, Lr: 0.000300\n", "2020-02-13 17:15:35,633 Epoch 5 Step: 41900 Batch Loss: 1.264861 Tokens per Sec: 6599, Lr: 0.000300\n", "2020-02-13 17:16:12,196 Epoch 5 Step: 42000 Batch Loss: 1.307562 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 17:16:48,914 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:16:48,915 Saving new checkpoint.\n", "2020-02-13 17:16:49,373 Example #0\n", "2020-02-13 17:16:49,373 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:16:49,373 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:16:49,374 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:16:49,374 Example #1\n", "2020-02-13 17:16:49,374 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:16:49,374 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:16:49,375 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:16:49,375 Example #2\n", "2020-02-13 17:16:49,375 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:16:49,375 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:16:49,375 \tHypothesis: Monna ea bohlale o ne a kōpa potso e ’ ngoe : Motho e mong le e mong o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:16:49,375 Example #3\n", "2020-02-13 17:16:49,375 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:16:49,376 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:16:49,376 \tHypothesis: Kaha nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 17:16:49,376 Validation result (greedy) at epoch 5, step 42000: bleu: 42.11, loss: 29745.0703, ppl: 3.2158, duration: 37.1792s\n", "2020-02-13 17:17:26,113 Epoch 5 Step: 42100 Batch Loss: 1.285939 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 17:18:02,059 Epoch 5 Step: 42200 Batch Loss: 1.541185 Tokens per Sec: 6642, Lr: 0.000300\n", "2020-02-13 17:18:38,634 Epoch 5 Step: 42300 Batch Loss: 1.492838 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 17:19:15,223 Epoch 5 Step: 42400 Batch Loss: 1.306477 Tokens per Sec: 6684, Lr: 0.000300\n", "2020-02-13 17:19:51,333 Epoch 5 Step: 42500 Batch Loss: 1.206918 Tokens per Sec: 6705, Lr: 0.000300\n", "2020-02-13 17:20:27,797 Epoch 5 Step: 42600 Batch Loss: 1.961212 Tokens per Sec: 6684, Lr: 0.000300\n", "2020-02-13 17:21:04,306 Epoch 5 Step: 42700 Batch Loss: 1.445234 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 17:21:40,547 Epoch 5 Step: 42800 Batch Loss: 1.937947 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 17:22:16,959 Epoch 5 Step: 42900 Batch Loss: 1.298615 Tokens per Sec: 6694, Lr: 0.000300\n", "2020-02-13 17:22:53,765 Epoch 5 Step: 43000 Batch Loss: 1.253910 Tokens per Sec: 6753, Lr: 0.000300\n", "2020-02-13 17:23:28,909 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:23:28,910 Saving new checkpoint.\n", "2020-02-13 17:23:29,380 Example #0\n", "2020-02-13 17:23:29,380 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:23:29,380 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:23:29,380 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:23:29,381 Example #1\n", "2020-02-13 17:23:29,381 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:23:29,381 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:23:29,381 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:23:29,381 Example #2\n", "2020-02-13 17:23:29,381 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:23:29,381 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:23:29,381 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Ho ile ha bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a jala masiba a moea .\n", "2020-02-13 17:23:29,381 Example #3\n", "2020-02-13 17:23:29,381 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:23:29,381 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:23:29,381 \tHypothesis: Kaha nako ea Masada e ne e le teng , libuka tse ngata tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 17:23:29,382 Validation result (greedy) at epoch 5, step 43000: bleu: 42.48, loss: 29463.8457, ppl: 3.1805, duration: 35.6163s\n", "2020-02-13 17:24:05,622 Epoch 5 Step: 43100 Batch Loss: 1.090297 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 17:24:42,267 Epoch 5 Step: 43200 Batch Loss: 1.244205 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 17:25:18,688 Epoch 5 Step: 43300 Batch Loss: 1.135204 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 17:25:54,963 Epoch 5 Step: 43400 Batch Loss: 1.411226 Tokens per Sec: 6719, Lr: 0.000300\n", "2020-02-13 17:26:31,638 Epoch 5 Step: 43500 Batch Loss: 1.322734 Tokens per Sec: 6688, Lr: 0.000300\n", "2020-02-13 17:27:04,537 Epoch 5: total training loss 11494.86\n", "2020-02-13 17:27:04,538 EPOCH 6\n", "2020-02-13 17:27:09,220 Epoch 6 Step: 43600 Batch Loss: 1.186600 Tokens per Sec: 5415, Lr: 0.000300\n", "2020-02-13 17:27:45,839 Epoch 6 Step: 43700 Batch Loss: 1.351231 Tokens per Sec: 6684, Lr: 0.000300\n", "2020-02-13 17:28:22,791 Epoch 6 Step: 43800 Batch Loss: 1.226101 Tokens per Sec: 6692, Lr: 0.000300\n", "2020-02-13 17:28:59,240 Epoch 6 Step: 43900 Batch Loss: 1.188249 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 17:29:35,914 Epoch 6 Step: 44000 Batch Loss: 1.234426 Tokens per Sec: 6700, Lr: 0.000300\n", "2020-02-13 17:30:11,780 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:30:11,781 Saving new checkpoint.\n", "2020-02-13 17:30:12,296 Example #0\n", "2020-02-13 17:30:12,296 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:30:12,296 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:30:12,296 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:30:12,296 Example #1\n", "2020-02-13 17:30:12,296 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:30:12,296 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:30:12,296 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:30:12,296 Example #2\n", "2020-02-13 17:30:12,297 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:30:12,297 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:30:12,297 \tHypothesis: Monna ea bohlale o ne a kōpa e mong : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo o mosehla ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:30:12,297 Example #3\n", "2020-02-13 17:30:12,297 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:30:12,297 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:30:12,297 \tHypothesis: Kaha nako ea Rotterdam , libuka tse ngata tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 17:30:12,297 Validation result (greedy) at epoch 6, step 44000: bleu: 42.76, loss: 29451.0820, ppl: 3.1789, duration: 36.3823s\n", "2020-02-13 17:30:48,739 Epoch 6 Step: 44100 Batch Loss: 1.201289 Tokens per Sec: 6704, Lr: 0.000300\n", "2020-02-13 17:31:25,427 Epoch 6 Step: 44200 Batch Loss: 1.253223 Tokens per Sec: 6735, Lr: 0.000300\n", "2020-02-13 17:32:01,850 Epoch 6 Step: 44300 Batch Loss: 1.142510 Tokens per Sec: 6681, Lr: 0.000300\n", "2020-02-13 17:32:38,667 Epoch 6 Step: 44400 Batch Loss: 1.222872 Tokens per Sec: 6757, Lr: 0.000300\n", "2020-02-13 17:33:15,160 Epoch 6 Step: 44500 Batch Loss: 1.239699 Tokens per Sec: 6660, Lr: 0.000300\n", "2020-02-13 17:33:51,570 Epoch 6 Step: 44600 Batch Loss: 1.209377 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 17:34:27,956 Epoch 6 Step: 44700 Batch Loss: 1.106605 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 17:35:04,227 Epoch 6 Step: 44800 Batch Loss: 1.186458 Tokens per Sec: 6740, Lr: 0.000300\n", "2020-02-13 17:35:40,881 Epoch 6 Step: 44900 Batch Loss: 1.146544 Tokens per Sec: 6617, Lr: 0.000300\n", "2020-02-13 17:36:17,308 Epoch 6 Step: 45000 Batch Loss: 1.362368 Tokens per Sec: 6633, Lr: 0.000300\n", "2020-02-13 17:36:55,320 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:36:55,320 Saving new checkpoint.\n", "2020-02-13 17:36:55,805 Example #0\n", "2020-02-13 17:36:55,805 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:36:55,805 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:36:55,805 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:36:55,805 Example #1\n", "2020-02-13 17:36:55,805 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:36:55,805 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:36:55,805 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:36:55,805 Example #2\n", "2020-02-13 17:36:55,806 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:36:55,806 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:36:55,806 \tHypothesis: Monna ea bohlale o ne a kōpa e mong le e mong : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:36:55,806 Example #3\n", "2020-02-13 17:36:55,806 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:36:55,806 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:36:55,806 \tHypothesis: Ho tloha ka nako ea Masada , libuka tse ngata tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 17:36:55,806 Validation result (greedy) at epoch 6, step 45000: bleu: 42.70, loss: 29204.7402, ppl: 3.1483, duration: 38.4977s\n", "2020-02-13 17:37:32,014 Epoch 6 Step: 45100 Batch Loss: 1.136108 Tokens per Sec: 6731, Lr: 0.000300\n", "2020-02-13 17:38:08,536 Epoch 6 Step: 45200 Batch Loss: 1.128790 Tokens per Sec: 6589, Lr: 0.000300\n", "2020-02-13 17:38:44,992 Epoch 6 Step: 45300 Batch Loss: 1.559348 Tokens per Sec: 6747, Lr: 0.000300\n", "2020-02-13 17:39:21,551 Epoch 6 Step: 45400 Batch Loss: 1.278577 Tokens per Sec: 6659, Lr: 0.000300\n", "2020-02-13 17:39:58,148 Epoch 6 Step: 45500 Batch Loss: 1.271537 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 17:40:34,934 Epoch 6 Step: 45600 Batch Loss: 1.156165 Tokens per Sec: 6616, Lr: 0.000300\n", "2020-02-13 17:41:11,660 Epoch 6 Step: 45700 Batch Loss: 1.216951 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 17:41:48,023 Epoch 6 Step: 45800 Batch Loss: 1.165641 Tokens per Sec: 6699, Lr: 0.000300\n", "2020-02-13 17:42:24,768 Epoch 6 Step: 45900 Batch Loss: 1.267656 Tokens per Sec: 6569, Lr: 0.000300\n", "2020-02-13 17:43:01,070 Epoch 6 Step: 46000 Batch Loss: 1.466257 Tokens per Sec: 6688, Lr: 0.000300\n", "2020-02-13 17:43:37,294 Example #0\n", "2020-02-13 17:43:37,294 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:43:37,294 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:43:37,294 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:43:37,294 Example #1\n", "2020-02-13 17:43:37,294 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:43:37,294 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:43:37,295 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:43:37,295 Example #2\n", "2020-02-13 17:43:37,295 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:43:37,295 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:43:37,295 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:43:37,295 Example #3\n", "2020-02-13 17:43:37,295 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:43:37,295 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:43:37,295 \tHypothesis: Kaha nako ea Rotterdam , libuka tse ngotsoeng ka letsoho li tletse khanya .\n", "2020-02-13 17:43:37,295 Validation result (greedy) at epoch 6, step 46000: bleu: 42.51, loss: 29253.8223, ppl: 3.1544, duration: 36.2250s\n", "2020-02-13 17:44:13,766 Epoch 6 Step: 46100 Batch Loss: 1.299812 Tokens per Sec: 6691, Lr: 0.000300\n", "2020-02-13 17:44:50,149 Epoch 6 Step: 46200 Batch Loss: 1.331483 Tokens per Sec: 6564, Lr: 0.000300\n", "2020-02-13 17:45:26,618 Epoch 6 Step: 46300 Batch Loss: 1.197941 Tokens per Sec: 6595, Lr: 0.000300\n", "2020-02-13 17:46:03,166 Epoch 6 Step: 46400 Batch Loss: 1.320754 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 17:46:39,627 Epoch 6 Step: 46500 Batch Loss: 1.043233 Tokens per Sec: 6793, Lr: 0.000300\n", "2020-02-13 17:47:15,865 Epoch 6 Step: 46600 Batch Loss: 1.210217 Tokens per Sec: 6516, Lr: 0.000300\n", "2020-02-13 17:47:52,487 Epoch 6 Step: 46700 Batch Loss: 1.513544 Tokens per Sec: 6635, Lr: 0.000300\n", "2020-02-13 17:48:29,359 Epoch 6 Step: 46800 Batch Loss: 1.315099 Tokens per Sec: 6692, Lr: 0.000300\n", "2020-02-13 17:49:06,094 Epoch 6 Step: 46900 Batch Loss: 1.253082 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 17:49:42,838 Epoch 6 Step: 47000 Batch Loss: 1.349571 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 17:50:19,780 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:50:19,780 Saving new checkpoint.\n", "2020-02-13 17:50:20,238 Example #0\n", "2020-02-13 17:50:20,238 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:50:20,238 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:50:20,238 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:50:20,238 Example #1\n", "2020-02-13 17:50:20,238 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:50:20,238 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:50:20,238 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:50:20,238 Example #2\n", "2020-02-13 17:50:20,240 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:50:20,240 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:50:20,240 \tHypothesis: Monna ea bohlale o ne a kōpa e mong : Motho ea sebang o ile a bolelloa hore a tsamaee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:50:20,240 Example #3\n", "2020-02-13 17:50:20,240 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:50:20,240 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:50:20,240 \tHypothesis: Kaha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ile tsa khanya .\n", "2020-02-13 17:50:20,241 Validation result (greedy) at epoch 6, step 47000: bleu: 43.27, loss: 29091.8008, ppl: 3.1344, duration: 37.4024s\n", "2020-02-13 17:50:56,973 Epoch 6 Step: 47100 Batch Loss: 1.202824 Tokens per Sec: 6786, Lr: 0.000300\n", "2020-02-13 17:51:33,244 Epoch 6 Step: 47200 Batch Loss: 1.362120 Tokens per Sec: 6681, Lr: 0.000300\n", "2020-02-13 17:52:09,612 Epoch 6 Step: 47300 Batch Loss: 1.388071 Tokens per Sec: 6607, Lr: 0.000300\n", "2020-02-13 17:52:46,089 Epoch 6 Step: 47400 Batch Loss: 1.337664 Tokens per Sec: 6644, Lr: 0.000300\n", "2020-02-13 17:53:22,796 Epoch 6 Step: 47500 Batch Loss: 1.286021 Tokens per Sec: 6784, Lr: 0.000300\n", "2020-02-13 17:53:59,528 Epoch 6 Step: 47600 Batch Loss: 1.332576 Tokens per Sec: 6647, Lr: 0.000300\n", "2020-02-13 17:54:35,943 Epoch 6 Step: 47700 Batch Loss: 1.344138 Tokens per Sec: 6525, Lr: 0.000300\n", "2020-02-13 17:55:12,333 Epoch 6 Step: 47800 Batch Loss: 1.294483 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 17:55:48,694 Epoch 6 Step: 47900 Batch Loss: 1.153355 Tokens per Sec: 6733, Lr: 0.000300\n", "2020-02-13 17:56:25,418 Epoch 6 Step: 48000 Batch Loss: 1.482689 Tokens per Sec: 6703, Lr: 0.000300\n", "2020-02-13 17:57:02,998 Hooray! New best validation result [ppl]!\n", "2020-02-13 17:57:02,998 Saving new checkpoint.\n", "2020-02-13 17:57:03,579 Example #0\n", "2020-02-13 17:57:03,579 \tSource: Has the situation improved since then ?\n", "2020-02-13 17:57:03,580 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 17:57:03,580 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 17:57:03,580 Example #1\n", "2020-02-13 17:57:03,580 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 17:57:03,580 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 17:57:03,580 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 17:57:03,580 Example #2\n", "2020-02-13 17:57:03,580 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 17:57:03,580 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 17:57:03,580 \tHypothesis: Monna ea bohlale o ne a kōpa potso e ’ ngoe : Motho ea sebang o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba a moea .\n", "2020-02-13 17:57:03,580 Example #3\n", "2020-02-13 17:57:03,580 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 17:57:03,580 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 17:57:03,580 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li tlile ho khanya .\n", "2020-02-13 17:57:03,580 Validation result (greedy) at epoch 6, step 48000: bleu: 43.50, loss: 28861.9609, ppl: 3.1062, duration: 38.1623s\n", "2020-02-13 17:57:40,114 Epoch 6 Step: 48100 Batch Loss: 1.228395 Tokens per Sec: 6679, Lr: 0.000300\n", "2020-02-13 17:58:17,012 Epoch 6 Step: 48200 Batch Loss: 1.263837 Tokens per Sec: 6761, Lr: 0.000300\n", "2020-02-13 17:58:53,481 Epoch 6 Step: 48300 Batch Loss: 1.165093 Tokens per Sec: 6623, Lr: 0.000300\n", "2020-02-13 17:59:30,247 Epoch 6 Step: 48400 Batch Loss: 1.359565 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 18:00:07,037 Epoch 6 Step: 48500 Batch Loss: 1.158476 Tokens per Sec: 6760, Lr: 0.000300\n", "2020-02-13 18:00:43,676 Epoch 6 Step: 48600 Batch Loss: 1.365399 Tokens per Sec: 6666, Lr: 0.000300\n", "2020-02-13 18:01:20,164 Epoch 6 Step: 48700 Batch Loss: 1.257676 Tokens per Sec: 6735, Lr: 0.000300\n", "2020-02-13 18:01:56,703 Epoch 6 Step: 48800 Batch Loss: 1.334424 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 18:02:33,030 Epoch 6 Step: 48900 Batch Loss: 1.222288 Tokens per Sec: 6646, Lr: 0.000300\n", "2020-02-13 18:03:09,592 Epoch 6 Step: 49000 Batch Loss: 1.221619 Tokens per Sec: 6840, Lr: 0.000300\n", "2020-02-13 18:03:44,630 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:03:44,631 Saving new checkpoint.\n", "2020-02-13 18:03:45,120 Example #0\n", "2020-02-13 18:03:45,121 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:03:45,121 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:03:45,121 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:03:45,121 Example #1\n", "2020-02-13 18:03:45,121 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:03:45,121 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:03:45,121 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:03:45,121 Example #2\n", "2020-02-13 18:03:45,121 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:03:45,121 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:03:45,121 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea sebetsoang o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a e khaola , a hasa masiba moeeng .\n", "2020-02-13 18:03:45,121 Example #3\n", "2020-02-13 18:03:45,121 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:03:45,121 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:03:45,122 \tHypothesis: Kaha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li tlile ho leseli .\n", "2020-02-13 18:03:45,122 Validation result (greedy) at epoch 6, step 49000: bleu: 42.83, loss: 28771.9023, ppl: 3.0952, duration: 35.5293s\n", "2020-02-13 18:04:21,674 Epoch 6 Step: 49100 Batch Loss: 1.270600 Tokens per Sec: 6623, Lr: 0.000300\n", "2020-02-13 18:04:58,274 Epoch 6 Step: 49200 Batch Loss: 1.197913 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 18:05:34,857 Epoch 6 Step: 49300 Batch Loss: 1.199050 Tokens per Sec: 6700, Lr: 0.000300\n", "2020-02-13 18:06:11,412 Epoch 6 Step: 49400 Batch Loss: 1.148593 Tokens per Sec: 6600, Lr: 0.000300\n", "2020-02-13 18:06:48,330 Epoch 6 Step: 49500 Batch Loss: 1.371954 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 18:07:24,640 Epoch 6 Step: 49600 Batch Loss: 1.120553 Tokens per Sec: 6599, Lr: 0.000300\n", "2020-02-13 18:08:01,598 Epoch 6 Step: 49700 Batch Loss: 1.270594 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 18:08:37,968 Epoch 6 Step: 49800 Batch Loss: 1.333892 Tokens per Sec: 6629, Lr: 0.000300\n", "2020-02-13 18:09:14,404 Epoch 6 Step: 49900 Batch Loss: 1.243850 Tokens per Sec: 6588, Lr: 0.000300\n", "2020-02-13 18:09:51,318 Epoch 6 Step: 50000 Batch Loss: 1.176693 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 18:10:27,721 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:10:27,721 Saving new checkpoint.\n", "2020-02-13 18:10:28,213 Example #0\n", "2020-02-13 18:10:28,213 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:10:28,213 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:10:28,213 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:10:28,213 Example #1\n", "2020-02-13 18:10:28,214 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:10:28,214 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:10:28,214 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:10:28,214 Example #2\n", "2020-02-13 18:10:28,214 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:10:28,214 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:10:28,214 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Ho ile ha bolelloa hore motho a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a qhalakanya masiba moeeng .\n", "2020-02-13 18:10:28,214 Example #3\n", "2020-02-13 18:10:28,214 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:10:28,214 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:10:28,214 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 18:10:28,214 Validation result (greedy) at epoch 6, step 50000: bleu: 42.97, loss: 28744.8750, ppl: 3.0919, duration: 36.8960s\n", "2020-02-13 18:11:05,050 Epoch 6 Step: 50100 Batch Loss: 1.124232 Tokens per Sec: 6774, Lr: 0.000300\n", "2020-02-13 18:11:41,543 Epoch 6 Step: 50200 Batch Loss: 1.298518 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 18:12:18,199 Epoch 6 Step: 50300 Batch Loss: 1.210134 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 18:12:54,470 Epoch 6 Step: 50400 Batch Loss: 1.394305 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-13 18:13:31,032 Epoch 6 Step: 50500 Batch Loss: 1.186073 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 18:14:07,391 Epoch 6 Step: 50600 Batch Loss: 1.034852 Tokens per Sec: 6776, Lr: 0.000300\n", "2020-02-13 18:14:43,858 Epoch 6 Step: 50700 Batch Loss: 1.183180 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 18:15:20,504 Epoch 6 Step: 50800 Batch Loss: 1.187327 Tokens per Sec: 6665, Lr: 0.000300\n", "2020-02-13 18:15:57,059 Epoch 6 Step: 50900 Batch Loss: 1.178638 Tokens per Sec: 6767, Lr: 0.000300\n", "2020-02-13 18:16:33,655 Epoch 6 Step: 51000 Batch Loss: 1.174770 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 18:17:10,928 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:17:10,928 Saving new checkpoint.\n", "2020-02-13 18:17:11,434 Example #0\n", "2020-02-13 18:17:11,434 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:17:11,434 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:17:11,434 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:17:11,434 Example #1\n", "2020-02-13 18:17:11,434 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:17:11,434 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:17:11,434 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:17:11,434 Example #2\n", "2020-02-13 18:17:11,434 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:17:11,434 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:17:11,435 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea buang le eena o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 18:17:11,435 Example #3\n", "2020-02-13 18:17:11,435 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:17:11,435 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:17:11,435 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 18:17:11,435 Validation result (greedy) at epoch 6, step 51000: bleu: 43.09, loss: 28496.4258, ppl: 3.0619, duration: 37.7791s\n", "2020-02-13 18:17:47,972 Epoch 6 Step: 51100 Batch Loss: 1.289446 Tokens per Sec: 6553, Lr: 0.000300\n", "2020-02-13 18:18:24,866 Epoch 6 Step: 51200 Batch Loss: 1.242627 Tokens per Sec: 6699, Lr: 0.000300\n", "2020-02-13 18:19:01,295 Epoch 6 Step: 51300 Batch Loss: 1.300009 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 18:19:37,593 Epoch 6 Step: 51400 Batch Loss: 1.322448 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 18:20:14,164 Epoch 6 Step: 51500 Batch Loss: 1.236816 Tokens per Sec: 6712, Lr: 0.000300\n", "2020-02-13 18:20:50,878 Epoch 6 Step: 51600 Batch Loss: 1.294579 Tokens per Sec: 6652, Lr: 0.000300\n", "2020-02-13 18:21:27,897 Epoch 6 Step: 51700 Batch Loss: 1.369861 Tokens per Sec: 6817, Lr: 0.000300\n", "2020-02-13 18:22:04,252 Epoch 6 Step: 51800 Batch Loss: 1.344814 Tokens per Sec: 6652, Lr: 0.000300\n", "2020-02-13 18:22:40,883 Epoch 6 Step: 51900 Batch Loss: 1.296863 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 18:23:17,348 Epoch 6 Step: 52000 Batch Loss: 1.281821 Tokens per Sec: 6734, Lr: 0.000300\n", "2020-02-13 18:23:55,466 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:23:55,466 Saving new checkpoint.\n", "2020-02-13 18:23:55,972 Example #0\n", "2020-02-13 18:23:55,972 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:23:55,972 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:23:55,972 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:23:55,972 Example #1\n", "2020-02-13 18:23:55,972 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:23:55,972 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:23:55,972 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:23:55,972 Example #2\n", "2020-02-13 18:23:55,973 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:23:55,973 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:23:55,973 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 18:23:55,973 Example #3\n", "2020-02-13 18:23:55,973 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:23:55,973 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:23:55,973 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ile tsa tla leseling .\n", "2020-02-13 18:23:55,973 Validation result (greedy) at epoch 6, step 52000: bleu: 43.45, loss: 28301.8086, ppl: 3.0386, duration: 38.6249s\n", "2020-02-13 18:24:32,643 Epoch 6 Step: 52100 Batch Loss: 1.197333 Tokens per Sec: 6712, Lr: 0.000300\n", "2020-02-13 18:25:09,140 Epoch 6 Step: 52200 Batch Loss: 1.229690 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 18:25:45,520 Epoch 6 Step: 52300 Batch Loss: 1.280903 Tokens per Sec: 6807, Lr: 0.000300\n", "2020-02-13 18:25:45,549 Epoch 6: total training loss 10993.07\n", "2020-02-13 18:25:45,549 EPOCH 7\n", "2020-02-13 18:26:23,028 Epoch 7 Step: 52400 Batch Loss: 1.165134 Tokens per Sec: 6475, Lr: 0.000300\n", "2020-02-13 18:26:59,395 Epoch 7 Step: 52500 Batch Loss: 1.268270 Tokens per Sec: 6743, Lr: 0.000300\n", "2020-02-13 18:27:35,727 Epoch 7 Step: 52600 Batch Loss: 1.083850 Tokens per Sec: 6730, Lr: 0.000300\n", "2020-02-13 18:28:12,153 Epoch 7 Step: 52700 Batch Loss: 1.216989 Tokens per Sec: 6614, Lr: 0.000300\n", "2020-02-13 18:28:48,452 Epoch 7 Step: 52800 Batch Loss: 1.025847 Tokens per Sec: 6716, Lr: 0.000300\n", "2020-02-13 18:29:24,849 Epoch 7 Step: 52900 Batch Loss: 1.362695 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 18:30:00,958 Epoch 7 Step: 53000 Batch Loss: 1.204186 Tokens per Sec: 6711, Lr: 0.000300\n", "2020-02-13 18:30:39,342 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:30:39,342 Saving new checkpoint.\n", "2020-02-13 18:30:39,850 Example #0\n", "2020-02-13 18:30:39,850 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:30:39,851 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:30:39,851 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:30:39,851 Example #1\n", "2020-02-13 18:30:39,851 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:30:39,851 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:30:39,851 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:30:39,851 Example #2\n", "2020-02-13 18:30:39,851 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:30:39,851 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:30:39,851 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 18:30:39,851 Example #3\n", "2020-02-13 18:30:39,851 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:30:39,851 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:30:39,851 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ngata tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 18:30:39,851 Validation result (greedy) at epoch 7, step 53000: bleu: 43.66, loss: 28258.1875, ppl: 3.0334, duration: 38.8930s\n", "2020-02-13 18:31:16,029 Epoch 7 Step: 53100 Batch Loss: 1.230265 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 18:31:52,660 Epoch 7 Step: 53200 Batch Loss: 1.284304 Tokens per Sec: 6790, Lr: 0.000300\n", "2020-02-13 18:32:28,777 Epoch 7 Step: 53300 Batch Loss: 1.181489 Tokens per Sec: 6655, Lr: 0.000300\n", "2020-02-13 18:33:05,202 Epoch 7 Step: 53400 Batch Loss: 1.267413 Tokens per Sec: 6586, Lr: 0.000300\n", "2020-02-13 18:33:41,952 Epoch 7 Step: 53500 Batch Loss: 1.403742 Tokens per Sec: 6654, Lr: 0.000300\n", "2020-02-13 18:34:18,027 Epoch 7 Step: 53600 Batch Loss: 1.229260 Tokens per Sec: 6571, Lr: 0.000300\n", "2020-02-13 18:34:54,188 Epoch 7 Step: 53700 Batch Loss: 1.164559 Tokens per Sec: 6608, Lr: 0.000300\n", "2020-02-13 18:35:30,650 Epoch 7 Step: 53800 Batch Loss: 1.246973 Tokens per Sec: 6889, Lr: 0.000300\n", "2020-02-13 18:36:06,959 Epoch 7 Step: 53900 Batch Loss: 1.100561 Tokens per Sec: 6740, Lr: 0.000300\n", "2020-02-13 18:36:43,413 Epoch 7 Step: 54000 Batch Loss: 0.979161 Tokens per Sec: 6701, Lr: 0.000300\n", "2020-02-13 18:37:20,485 Example #0\n", "2020-02-13 18:37:20,485 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:37:20,485 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:37:20,485 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:37:20,485 Example #1\n", "2020-02-13 18:37:20,485 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:37:20,485 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:37:20,485 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:37:20,486 Example #2\n", "2020-02-13 18:37:20,486 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:37:20,486 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:37:20,486 \tHypothesis: Monna ea bohlale o ne a kōpa ntho e le ’ ngoe : Motho ea sebang o ile a bolelloa hore a ee ’ me a nke mosamo oa ho kuta , a hasanya masiba moeeng .\n", "2020-02-13 18:37:20,486 Example #3\n", "2020-02-13 18:37:20,486 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:37:20,486 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:37:20,486 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 18:37:20,486 Validation result (greedy) at epoch 7, step 54000: bleu: 43.33, loss: 28327.1777, ppl: 3.0416, duration: 37.0736s\n", "2020-02-13 18:37:56,946 Epoch 7 Step: 54100 Batch Loss: 1.343610 Tokens per Sec: 6710, Lr: 0.000300\n", "2020-02-13 18:38:33,355 Epoch 7 Step: 54200 Batch Loss: 1.196584 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 18:39:09,548 Epoch 7 Step: 54300 Batch Loss: 1.259810 Tokens per Sec: 6672, Lr: 0.000300\n", "2020-02-13 18:39:46,119 Epoch 7 Step: 54400 Batch Loss: 1.259917 Tokens per Sec: 6675, Lr: 0.000300\n", "2020-02-13 18:40:22,598 Epoch 7 Step: 54500 Batch Loss: 1.457117 Tokens per Sec: 6593, Lr: 0.000300\n", "2020-02-13 18:40:58,931 Epoch 7 Step: 54600 Batch Loss: 2.003730 Tokens per Sec: 6511, Lr: 0.000300\n", "2020-02-13 18:41:35,286 Epoch 7 Step: 54700 Batch Loss: 1.263600 Tokens per Sec: 6704, Lr: 0.000300\n", "2020-02-13 18:42:11,863 Epoch 7 Step: 54800 Batch Loss: 1.237908 Tokens per Sec: 6683, Lr: 0.000300\n", "2020-02-13 18:42:48,445 Epoch 7 Step: 54900 Batch Loss: 1.347569 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 18:43:24,896 Epoch 7 Step: 55000 Batch Loss: 1.226582 Tokens per Sec: 6761, Lr: 0.000300\n", "2020-02-13 18:43:59,997 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:43:59,997 Saving new checkpoint.\n", "2020-02-13 18:44:00,533 Example #0\n", "2020-02-13 18:44:00,533 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:44:00,533 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:44:00,533 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:44:00,533 Example #1\n", "2020-02-13 18:44:00,533 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:44:00,533 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:44:00,533 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:44:00,533 Example #2\n", "2020-02-13 18:44:00,534 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:44:00,534 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:44:00,534 \tHypothesis: Monna ea bohlale o ne a kōpa e mong : Motho ea sebang o ile a bolelloa hore a tsamaee ’ me a nke mosamo o mofubelu ’ me a o rema , a hasanya masiba moeeng .\n", "2020-02-13 18:44:00,534 Example #3\n", "2020-02-13 18:44:00,534 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:44:00,534 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:44:00,534 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 18:44:00,534 Validation result (greedy) at epoch 7, step 55000: bleu: 43.16, loss: 28253.7285, ppl: 3.0329, duration: 35.6382s\n", "2020-02-13 18:44:36,986 Epoch 7 Step: 55100 Batch Loss: 1.061666 Tokens per Sec: 6579, Lr: 0.000300\n", "2020-02-13 18:45:13,502 Epoch 7 Step: 55200 Batch Loss: 1.120197 Tokens per Sec: 6686, Lr: 0.000300\n", "2020-02-13 18:45:49,965 Epoch 7 Step: 55300 Batch Loss: 1.289261 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 18:46:26,546 Epoch 7 Step: 55400 Batch Loss: 1.219827 Tokens per Sec: 6789, Lr: 0.000300\n", "2020-02-13 18:47:02,771 Epoch 7 Step: 55500 Batch Loss: 1.215217 Tokens per Sec: 6653, Lr: 0.000300\n", "2020-02-13 18:47:39,421 Epoch 7 Step: 55600 Batch Loss: 1.282274 Tokens per Sec: 6655, Lr: 0.000300\n", "2020-02-13 18:48:15,953 Epoch 7 Step: 55700 Batch Loss: 1.144995 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 18:48:52,843 Epoch 7 Step: 55800 Batch Loss: 1.369055 Tokens per Sec: 6676, Lr: 0.000300\n", "2020-02-13 18:49:29,330 Epoch 7 Step: 55900 Batch Loss: 1.170051 Tokens per Sec: 6744, Lr: 0.000300\n", "2020-02-13 18:50:05,338 Epoch 7 Step: 56000 Batch Loss: 1.228079 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 18:50:42,569 Example #0\n", "2020-02-13 18:50:42,569 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:50:42,569 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:50:42,569 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:50:42,569 Example #1\n", "2020-02-13 18:50:42,569 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:50:42,569 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:50:42,569 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:50:42,569 Example #2\n", "2020-02-13 18:50:42,570 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:50:42,570 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:50:42,570 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o seha , a hasanya masiba moeeng .\n", "2020-02-13 18:50:42,570 Example #3\n", "2020-02-13 18:50:42,570 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:50:42,570 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:50:42,570 \tHypothesis: Kaha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 18:50:42,570 Validation result (greedy) at epoch 7, step 56000: bleu: 43.09, loss: 28331.0156, ppl: 3.0421, duration: 37.2320s\n", "2020-02-13 18:51:19,412 Epoch 7 Step: 56100 Batch Loss: 1.189920 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 18:51:55,956 Epoch 7 Step: 56200 Batch Loss: 1.222248 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 18:52:32,515 Epoch 7 Step: 56300 Batch Loss: 1.233979 Tokens per Sec: 6574, Lr: 0.000300\n", "2020-02-13 18:53:08,828 Epoch 7 Step: 56400 Batch Loss: 1.220240 Tokens per Sec: 6662, Lr: 0.000300\n", "2020-02-13 18:53:45,399 Epoch 7 Step: 56500 Batch Loss: 1.175251 Tokens per Sec: 6616, Lr: 0.000300\n", "2020-02-13 18:54:22,040 Epoch 7 Step: 56600 Batch Loss: 1.110440 Tokens per Sec: 6792, Lr: 0.000300\n", "2020-02-13 18:54:58,283 Epoch 7 Step: 56700 Batch Loss: 1.304072 Tokens per Sec: 6581, Lr: 0.000300\n", "2020-02-13 18:55:34,895 Epoch 7 Step: 56800 Batch Loss: 1.186534 Tokens per Sec: 6596, Lr: 0.000300\n", "2020-02-13 18:56:11,209 Epoch 7 Step: 56900 Batch Loss: 1.159507 Tokens per Sec: 6796, Lr: 0.000300\n", "2020-02-13 18:56:47,679 Epoch 7 Step: 57000 Batch Loss: 1.390868 Tokens per Sec: 6685, Lr: 0.000300\n", "2020-02-13 18:57:23,923 Hooray! New best validation result [ppl]!\n", "2020-02-13 18:57:23,923 Saving new checkpoint.\n", "2020-02-13 18:57:24,407 Example #0\n", "2020-02-13 18:57:24,407 \tSource: Has the situation improved since then ?\n", "2020-02-13 18:57:24,407 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 18:57:24,407 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 18:57:24,407 Example #1\n", "2020-02-13 18:57:24,408 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 18:57:24,408 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 18:57:24,408 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 18:57:24,408 Example #2\n", "2020-02-13 18:57:24,408 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 18:57:24,408 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 18:57:24,408 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 18:57:24,408 Example #3\n", "2020-02-13 18:57:24,408 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 18:57:24,408 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 18:57:24,408 \tHypothesis: Kaha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 18:57:24,408 Validation result (greedy) at epoch 7, step 57000: bleu: 43.35, loss: 27946.9707, ppl: 2.9966, duration: 36.7290s\n", "2020-02-13 18:58:01,263 Epoch 7 Step: 57100 Batch Loss: 1.328038 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 18:58:37,583 Epoch 7 Step: 57200 Batch Loss: 1.166311 Tokens per Sec: 6567, Lr: 0.000300\n", "2020-02-13 18:59:14,114 Epoch 7 Step: 57300 Batch Loss: 1.056441 Tokens per Sec: 6680, Lr: 0.000300\n", "2020-02-13 18:59:50,630 Epoch 7 Step: 57400 Batch Loss: 1.375558 Tokens per Sec: 6674, Lr: 0.000300\n", "2020-02-13 19:00:27,145 Epoch 7 Step: 57500 Batch Loss: 1.220669 Tokens per Sec: 6691, Lr: 0.000300\n", "2020-02-13 19:01:03,297 Epoch 7 Step: 57600 Batch Loss: 1.294428 Tokens per Sec: 6734, Lr: 0.000300\n", "2020-02-13 19:01:40,388 Epoch 7 Step: 57700 Batch Loss: 1.325870 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 19:02:16,987 Epoch 7 Step: 57800 Batch Loss: 1.330151 Tokens per Sec: 6706, Lr: 0.000300\n", "2020-02-13 19:02:53,860 Epoch 7 Step: 57900 Batch Loss: 1.193484 Tokens per Sec: 6672, Lr: 0.000300\n", "2020-02-13 19:03:30,293 Epoch 7 Step: 58000 Batch Loss: 1.275565 Tokens per Sec: 6698, Lr: 0.000300\n", "2020-02-13 19:04:04,910 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:04:04,910 Saving new checkpoint.\n", "2020-02-13 19:04:05,435 Example #0\n", "2020-02-13 19:04:05,435 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:04:05,435 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:04:05,435 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:04:05,435 Example #1\n", "2020-02-13 19:04:05,435 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:04:05,435 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:04:05,435 \tHypothesis: Ho feta moo , ha Jehova a ne a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:04:05,435 Example #2\n", "2020-02-13 19:04:05,435 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:04:05,436 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:04:05,436 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo oa masiba ’ me a o ntše , a qhalakanya masiba moeeng .\n", "2020-02-13 19:04:05,436 Example #3\n", "2020-02-13 19:04:05,436 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:04:05,436 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:04:05,436 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ile tsa tla leseling .\n", "2020-02-13 19:04:05,436 Validation result (greedy) at epoch 7, step 58000: bleu: 43.85, loss: 27931.4219, ppl: 2.9947, duration: 35.1430s\n", "2020-02-13 19:04:42,357 Epoch 7 Step: 58100 Batch Loss: 1.291068 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 19:05:19,204 Epoch 7 Step: 58200 Batch Loss: 1.199566 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 19:05:55,902 Epoch 7 Step: 58300 Batch Loss: 1.393063 Tokens per Sec: 6668, Lr: 0.000300\n", "2020-02-13 19:06:32,622 Epoch 7 Step: 58400 Batch Loss: 1.163222 Tokens per Sec: 6754, Lr: 0.000300\n", "2020-02-13 19:07:09,408 Epoch 7 Step: 58500 Batch Loss: 1.280603 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 19:07:46,029 Epoch 7 Step: 58600 Batch Loss: 1.285695 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 19:08:22,430 Epoch 7 Step: 58700 Batch Loss: 1.162767 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 19:08:59,192 Epoch 7 Step: 58800 Batch Loss: 1.238451 Tokens per Sec: 6702, Lr: 0.000300\n", "2020-02-13 19:09:35,751 Epoch 7 Step: 58900 Batch Loss: 1.170632 Tokens per Sec: 6582, Lr: 0.000300\n", "2020-02-13 19:10:12,121 Epoch 7 Step: 59000 Batch Loss: 1.261804 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 19:10:47,749 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:10:47,750 Saving new checkpoint.\n", "2020-02-13 19:10:48,225 Example #0\n", "2020-02-13 19:10:48,226 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:10:48,226 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:10:48,226 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:10:48,226 Example #1\n", "2020-02-13 19:10:48,226 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:10:48,226 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:10:48,226 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:10:48,226 Example #2\n", "2020-02-13 19:10:48,226 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:10:48,226 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:10:48,226 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:10:48,226 Example #3\n", "2020-02-13 19:10:48,226 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:10:48,226 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:10:48,226 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 19:10:48,227 Validation result (greedy) at epoch 7, step 59000: bleu: 43.66, loss: 27766.7051, ppl: 2.9754, duration: 36.1056s\n", "2020-02-13 19:11:24,439 Epoch 7 Step: 59100 Batch Loss: 1.319238 Tokens per Sec: 6533, Lr: 0.000300\n", "2020-02-13 19:12:01,117 Epoch 7 Step: 59200 Batch Loss: 1.351371 Tokens per Sec: 6685, Lr: 0.000300\n", "2020-02-13 19:12:37,772 Epoch 7 Step: 59300 Batch Loss: 1.267784 Tokens per Sec: 6730, Lr: 0.000300\n", "2020-02-13 19:13:14,125 Epoch 7 Step: 59400 Batch Loss: 1.283904 Tokens per Sec: 6647, Lr: 0.000300\n", "2020-02-13 19:13:50,627 Epoch 7 Step: 59500 Batch Loss: 1.182260 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 19:14:26,999 Epoch 7 Step: 59600 Batch Loss: 1.219555 Tokens per Sec: 6809, Lr: 0.000300\n", "2020-02-13 19:15:03,467 Epoch 7 Step: 59700 Batch Loss: 1.143651 Tokens per Sec: 6726, Lr: 0.000300\n", "2020-02-13 19:15:40,109 Epoch 7 Step: 59800 Batch Loss: 1.124798 Tokens per Sec: 6733, Lr: 0.000300\n", "2020-02-13 19:16:16,408 Epoch 7 Step: 59900 Batch Loss: 1.210898 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 19:16:52,914 Epoch 7 Step: 60000 Batch Loss: 1.106294 Tokens per Sec: 6607, Lr: 0.000300\n", "2020-02-13 19:17:29,901 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:17:29,902 Saving new checkpoint.\n", "2020-02-13 19:17:30,428 Example #0\n", "2020-02-13 19:17:30,428 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:17:30,428 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:17:30,428 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:17:30,428 Example #1\n", "2020-02-13 19:17:30,428 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:17:30,428 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:17:30,428 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:17:30,428 Example #2\n", "2020-02-13 19:17:30,428 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:17:30,428 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:17:30,428 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:17:30,428 Example #3\n", "2020-02-13 19:17:30,429 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:17:30,429 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:17:30,429 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 19:17:30,429 Validation result (greedy) at epoch 7, step 60000: bleu: 43.68, loss: 27651.6094, ppl: 2.9620, duration: 37.5142s\n", "2020-02-13 19:18:07,011 Epoch 7 Step: 60100 Batch Loss: 1.219449 Tokens per Sec: 6659, Lr: 0.000300\n", "2020-02-13 19:18:43,825 Epoch 7 Step: 60200 Batch Loss: 1.089618 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 19:19:20,572 Epoch 7 Step: 60300 Batch Loss: 1.057329 Tokens per Sec: 6615, Lr: 0.000300\n", "2020-02-13 19:19:56,904 Epoch 7 Step: 60400 Batch Loss: 1.276045 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 19:20:33,892 Epoch 7 Step: 60500 Batch Loss: 1.131004 Tokens per Sec: 6676, Lr: 0.000300\n", "2020-02-13 19:21:10,504 Epoch 7 Step: 60600 Batch Loss: 1.216222 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 19:21:47,393 Epoch 7 Step: 60700 Batch Loss: 1.252294 Tokens per Sec: 6740, Lr: 0.000300\n", "2020-02-13 19:22:24,042 Epoch 7 Step: 60800 Batch Loss: 1.199789 Tokens per Sec: 6553, Lr: 0.000300\n", "2020-02-13 19:23:00,215 Epoch 7 Step: 60900 Batch Loss: 1.093223 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 19:23:36,705 Epoch 7 Step: 61000 Batch Loss: 1.075059 Tokens per Sec: 6530, Lr: 0.000300\n", "2020-02-13 19:24:11,757 Example #0\n", "2020-02-13 19:24:11,757 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:24:11,757 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:24:11,757 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:24:11,758 Example #1\n", "2020-02-13 19:24:11,758 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:24:11,758 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:24:11,758 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:24:11,758 Example #2\n", "2020-02-13 19:24:11,758 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:24:11,758 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:24:11,758 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:24:11,758 Example #3\n", "2020-02-13 19:24:11,758 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:24:11,758 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:24:11,758 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ile tsa tla leseling .\n", "2020-02-13 19:24:11,758 Validation result (greedy) at epoch 7, step 61000: bleu: 43.69, loss: 27738.0566, ppl: 2.9721, duration: 35.0535s\n", "2020-02-13 19:24:21,530 Epoch 7: total training loss 10648.65\n", "2020-02-13 19:24:21,530 EPOCH 8\n", "2020-02-13 19:24:49,319 Epoch 8 Step: 61100 Batch Loss: 1.196892 Tokens per Sec: 6321, Lr: 0.000300\n", "2020-02-13 19:25:26,136 Epoch 8 Step: 61200 Batch Loss: 1.300145 Tokens per Sec: 6780, Lr: 0.000300\n", "2020-02-13 19:26:02,535 Epoch 8 Step: 61300 Batch Loss: 1.137733 Tokens per Sec: 6691, Lr: 0.000300\n", "2020-02-13 19:26:39,228 Epoch 8 Step: 61400 Batch Loss: 1.151722 Tokens per Sec: 6632, Lr: 0.000300\n", "2020-02-13 19:27:16,147 Epoch 8 Step: 61500 Batch Loss: 1.246234 Tokens per Sec: 6679, Lr: 0.000300\n", "2020-02-13 19:27:52,838 Epoch 8 Step: 61600 Batch Loss: 1.233277 Tokens per Sec: 6730, Lr: 0.000300\n", "2020-02-13 19:28:29,151 Epoch 8 Step: 61700 Batch Loss: 0.927982 Tokens per Sec: 6566, Lr: 0.000300\n", "2020-02-13 19:29:05,782 Epoch 8 Step: 61800 Batch Loss: 1.266898 Tokens per Sec: 6762, Lr: 0.000300\n", "2020-02-13 19:29:42,402 Epoch 8 Step: 61900 Batch Loss: 1.114007 Tokens per Sec: 6588, Lr: 0.000300\n", "2020-02-13 19:30:19,234 Epoch 8 Step: 62000 Batch Loss: 1.083528 Tokens per Sec: 6719, Lr: 0.000300\n", "2020-02-13 19:30:55,608 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:30:55,608 Saving new checkpoint.\n", "2020-02-13 19:30:56,089 Example #0\n", "2020-02-13 19:30:56,089 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:30:56,089 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:30:56,089 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:30:56,089 Example #1\n", "2020-02-13 19:30:56,090 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:30:56,090 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:30:56,090 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:30:56,090 Example #2\n", "2020-02-13 19:30:56,090 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:30:56,090 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:30:56,090 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaole , a qhalla masiba moeeng .\n", "2020-02-13 19:30:56,090 Example #3\n", "2020-02-13 19:30:56,090 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:30:56,090 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:30:56,090 \tHypothesis: Kaha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li tlile leseling .\n", "2020-02-13 19:30:56,090 Validation result (greedy) at epoch 8, step 62000: bleu: 44.72, loss: 27610.0352, ppl: 2.9572, duration: 36.8556s\n", "2020-02-13 19:31:32,248 Epoch 8 Step: 62100 Batch Loss: 1.164930 Tokens per Sec: 6574, Lr: 0.000300\n", "2020-02-13 19:32:08,641 Epoch 8 Step: 62200 Batch Loss: 1.150347 Tokens per Sec: 6622, Lr: 0.000300\n", "2020-02-13 19:32:45,392 Epoch 8 Step: 62300 Batch Loss: 1.124256 Tokens per Sec: 6711, Lr: 0.000300\n", "2020-02-13 19:33:21,902 Epoch 8 Step: 62400 Batch Loss: 1.152305 Tokens per Sec: 6663, Lr: 0.000300\n", "2020-02-13 19:33:58,351 Epoch 8 Step: 62500 Batch Loss: 1.205153 Tokens per Sec: 6731, Lr: 0.000300\n", "2020-02-13 19:34:35,166 Epoch 8 Step: 62600 Batch Loss: 1.084226 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 19:35:11,323 Epoch 8 Step: 62700 Batch Loss: 1.272275 Tokens per Sec: 6669, Lr: 0.000300\n", "2020-02-13 19:35:47,997 Epoch 8 Step: 62800 Batch Loss: 1.198144 Tokens per Sec: 6625, Lr: 0.000300\n", "2020-02-13 19:36:24,645 Epoch 8 Step: 62900 Batch Loss: 1.054329 Tokens per Sec: 6602, Lr: 0.000300\n", "2020-02-13 19:37:00,937 Epoch 8 Step: 63000 Batch Loss: 1.030788 Tokens per Sec: 6676, Lr: 0.000300\n", "2020-02-13 19:37:43,570 Example #0\n", "2020-02-13 19:37:43,570 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:37:43,570 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:37:43,570 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:37:43,570 Example #1\n", "2020-02-13 19:37:43,570 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:37:43,570 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:37:43,571 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:37:43,571 Example #2\n", "2020-02-13 19:37:43,571 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:37:43,571 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:37:43,571 \tHypothesis: Monna ea bohlale o ne a kōpa e mong : Motho ea sebang o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:37:43,571 Example #3\n", "2020-02-13 19:37:43,571 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:37:43,571 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:37:43,571 \tHypothesis: Kaha mehleng ea Tischendorf , libuka tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 19:37:43,571 Validation result (greedy) at epoch 8, step 63000: bleu: 43.84, loss: 27856.8145, ppl: 2.9860, duration: 42.6335s\n", "2020-02-13 19:38:20,217 Epoch 8 Step: 63100 Batch Loss: 1.170221 Tokens per Sec: 6662, Lr: 0.000300\n", "2020-02-13 19:38:56,467 Epoch 8 Step: 63200 Batch Loss: 1.107851 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 19:39:33,067 Epoch 8 Step: 63300 Batch Loss: 1.098793 Tokens per Sec: 6786, Lr: 0.000300\n", "2020-02-13 19:40:09,715 Epoch 8 Step: 63400 Batch Loss: 1.122432 Tokens per Sec: 6710, Lr: 0.000300\n", "2020-02-13 19:40:46,203 Epoch 8 Step: 63500 Batch Loss: 1.361249 Tokens per Sec: 6651, Lr: 0.000300\n", "2020-02-13 19:41:23,004 Epoch 8 Step: 63600 Batch Loss: 1.338939 Tokens per Sec: 6670, Lr: 0.000300\n", "2020-02-13 19:41:59,621 Epoch 8 Step: 63700 Batch Loss: 1.230793 Tokens per Sec: 6720, Lr: 0.000300\n", "2020-02-13 19:42:36,055 Epoch 8 Step: 63800 Batch Loss: 1.209454 Tokens per Sec: 6644, Lr: 0.000300\n", "2020-02-13 19:43:12,842 Epoch 8 Step: 63900 Batch Loss: 1.290003 Tokens per Sec: 6702, Lr: 0.000300\n", "2020-02-13 19:43:49,106 Epoch 8 Step: 64000 Batch Loss: 1.027844 Tokens per Sec: 6687, Lr: 0.000300\n", "2020-02-13 19:44:26,588 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:44:26,588 Saving new checkpoint.\n", "2020-02-13 19:44:27,173 Example #0\n", "2020-02-13 19:44:27,173 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:44:27,173 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:44:27,173 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:44:27,173 Example #1\n", "2020-02-13 19:44:27,174 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:44:27,174 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:44:27,174 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:44:27,174 Example #2\n", "2020-02-13 19:44:27,174 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:44:27,174 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:44:27,174 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o mofubelu ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:44:27,174 Example #3\n", "2020-02-13 19:44:27,174 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:44:27,174 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:44:27,174 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa fihla leseling .\n", "2020-02-13 19:44:27,174 Validation result (greedy) at epoch 8, step 64000: bleu: 44.36, loss: 27382.6172, ppl: 2.9309, duration: 38.0678s\n", "2020-02-13 19:45:03,574 Epoch 8 Step: 64100 Batch Loss: 1.090608 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 19:45:40,628 Epoch 8 Step: 64200 Batch Loss: 1.101823 Tokens per Sec: 6703, Lr: 0.000300\n", "2020-02-13 19:46:16,932 Epoch 8 Step: 64300 Batch Loss: 1.323335 Tokens per Sec: 6777, Lr: 0.000300\n", "2020-02-13 19:46:52,985 Epoch 8 Step: 64400 Batch Loss: 1.210236 Tokens per Sec: 6562, Lr: 0.000300\n", "2020-02-13 19:47:29,231 Epoch 8 Step: 64500 Batch Loss: 1.194548 Tokens per Sec: 6654, Lr: 0.000300\n", "2020-02-13 19:48:05,927 Epoch 8 Step: 64600 Batch Loss: 1.025151 Tokens per Sec: 6764, Lr: 0.000300\n", "2020-02-13 19:48:42,330 Epoch 8 Step: 64700 Batch Loss: 1.084784 Tokens per Sec: 6775, Lr: 0.000300\n", "2020-02-13 19:49:18,978 Epoch 8 Step: 64800 Batch Loss: 1.199512 Tokens per Sec: 6659, Lr: 0.000300\n", "2020-02-13 19:49:55,791 Epoch 8 Step: 64900 Batch Loss: 1.098021 Tokens per Sec: 6541, Lr: 0.000300\n", "2020-02-13 19:50:32,423 Epoch 8 Step: 65000 Batch Loss: 1.264663 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 19:51:08,547 Example #0\n", "2020-02-13 19:51:08,548 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:51:08,548 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:51:08,548 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:51:08,548 Example #1\n", "2020-02-13 19:51:08,548 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:51:08,548 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:51:08,548 \tHypothesis: Ho feta moo , ha Jehova a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:51:08,548 Example #2\n", "2020-02-13 19:51:08,548 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:51:08,548 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:51:08,548 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:51:08,548 Example #3\n", "2020-02-13 19:51:08,549 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:51:08,549 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:51:08,549 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 19:51:08,549 Validation result (greedy) at epoch 8, step 65000: bleu: 44.25, loss: 27388.6367, ppl: 2.9316, duration: 36.1251s\n", "2020-02-13 19:51:45,174 Epoch 8 Step: 65100 Batch Loss: 1.150341 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 19:52:21,630 Epoch 8 Step: 65200 Batch Loss: 1.144006 Tokens per Sec: 6759, Lr: 0.000300\n", "2020-02-13 19:52:57,949 Epoch 8 Step: 65300 Batch Loss: 1.328417 Tokens per Sec: 6506, Lr: 0.000300\n", "2020-02-13 19:53:34,306 Epoch 8 Step: 65400 Batch Loss: 1.486273 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 19:54:10,833 Epoch 8 Step: 65500 Batch Loss: 1.271719 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 19:54:46,759 Epoch 8 Step: 65600 Batch Loss: 1.278054 Tokens per Sec: 6617, Lr: 0.000300\n", "2020-02-13 19:55:23,162 Epoch 8 Step: 65700 Batch Loss: 1.275582 Tokens per Sec: 6788, Lr: 0.000300\n", "2020-02-13 19:55:59,586 Epoch 8 Step: 65800 Batch Loss: 1.090104 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 19:56:36,260 Epoch 8 Step: 65900 Batch Loss: 1.191728 Tokens per Sec: 6744, Lr: 0.000300\n", "2020-02-13 19:57:12,872 Epoch 8 Step: 66000 Batch Loss: 1.200810 Tokens per Sec: 6700, Lr: 0.000300\n", "2020-02-13 19:57:48,939 Hooray! New best validation result [ppl]!\n", "2020-02-13 19:57:48,939 Saving new checkpoint.\n", "2020-02-13 19:57:49,421 Example #0\n", "2020-02-13 19:57:49,421 \tSource: Has the situation improved since then ?\n", "2020-02-13 19:57:49,421 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 19:57:49,421 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 19:57:49,421 Example #1\n", "2020-02-13 19:57:49,422 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 19:57:49,422 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 19:57:49,422 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 19:57:49,422 Example #2\n", "2020-02-13 19:57:49,422 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 19:57:49,422 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 19:57:49,422 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 19:57:49,422 Example #3\n", "2020-02-13 19:57:49,423 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 19:57:49,423 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 19:57:49,423 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa leseling .\n", "2020-02-13 19:57:49,423 Validation result (greedy) at epoch 8, step 66000: bleu: 44.52, loss: 27272.5898, ppl: 2.9182, duration: 36.5510s\n", "2020-02-13 19:58:25,701 Epoch 8 Step: 66100 Batch Loss: 1.187721 Tokens per Sec: 6805, Lr: 0.000300\n", "2020-02-13 19:59:02,450 Epoch 8 Step: 66200 Batch Loss: 1.149776 Tokens per Sec: 6671, Lr: 0.000300\n", "2020-02-13 19:59:38,774 Epoch 8 Step: 66300 Batch Loss: 1.164574 Tokens per Sec: 6626, Lr: 0.000300\n", "2020-02-13 20:00:15,163 Epoch 8 Step: 66400 Batch Loss: 1.069473 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 20:00:51,773 Epoch 8 Step: 66500 Batch Loss: 1.312275 Tokens per Sec: 6690, Lr: 0.000300\n", "2020-02-13 20:01:28,354 Epoch 8 Step: 66600 Batch Loss: 1.190803 Tokens per Sec: 6688, Lr: 0.000300\n", "2020-02-13 20:02:04,875 Epoch 8 Step: 66700 Batch Loss: 1.096692 Tokens per Sec: 6613, Lr: 0.000300\n", "2020-02-13 20:02:41,617 Epoch 8 Step: 66800 Batch Loss: 1.146086 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 20:03:17,831 Epoch 8 Step: 66900 Batch Loss: 1.233878 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 20:03:54,368 Epoch 8 Step: 67000 Batch Loss: 1.090676 Tokens per Sec: 6546, Lr: 0.000300\n", "2020-02-13 20:04:30,265 Example #0\n", "2020-02-13 20:04:30,265 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:04:30,265 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:04:30,266 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:04:30,266 Example #1\n", "2020-02-13 20:04:30,266 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:04:30,266 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:04:30,266 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:04:30,266 Example #2\n", "2020-02-13 20:04:30,266 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:04:30,266 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:04:30,266 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o bula , a hasanya masiba moeeng .\n", "2020-02-13 20:04:30,266 Example #3\n", "2020-02-13 20:04:30,266 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:04:30,266 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:04:30,266 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 20:04:30,267 Validation result (greedy) at epoch 8, step 67000: bleu: 44.43, loss: 27316.3574, ppl: 2.9233, duration: 35.8980s\n", "2020-02-13 20:05:06,724 Epoch 8 Step: 67100 Batch Loss: 1.251863 Tokens per Sec: 6698, Lr: 0.000300\n", "2020-02-13 20:05:43,481 Epoch 8 Step: 67200 Batch Loss: 1.236130 Tokens per Sec: 6678, Lr: 0.000300\n", "2020-02-13 20:06:19,812 Epoch 8 Step: 67300 Batch Loss: 1.181517 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 20:06:56,361 Epoch 8 Step: 67400 Batch Loss: 1.043763 Tokens per Sec: 6797, Lr: 0.000300\n", "2020-02-13 20:07:33,231 Epoch 8 Step: 67500 Batch Loss: 1.127803 Tokens per Sec: 6614, Lr: 0.000300\n", "2020-02-13 20:08:09,250 Epoch 8 Step: 67600 Batch Loss: 1.244801 Tokens per Sec: 6664, Lr: 0.000300\n", "2020-02-13 20:08:45,545 Epoch 8 Step: 67700 Batch Loss: 1.153154 Tokens per Sec: 6678, Lr: 0.000300\n", "2020-02-13 20:09:21,664 Epoch 8 Step: 67800 Batch Loss: 1.070777 Tokens per Sec: 6725, Lr: 0.000300\n", "2020-02-13 20:09:58,362 Epoch 8 Step: 67900 Batch Loss: 1.188867 Tokens per Sec: 6779, Lr: 0.000300\n", "2020-02-13 20:10:35,077 Epoch 8 Step: 68000 Batch Loss: 1.175206 Tokens per Sec: 6688, Lr: 0.000300\n", "2020-02-13 20:11:11,347 Hooray! New best validation result [ppl]!\n", "2020-02-13 20:11:11,347 Saving new checkpoint.\n", "2020-02-13 20:11:11,770 Example #0\n", "2020-02-13 20:11:11,771 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:11:11,771 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:11:11,771 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:11:11,771 Example #1\n", "2020-02-13 20:11:11,771 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:11:11,771 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:11:11,771 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:11:11,771 Example #2\n", "2020-02-13 20:11:11,772 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:11:11,772 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:11:11,772 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 20:11:11,772 Example #3\n", "2020-02-13 20:11:11,772 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:11:11,772 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:11:11,772 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 20:11:11,772 Validation result (greedy) at epoch 8, step 68000: bleu: 44.58, loss: 27134.1191, ppl: 2.9024, duration: 36.6945s\n", "2020-02-13 20:11:48,130 Epoch 8 Step: 68100 Batch Loss: 1.024865 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 20:12:24,664 Epoch 8 Step: 68200 Batch Loss: 1.205576 Tokens per Sec: 6709, Lr: 0.000300\n", "2020-02-13 20:13:01,028 Epoch 8 Step: 68300 Batch Loss: 1.275324 Tokens per Sec: 6689, Lr: 0.000300\n", "2020-02-13 20:13:37,612 Epoch 8 Step: 68400 Batch Loss: 1.049008 Tokens per Sec: 6763, Lr: 0.000300\n", "2020-02-13 20:14:13,934 Epoch 8 Step: 68500 Batch Loss: 1.291912 Tokens per Sec: 6643, Lr: 0.000300\n", "2020-02-13 20:14:50,574 Epoch 8 Step: 68600 Batch Loss: 1.254712 Tokens per Sec: 6628, Lr: 0.000300\n", "2020-02-13 20:15:26,875 Epoch 8 Step: 68700 Batch Loss: 1.147279 Tokens per Sec: 6704, Lr: 0.000300\n", "2020-02-13 20:16:03,089 Epoch 8 Step: 68800 Batch Loss: 1.149553 Tokens per Sec: 6576, Lr: 0.000300\n", "2020-02-13 20:16:39,675 Epoch 8 Step: 68900 Batch Loss: 1.311898 Tokens per Sec: 6754, Lr: 0.000300\n", "2020-02-13 20:17:15,851 Epoch 8 Step: 69000 Batch Loss: 1.339587 Tokens per Sec: 6601, Lr: 0.000300\n", "2020-02-13 20:17:50,929 Example #0\n", "2020-02-13 20:17:50,929 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:17:50,929 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:17:50,929 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:17:50,929 Example #1\n", "2020-02-13 20:17:50,929 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:17:50,929 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:17:50,929 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:17:50,929 Example #2\n", "2020-02-13 20:17:50,930 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:17:50,930 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:17:50,930 \tHypothesis: Monna ea bohlale o ile a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasa masiba moeeng .\n", "2020-02-13 20:17:50,930 Example #3\n", "2020-02-13 20:17:50,930 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:17:50,930 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:17:50,930 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 20:17:50,930 Validation result (greedy) at epoch 8, step 69000: bleu: 43.97, loss: 27141.8398, ppl: 2.9033, duration: 35.0792s\n", "2020-02-13 20:18:27,343 Epoch 8 Step: 69100 Batch Loss: 1.879675 Tokens per Sec: 6786, Lr: 0.000300\n", "2020-02-13 20:19:04,267 Epoch 8 Step: 69200 Batch Loss: 1.305182 Tokens per Sec: 6657, Lr: 0.000300\n", "2020-02-13 20:19:40,581 Epoch 8 Step: 69300 Batch Loss: 1.220556 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 20:20:17,248 Epoch 8 Step: 69400 Batch Loss: 1.163824 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 20:20:54,103 Epoch 8 Step: 69500 Batch Loss: 1.221807 Tokens per Sec: 6724, Lr: 0.000300\n", "2020-02-13 20:21:30,685 Epoch 8 Step: 69600 Batch Loss: 1.122687 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 20:22:07,288 Epoch 8 Step: 69700 Batch Loss: 1.165181 Tokens per Sec: 6644, Lr: 0.000300\n", "2020-02-13 20:22:23,604 Epoch 8: total training loss 10352.72\n", "2020-02-13 20:22:23,605 EPOCH 9\n", "2020-02-13 20:22:44,698 Epoch 9 Step: 69800 Batch Loss: 1.057369 Tokens per Sec: 6393, Lr: 0.000300\n", "2020-02-13 20:23:21,124 Epoch 9 Step: 69900 Batch Loss: 1.221510 Tokens per Sec: 6759, Lr: 0.000300\n", "2020-02-13 20:23:57,487 Epoch 9 Step: 70000 Batch Loss: 1.292441 Tokens per Sec: 6621, Lr: 0.000300\n", "2020-02-13 20:24:34,796 Example #0\n", "2020-02-13 20:24:34,797 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:24:34,797 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:24:34,797 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:24:34,797 Example #1\n", "2020-02-13 20:24:34,797 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:24:34,797 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:24:34,797 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:24:34,797 Example #2\n", "2020-02-13 20:24:34,797 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:24:34,797 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:24:34,797 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 20:24:34,797 Example #3\n", "2020-02-13 20:24:34,798 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:24:34,798 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:24:34,798 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 20:24:34,798 Validation result (greedy) at epoch 9, step 70000: bleu: 44.22, loss: 27172.2812, ppl: 2.9068, duration: 37.3108s\n", "2020-02-13 20:25:11,413 Epoch 9 Step: 70100 Batch Loss: 1.244911 Tokens per Sec: 6631, Lr: 0.000300\n", "2020-02-13 20:25:47,880 Epoch 9 Step: 70200 Batch Loss: 1.088070 Tokens per Sec: 6639, Lr: 0.000300\n", "2020-02-13 20:26:24,249 Epoch 9 Step: 70300 Batch Loss: 1.087914 Tokens per Sec: 6732, Lr: 0.000300\n", "2020-02-13 20:27:00,610 Epoch 9 Step: 70400 Batch Loss: 1.211620 Tokens per Sec: 6592, Lr: 0.000300\n", "2020-02-13 20:27:37,517 Epoch 9 Step: 70500 Batch Loss: 1.143860 Tokens per Sec: 6760, Lr: 0.000300\n", "2020-02-13 20:28:14,141 Epoch 9 Step: 70600 Batch Loss: 1.085271 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 20:28:50,607 Epoch 9 Step: 70700 Batch Loss: 1.131836 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 20:29:26,691 Epoch 9 Step: 70800 Batch Loss: 1.356751 Tokens per Sec: 6610, Lr: 0.000300\n", "2020-02-13 20:30:03,504 Epoch 9 Step: 70900 Batch Loss: 1.025975 Tokens per Sec: 6756, Lr: 0.000300\n", "2020-02-13 20:30:39,393 Epoch 9 Step: 71000 Batch Loss: 1.060901 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 20:31:16,498 Example #0\n", "2020-02-13 20:31:16,499 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:31:16,499 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:31:16,499 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:31:16,499 Example #1\n", "2020-02-13 20:31:16,499 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:31:16,499 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:31:16,499 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:31:16,499 Example #2\n", "2020-02-13 20:31:16,499 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:31:16,499 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:31:16,499 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 20:31:16,499 Example #3\n", "2020-02-13 20:31:16,499 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:31:16,499 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:31:16,500 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 20:31:16,500 Validation result (greedy) at epoch 9, step 71000: bleu: 44.12, loss: 27257.7012, ppl: 2.9165, duration: 37.1059s\n", "2020-02-13 20:31:53,105 Epoch 9 Step: 71100 Batch Loss: 1.065543 Tokens per Sec: 6747, Lr: 0.000300\n", "2020-02-13 20:32:29,663 Epoch 9 Step: 71200 Batch Loss: 1.310634 Tokens per Sec: 6576, Lr: 0.000300\n", "2020-02-13 20:33:06,390 Epoch 9 Step: 71300 Batch Loss: 1.014558 Tokens per Sec: 6718, Lr: 0.000300\n", "2020-02-13 20:33:42,920 Epoch 9 Step: 71400 Batch Loss: 1.215661 Tokens per Sec: 6722, Lr: 0.000300\n", "2020-02-13 20:34:19,537 Epoch 9 Step: 71500 Batch Loss: 1.124036 Tokens per Sec: 6616, Lr: 0.000300\n", "2020-02-13 20:34:55,987 Epoch 9 Step: 71600 Batch Loss: 1.203260 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 20:35:32,805 Epoch 9 Step: 71700 Batch Loss: 1.105157 Tokens per Sec: 6769, Lr: 0.000300\n", "2020-02-13 20:36:09,246 Epoch 9 Step: 71800 Batch Loss: 1.172253 Tokens per Sec: 6630, Lr: 0.000300\n", "2020-02-13 20:36:45,663 Epoch 9 Step: 71900 Batch Loss: 1.273720 Tokens per Sec: 6739, Lr: 0.000300\n", "2020-02-13 20:37:22,256 Epoch 9 Step: 72000 Batch Loss: 1.206483 Tokens per Sec: 6595, Lr: 0.000300\n", "2020-02-13 20:37:58,898 Hooray! New best validation result [ppl]!\n", "2020-02-13 20:37:58,898 Saving new checkpoint.\n", "2020-02-13 20:37:59,388 Example #0\n", "2020-02-13 20:37:59,388 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:37:59,388 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:37:59,388 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:37:59,389 Example #1\n", "2020-02-13 20:37:59,389 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:37:59,389 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:37:59,389 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:37:59,389 Example #2\n", "2020-02-13 20:37:59,389 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:37:59,389 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:37:59,389 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 20:37:59,389 Example #3\n", "2020-02-13 20:37:59,389 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:37:59,389 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:37:59,389 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 20:37:59,389 Validation result (greedy) at epoch 9, step 72000: bleu: 44.36, loss: 27107.9570, ppl: 2.8994, duration: 37.1333s\n", "2020-02-13 20:38:36,096 Epoch 9 Step: 72100 Batch Loss: 1.213004 Tokens per Sec: 6808, Lr: 0.000300\n", "2020-02-13 20:39:12,904 Epoch 9 Step: 72200 Batch Loss: 1.211235 Tokens per Sec: 6767, Lr: 0.000300\n", "2020-02-13 20:39:49,379 Epoch 9 Step: 72300 Batch Loss: 1.179778 Tokens per Sec: 6619, Lr: 0.000300\n", "2020-02-13 20:40:25,519 Epoch 9 Step: 72400 Batch Loss: 1.174392 Tokens per Sec: 6677, Lr: 0.000300\n", "2020-02-13 20:41:02,174 Epoch 9 Step: 72500 Batch Loss: 1.155512 Tokens per Sec: 6744, Lr: 0.000300\n", "2020-02-13 20:41:38,701 Epoch 9 Step: 72600 Batch Loss: 1.276302 Tokens per Sec: 6725, Lr: 0.000300\n", "2020-02-13 20:42:14,899 Epoch 9 Step: 72700 Batch Loss: 1.126806 Tokens per Sec: 6713, Lr: 0.000300\n", "2020-02-13 20:42:51,875 Epoch 9 Step: 72800 Batch Loss: 1.121090 Tokens per Sec: 6737, Lr: 0.000300\n", "2020-02-13 20:43:28,492 Epoch 9 Step: 72900 Batch Loss: 1.074873 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 20:44:05,144 Epoch 9 Step: 73000 Batch Loss: 1.059863 Tokens per Sec: 6796, Lr: 0.000300\n", "2020-02-13 20:44:41,784 Hooray! New best validation result [ppl]!\n", "2020-02-13 20:44:41,784 Saving new checkpoint.\n", "2020-02-13 20:44:42,387 Example #0\n", "2020-02-13 20:44:42,387 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:44:42,387 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:44:42,387 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:44:42,387 Example #1\n", "2020-02-13 20:44:42,387 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:44:42,387 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:44:42,388 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:44:42,388 Example #2\n", "2020-02-13 20:44:42,388 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:44:42,388 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:44:42,388 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a qhala masiba moeeng .\n", "2020-02-13 20:44:42,388 Example #3\n", "2020-02-13 20:44:42,388 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:44:42,388 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:44:42,388 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 20:44:42,388 Validation result (greedy) at epoch 9, step 73000: bleu: 44.53, loss: 26895.1406, ppl: 2.8753, duration: 37.2443s\n", "2020-02-13 20:45:18,723 Epoch 9 Step: 73100 Batch Loss: 1.202709 Tokens per Sec: 6693, Lr: 0.000300\n", "2020-02-13 20:45:54,736 Epoch 9 Step: 73200 Batch Loss: 1.120478 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 20:46:31,279 Epoch 9 Step: 73300 Batch Loss: 1.185031 Tokens per Sec: 6622, Lr: 0.000300\n", "2020-02-13 20:47:07,884 Epoch 9 Step: 73400 Batch Loss: 1.143274 Tokens per Sec: 6688, Lr: 0.000300\n", "2020-02-13 20:47:44,708 Epoch 9 Step: 73500 Batch Loss: 1.021330 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 20:48:20,900 Epoch 9 Step: 73600 Batch Loss: 1.069010 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 20:48:57,063 Epoch 9 Step: 73700 Batch Loss: 1.168250 Tokens per Sec: 6721, Lr: 0.000300\n", "2020-02-13 20:49:33,927 Epoch 9 Step: 73800 Batch Loss: 1.120752 Tokens per Sec: 6589, Lr: 0.000300\n", "2020-02-13 20:50:10,631 Epoch 9 Step: 73900 Batch Loss: 1.135237 Tokens per Sec: 6716, Lr: 0.000300\n", "2020-02-13 20:50:47,384 Epoch 9 Step: 74000 Batch Loss: 1.196277 Tokens per Sec: 6591, Lr: 0.000300\n", "2020-02-13 20:51:23,164 Example #0\n", "2020-02-13 20:51:23,165 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:51:23,165 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:51:23,165 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:51:23,165 Example #1\n", "2020-02-13 20:51:23,165 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:51:23,165 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:51:23,165 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:51:23,165 Example #2\n", "2020-02-13 20:51:23,165 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:51:23,165 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:51:23,165 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o motle ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 20:51:23,165 Example #3\n", "2020-02-13 20:51:23,166 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:51:23,166 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:51:23,166 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 20:51:23,166 Validation result (greedy) at epoch 9, step 74000: bleu: 44.61, loss: 26977.7051, ppl: 2.8846, duration: 35.7812s\n", "2020-02-13 20:51:59,848 Epoch 9 Step: 74100 Batch Loss: 1.276710 Tokens per Sec: 6807, Lr: 0.000300\n", "2020-02-13 20:52:36,413 Epoch 9 Step: 74200 Batch Loss: 1.124421 Tokens per Sec: 6736, Lr: 0.000300\n", "2020-02-13 20:53:12,882 Epoch 9 Step: 74300 Batch Loss: 1.200625 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 20:53:49,639 Epoch 9 Step: 74400 Batch Loss: 1.271102 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 20:54:26,085 Epoch 9 Step: 74500 Batch Loss: 1.310334 Tokens per Sec: 6830, Lr: 0.000300\n", "2020-02-13 20:55:02,639 Epoch 9 Step: 74600 Batch Loss: 1.335460 Tokens per Sec: 6644, Lr: 0.000300\n", "2020-02-13 20:55:39,339 Epoch 9 Step: 74700 Batch Loss: 1.202090 Tokens per Sec: 6641, Lr: 0.000300\n", "2020-02-13 20:56:16,019 Epoch 9 Step: 74800 Batch Loss: 1.203300 Tokens per Sec: 6693, Lr: 0.000300\n", "2020-02-13 20:56:52,583 Epoch 9 Step: 74900 Batch Loss: 1.149651 Tokens per Sec: 6706, Lr: 0.000300\n", "2020-02-13 20:57:29,238 Epoch 9 Step: 75000 Batch Loss: 1.191752 Tokens per Sec: 6681, Lr: 0.000300\n", "2020-02-13 20:58:06,100 Hooray! New best validation result [ppl]!\n", "2020-02-13 20:58:06,100 Saving new checkpoint.\n", "2020-02-13 20:58:06,515 Example #0\n", "2020-02-13 20:58:06,515 \tSource: Has the situation improved since then ?\n", "2020-02-13 20:58:06,515 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 20:58:06,515 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 20:58:06,516 Example #1\n", "2020-02-13 20:58:06,516 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 20:58:06,516 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 20:58:06,516 \tHypothesis: Ho feta moo , ha Jehova a ne a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 20:58:06,516 Example #2\n", "2020-02-13 20:58:06,516 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 20:58:06,516 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 20:58:06,516 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Motho ea sebetsoang o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o kuta , a hasanya masiba a moea .\n", "2020-02-13 20:58:06,516 Example #3\n", "2020-02-13 20:58:06,516 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 20:58:06,516 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 20:58:06,516 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li fihlile leseling .\n", "2020-02-13 20:58:06,516 Validation result (greedy) at epoch 9, step 75000: bleu: 44.61, loss: 26818.1465, ppl: 2.8666, duration: 37.2783s\n", "2020-02-13 20:58:42,755 Epoch 9 Step: 75100 Batch Loss: 1.226291 Tokens per Sec: 6658, Lr: 0.000300\n", "2020-02-13 20:59:19,562 Epoch 9 Step: 75200 Batch Loss: 1.145201 Tokens per Sec: 6588, Lr: 0.000300\n", "2020-02-13 20:59:55,998 Epoch 9 Step: 75300 Batch Loss: 1.206741 Tokens per Sec: 6774, Lr: 0.000300\n", "2020-02-13 21:00:32,568 Epoch 9 Step: 75400 Batch Loss: 1.127421 Tokens per Sec: 6736, Lr: 0.000300\n", "2020-02-13 21:01:08,751 Epoch 9 Step: 75500 Batch Loss: 1.085973 Tokens per Sec: 6581, Lr: 0.000300\n", "2020-02-13 21:01:45,427 Epoch 9 Step: 75600 Batch Loss: 1.328040 Tokens per Sec: 6604, Lr: 0.000300\n", "2020-02-13 21:02:21,965 Epoch 9 Step: 75700 Batch Loss: 1.176840 Tokens per Sec: 6738, Lr: 0.000300\n", "2020-02-13 21:02:58,442 Epoch 9 Step: 75800 Batch Loss: 1.244229 Tokens per Sec: 6710, Lr: 0.000300\n", "2020-02-13 21:03:34,816 Epoch 9 Step: 75900 Batch Loss: 1.257721 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 21:04:11,437 Epoch 9 Step: 76000 Batch Loss: 1.117521 Tokens per Sec: 6627, Lr: 0.000300\n", "2020-02-13 21:04:49,203 Hooray! New best validation result [ppl]!\n", "2020-02-13 21:04:49,204 Saving new checkpoint.\n", "2020-02-13 21:04:49,659 Example #0\n", "2020-02-13 21:04:49,660 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:04:49,660 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:04:49,660 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:04:49,660 Example #1\n", "2020-02-13 21:04:49,660 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:04:49,660 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:04:49,660 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:04:49,660 Example #2\n", "2020-02-13 21:04:49,660 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:04:49,660 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:04:49,660 \tHypothesis: Monna ea bohlale o ile a kōpa : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 21:04:49,661 Example #3\n", "2020-02-13 21:04:49,661 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:04:49,661 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:04:49,662 \tHypothesis: Ho tloha ka nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:04:49,662 Validation result (greedy) at epoch 9, step 76000: bleu: 44.65, loss: 26784.3711, ppl: 2.8628, duration: 38.2247s\n", "2020-02-13 21:05:26,301 Epoch 9 Step: 76100 Batch Loss: 1.130642 Tokens per Sec: 6697, Lr: 0.000300\n", "2020-02-13 21:06:02,633 Epoch 9 Step: 76200 Batch Loss: 1.091443 Tokens per Sec: 6590, Lr: 0.000300\n", "2020-02-13 21:06:38,966 Epoch 9 Step: 76300 Batch Loss: 1.287957 Tokens per Sec: 6722, Lr: 0.000300\n", "2020-02-13 21:07:15,511 Epoch 9 Step: 76400 Batch Loss: 1.240031 Tokens per Sec: 6711, Lr: 0.000300\n", "2020-02-13 21:07:52,259 Epoch 9 Step: 76500 Batch Loss: 0.967742 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 21:08:28,467 Epoch 9 Step: 76600 Batch Loss: 1.212625 Tokens per Sec: 6568, Lr: 0.000300\n", "2020-02-13 21:09:05,157 Epoch 9 Step: 76700 Batch Loss: 1.379592 Tokens per Sec: 6534, Lr: 0.000300\n", "2020-02-13 21:09:41,875 Epoch 9 Step: 76800 Batch Loss: 1.348656 Tokens per Sec: 6702, Lr: 0.000300\n", "2020-02-13 21:10:18,599 Epoch 9 Step: 76900 Batch Loss: 1.302954 Tokens per Sec: 6668, Lr: 0.000300\n", "2020-02-13 21:10:55,124 Epoch 9 Step: 77000 Batch Loss: 1.121680 Tokens per Sec: 6764, Lr: 0.000300\n", "2020-02-13 21:11:32,434 Hooray! New best validation result [ppl]!\n", "2020-02-13 21:11:32,434 Saving new checkpoint.\n", "2020-02-13 21:11:32,919 Example #0\n", "2020-02-13 21:11:32,919 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:11:32,919 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:11:32,919 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:11:32,919 Example #1\n", "2020-02-13 21:11:32,920 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:11:32,920 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:11:32,920 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:11:32,920 Example #2\n", "2020-02-13 21:11:32,920 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:11:32,920 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:11:32,920 \tHypothesis: Monna ea bohlale o ne a kōpa ka tieo : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasa masiba moeeng .\n", "2020-02-13 21:11:32,920 Example #3\n", "2020-02-13 21:11:32,920 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:11:32,920 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:11:32,920 \tHypothesis: Kaha nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:11:32,920 Validation result (greedy) at epoch 9, step 77000: bleu: 44.97, loss: 26617.1992, ppl: 2.8441, duration: 37.7963s\n", "2020-02-13 21:12:09,449 Epoch 9 Step: 77100 Batch Loss: 1.077489 Tokens per Sec: 6696, Lr: 0.000300\n", "2020-02-13 21:12:46,086 Epoch 9 Step: 77200 Batch Loss: 1.118826 Tokens per Sec: 6626, Lr: 0.000300\n", "2020-02-13 21:13:22,455 Epoch 9 Step: 77300 Batch Loss: 1.157404 Tokens per Sec: 6695, Lr: 0.000300\n", "2020-02-13 21:13:59,063 Epoch 9 Step: 77400 Batch Loss: 1.125489 Tokens per Sec: 6758, Lr: 0.000300\n", "2020-02-13 21:14:35,425 Epoch 9 Step: 77500 Batch Loss: 1.179061 Tokens per Sec: 6652, Lr: 0.000300\n", "2020-02-13 21:15:11,494 Epoch 9 Step: 77600 Batch Loss: 1.217628 Tokens per Sec: 6599, Lr: 0.000300\n", "2020-02-13 21:15:48,291 Epoch 9 Step: 77700 Batch Loss: 1.251714 Tokens per Sec: 6816, Lr: 0.000300\n", "2020-02-13 21:16:24,884 Epoch 9 Step: 77800 Batch Loss: 1.257298 Tokens per Sec: 6618, Lr: 0.000300\n", "2020-02-13 21:17:01,211 Epoch 9 Step: 77900 Batch Loss: 1.077361 Tokens per Sec: 6711, Lr: 0.000300\n", "2020-02-13 21:17:37,485 Epoch 9 Step: 78000 Batch Loss: 1.249065 Tokens per Sec: 6690, Lr: 0.000300\n", "2020-02-13 21:18:14,854 Example #0\n", "2020-02-13 21:18:14,855 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:18:14,855 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:18:14,855 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:18:14,855 Example #1\n", "2020-02-13 21:18:14,855 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:18:14,855 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:18:14,855 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:18:14,855 Example #2\n", "2020-02-13 21:18:14,855 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:18:14,855 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:18:14,856 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo oa masiba ’ me a o kuta , a qhalla masiba moeeng .\n", "2020-02-13 21:18:14,856 Example #3\n", "2020-02-13 21:18:14,856 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:18:14,856 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:18:14,856 \tHypothesis: Ho tloha ha Tischendorf a ne a le teng , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:18:14,856 Validation result (greedy) at epoch 9, step 78000: bleu: 44.44, loss: 26725.0742, ppl: 2.8562, duration: 37.3679s\n", "2020-02-13 21:18:51,420 Epoch 9 Step: 78100 Batch Loss: 1.205366 Tokens per Sec: 6708, Lr: 0.000300\n", "2020-02-13 21:19:28,104 Epoch 9 Step: 78200 Batch Loss: 1.101239 Tokens per Sec: 6656, Lr: 0.000300\n", "2020-02-13 21:20:04,826 Epoch 9 Step: 78300 Batch Loss: 1.058297 Tokens per Sec: 6668, Lr: 0.000300\n", "2020-02-13 21:20:41,384 Epoch 9 Step: 78400 Batch Loss: 1.341530 Tokens per Sec: 6550, Lr: 0.000300\n", "2020-02-13 21:20:59,851 Epoch 9: total training loss 10111.11\n", "2020-02-13 21:20:59,851 EPOCH 10\n", "2020-02-13 21:21:18,407 Epoch 10 Step: 78500 Batch Loss: 1.170076 Tokens per Sec: 6233, Lr: 0.000300\n", "2020-02-13 21:21:54,560 Epoch 10 Step: 78600 Batch Loss: 1.135146 Tokens per Sec: 6592, Lr: 0.000300\n", "2020-02-13 21:22:31,164 Epoch 10 Step: 78700 Batch Loss: 1.166844 Tokens per Sec: 6785, Lr: 0.000300\n", "2020-02-13 21:23:07,663 Epoch 10 Step: 78800 Batch Loss: 1.219411 Tokens per Sec: 6645, Lr: 0.000300\n", "2020-02-13 21:23:44,227 Epoch 10 Step: 78900 Batch Loss: 1.154973 Tokens per Sec: 6695, Lr: 0.000300\n", "2020-02-13 21:24:20,813 Epoch 10 Step: 79000 Batch Loss: 1.210241 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 21:24:58,133 Hooray! New best validation result [ppl]!\n", "2020-02-13 21:24:58,133 Saving new checkpoint.\n", "2020-02-13 21:24:58,569 Example #0\n", "2020-02-13 21:24:58,569 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:24:58,569 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:24:58,569 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:24:58,569 Example #1\n", "2020-02-13 21:24:58,569 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:24:58,569 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:24:58,570 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:24:58,570 Example #2\n", "2020-02-13 21:24:58,570 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:24:58,570 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:24:58,570 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 21:24:58,570 Example #3\n", "2020-02-13 21:24:58,570 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:24:58,570 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:24:58,570 \tHypothesis: Ho tloha nakong ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:24:58,570 Validation result (greedy) at epoch 10, step 79000: bleu: 45.00, loss: 26483.3848, ppl: 2.8292, duration: 37.7565s\n", "2020-02-13 21:25:35,292 Epoch 10 Step: 79100 Batch Loss: 1.260976 Tokens per Sec: 6795, Lr: 0.000300\n", "2020-02-13 21:26:11,816 Epoch 10 Step: 79200 Batch Loss: 1.220501 Tokens per Sec: 6735, Lr: 0.000300\n", "2020-02-13 21:26:48,377 Epoch 10 Step: 79300 Batch Loss: 1.291631 Tokens per Sec: 6686, Lr: 0.000300\n", "2020-02-13 21:27:24,561 Epoch 10 Step: 79400 Batch Loss: 1.171741 Tokens per Sec: 6687, Lr: 0.000300\n", "2020-02-13 21:28:00,825 Epoch 10 Step: 79500 Batch Loss: 1.173795 Tokens per Sec: 6581, Lr: 0.000300\n", "2020-02-13 21:28:37,072 Epoch 10 Step: 79600 Batch Loss: 1.072519 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 21:29:13,746 Epoch 10 Step: 79700 Batch Loss: 1.287936 Tokens per Sec: 6787, Lr: 0.000300\n", "2020-02-13 21:29:50,230 Epoch 10 Step: 79800 Batch Loss: 1.092189 Tokens per Sec: 6638, Lr: 0.000300\n", "2020-02-13 21:30:26,478 Epoch 10 Step: 79900 Batch Loss: 1.074543 Tokens per Sec: 6731, Lr: 0.000300\n", "2020-02-13 21:31:02,992 Epoch 10 Step: 80000 Batch Loss: 1.073415 Tokens per Sec: 6720, Lr: 0.000300\n", "2020-02-13 21:31:39,956 Example #0\n", "2020-02-13 21:31:39,957 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:31:39,957 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:31:39,957 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:31:39,957 Example #1\n", "2020-02-13 21:31:39,957 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:31:39,957 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:31:39,957 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:31:39,957 Example #2\n", "2020-02-13 21:31:39,957 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:31:39,957 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:31:39,957 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o kuta , a hasanya masiba moeeng .\n", "2020-02-13 21:31:39,957 Example #3\n", "2020-02-13 21:31:39,957 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:31:39,957 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:31:39,958 \tHypothesis: Kaha nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa tla leseling .\n", "2020-02-13 21:31:39,958 Validation result (greedy) at epoch 10, step 80000: bleu: 44.92, loss: 26543.1914, ppl: 2.8358, duration: 36.9651s\n", "2020-02-13 21:32:16,263 Epoch 10 Step: 80100 Batch Loss: 1.132336 Tokens per Sec: 6620, Lr: 0.000300\n", "2020-02-13 21:32:52,689 Epoch 10 Step: 80200 Batch Loss: 1.309429 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 21:33:28,913 Epoch 10 Step: 80300 Batch Loss: 1.093549 Tokens per Sec: 6694, Lr: 0.000300\n", "2020-02-13 21:34:05,098 Epoch 10 Step: 80400 Batch Loss: 1.073271 Tokens per Sec: 6565, Lr: 0.000300\n", "2020-02-13 21:34:41,696 Epoch 10 Step: 80500 Batch Loss: 1.234977 Tokens per Sec: 6661, Lr: 0.000300\n", "2020-02-13 21:35:17,970 Epoch 10 Step: 80600 Batch Loss: 1.152104 Tokens per Sec: 6522, Lr: 0.000300\n", "2020-02-13 21:35:54,329 Epoch 10 Step: 80700 Batch Loss: 0.999964 Tokens per Sec: 6637, Lr: 0.000300\n", "2020-02-13 21:36:30,314 Epoch 10 Step: 80800 Batch Loss: 1.076617 Tokens per Sec: 6682, Lr: 0.000300\n", "2020-02-13 21:37:06,580 Epoch 10 Step: 80900 Batch Loss: 1.023519 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 21:37:43,248 Epoch 10 Step: 81000 Batch Loss: 1.145464 Tokens per Sec: 6550, Lr: 0.000300\n", "2020-02-13 21:38:20,599 Example #0\n", "2020-02-13 21:38:20,599 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:38:20,599 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:38:20,599 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:38:20,599 Example #1\n", "2020-02-13 21:38:20,600 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:38:20,600 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:38:20,600 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:38:20,600 Example #2\n", "2020-02-13 21:38:20,600 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:38:20,600 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:38:20,600 \tHypothesis: Monna ea bohlale o ne a kōpa ka ho le leng : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo oa masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 21:38:20,600 Example #3\n", "2020-02-13 21:38:20,600 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:38:20,600 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:38:20,600 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:38:20,600 Validation result (greedy) at epoch 10, step 81000: bleu: 44.89, loss: 26605.9434, ppl: 2.8428, duration: 37.3520s\n", "2020-02-13 21:38:57,004 Epoch 10 Step: 81100 Batch Loss: 1.210611 Tokens per Sec: 6715, Lr: 0.000300\n", "2020-02-13 21:39:34,109 Epoch 10 Step: 81200 Batch Loss: 1.015399 Tokens per Sec: 6709, Lr: 0.000300\n", "2020-02-13 21:40:10,444 Epoch 10 Step: 81300 Batch Loss: 1.273607 Tokens per Sec: 6575, Lr: 0.000300\n", "2020-02-13 21:40:47,125 Epoch 10 Step: 81400 Batch Loss: 1.064003 Tokens per Sec: 6683, Lr: 0.000300\n", "2020-02-13 21:41:23,934 Epoch 10 Step: 81500 Batch Loss: 1.120895 Tokens per Sec: 6622, Lr: 0.000300\n", "2020-02-13 21:42:00,731 Epoch 10 Step: 81600 Batch Loss: 1.159207 Tokens per Sec: 6723, Lr: 0.000300\n", "2020-02-13 21:42:36,965 Epoch 10 Step: 81700 Batch Loss: 1.197898 Tokens per Sec: 6676, Lr: 0.000300\n", "2020-02-13 21:43:13,644 Epoch 10 Step: 81800 Batch Loss: 1.247367 Tokens per Sec: 6672, Lr: 0.000300\n", "2020-02-13 21:43:50,280 Epoch 10 Step: 81900 Batch Loss: 1.226434 Tokens per Sec: 6707, Lr: 0.000300\n", "2020-02-13 21:44:27,040 Epoch 10 Step: 82000 Batch Loss: 1.142739 Tokens per Sec: 6636, Lr: 0.000300\n", "2020-02-13 21:45:07,382 Example #0\n", "2020-02-13 21:45:07,382 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:45:07,382 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:45:07,382 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:45:07,382 Example #1\n", "2020-02-13 21:45:07,382 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:45:07,382 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:45:07,382 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:45:07,382 Example #2\n", "2020-02-13 21:45:07,383 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:45:07,383 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:45:07,383 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo o masiba ’ me a o kuta , a hasanya masiba a moea .\n", "2020-02-13 21:45:07,383 Example #3\n", "2020-02-13 21:45:07,383 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:45:07,383 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:45:07,383 \tHypothesis: Kaha nako ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:45:07,383 Validation result (greedy) at epoch 10, step 82000: bleu: 44.84, loss: 26773.8320, ppl: 2.8616, duration: 40.3429s\n", "2020-02-13 21:45:44,065 Epoch 10 Step: 82100 Batch Loss: 1.082335 Tokens per Sec: 6685, Lr: 0.000300\n", "2020-02-13 21:46:20,933 Epoch 10 Step: 82200 Batch Loss: 1.084700 Tokens per Sec: 6660, Lr: 0.000300\n", "2020-02-13 21:46:57,602 Epoch 10 Step: 82300 Batch Loss: 1.164224 Tokens per Sec: 6646, Lr: 0.000300\n", "2020-02-13 21:47:34,800 Epoch 10 Step: 82400 Batch Loss: 1.108893 Tokens per Sec: 6728, Lr: 0.000300\n", "2020-02-13 21:48:11,610 Epoch 10 Step: 82500 Batch Loss: 1.154751 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 21:48:48,449 Epoch 10 Step: 82600 Batch Loss: 1.151536 Tokens per Sec: 6565, Lr: 0.000300\n", "2020-02-13 21:49:25,187 Epoch 10 Step: 82700 Batch Loss: 1.125433 Tokens per Sec: 6581, Lr: 0.000300\n", "2020-02-13 21:50:02,200 Epoch 10 Step: 82800 Batch Loss: 0.996322 Tokens per Sec: 6640, Lr: 0.000300\n", "2020-02-13 21:50:38,613 Epoch 10 Step: 82900 Batch Loss: 1.254086 Tokens per Sec: 6563, Lr: 0.000300\n", "2020-02-13 21:51:15,466 Epoch 10 Step: 83000 Batch Loss: 0.964183 Tokens per Sec: 6511, Lr: 0.000300\n", "2020-02-13 21:51:51,196 Example #0\n", "2020-02-13 21:51:51,197 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:51:51,197 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:51:51,197 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:51:51,197 Example #1\n", "2020-02-13 21:51:51,197 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:51:51,197 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:51:51,197 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:51:51,197 Example #2\n", "2020-02-13 21:51:51,197 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:51:51,197 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:51:51,198 \tHypothesis: Monna ea bohlale o ne a kōpa : Mosebi o ile a bolelloa hore a tsamaee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 21:51:51,198 Example #3\n", "2020-02-13 21:51:51,198 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:51:51,198 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:51:51,198 \tHypothesis: Kaha nako ea Tischendorf , libuka tse eketsehileng tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:51:51,198 Validation result (greedy) at epoch 10, step 83000: bleu: 45.21, loss: 26619.6387, ppl: 2.8444, duration: 35.7320s\n", "2020-02-13 21:52:28,020 Epoch 10 Step: 83100 Batch Loss: 1.049704 Tokens per Sec: 6611, Lr: 0.000300\n", "2020-02-13 21:53:05,126 Epoch 10 Step: 83200 Batch Loss: 1.103581 Tokens per Sec: 6541, Lr: 0.000300\n", "2020-02-13 21:53:41,742 Epoch 10 Step: 83300 Batch Loss: 1.126817 Tokens per Sec: 6613, Lr: 0.000300\n", "2020-02-13 21:54:18,458 Epoch 10 Step: 83400 Batch Loss: 1.188678 Tokens per Sec: 6606, Lr: 0.000300\n", "2020-02-13 21:54:55,475 Epoch 10 Step: 83500 Batch Loss: 1.114533 Tokens per Sec: 6671, Lr: 0.000300\n", "2020-02-13 21:55:32,514 Epoch 10 Step: 83600 Batch Loss: 1.187505 Tokens per Sec: 6578, Lr: 0.000300\n", "2020-02-13 21:56:08,749 Epoch 10 Step: 83700 Batch Loss: 1.087913 Tokens per Sec: 6505, Lr: 0.000300\n", "2020-02-13 21:56:45,437 Epoch 10 Step: 83800 Batch Loss: 1.100294 Tokens per Sec: 6464, Lr: 0.000300\n", "2020-02-13 21:57:22,488 Epoch 10 Step: 83900 Batch Loss: 0.990175 Tokens per Sec: 6731, Lr: 0.000300\n", "2020-02-13 21:57:59,026 Epoch 10 Step: 84000 Batch Loss: 1.372760 Tokens per Sec: 6714, Lr: 0.000300\n", "2020-02-13 21:58:34,790 Hooray! New best validation result [ppl]!\n", "2020-02-13 21:58:34,790 Saving new checkpoint.\n", "2020-02-13 21:58:35,227 Example #0\n", "2020-02-13 21:58:35,227 \tSource: Has the situation improved since then ?\n", "2020-02-13 21:58:35,227 \tReference: Na ho tloha ka nako eo boemo bo ntlafetse ?\n", "2020-02-13 21:58:35,227 \tHypothesis: Na boemo bo ntlafetse ho tloha ka nako eo ?\n", "2020-02-13 21:58:35,227 Example #1\n", "2020-02-13 21:58:35,227 \tSource: Furthermore , as Jehovah reminded Job , animals testify to His strength .\n", "2020-02-13 21:58:35,227 \tReference: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li fana ka bopaki ba hore O matla .\n", "2020-02-13 21:58:35,227 \tHypothesis: Ho feta moo , joalokaha Jehova a ile a hopotsa Jobo , liphoofolo li paka matla a Hae .\n", "2020-02-13 21:58:35,227 Example #2\n", "2020-02-13 21:58:35,228 \tSource: The wise man had one request : The gossiper was told to go and take a feather pillow and cut it open , scattering the feathers to the wind .\n", "2020-02-13 21:58:35,228 \tReference: Monna ea bohlale o ile a kōpa mosebi eo ntho e le ’ ngoe : Hore a tsamaee ’ me a tle le mosamo oa masiba , a o bule ebe o hasa masiba ao moeeng .\n", "2020-02-13 21:58:35,228 \tHypothesis: Monna ea bohlale o ne a e - na le kōpo e le ’ ngoe : Mosebi o ile a bolelloa hore a ee ’ me a nke mosamo o masiba ’ me a o khaola , a hasanya masiba moeeng .\n", "2020-02-13 21:58:35,228 Example #3\n", "2020-02-13 21:58:35,228 \tSource: Since Tischendorf’s time , more manuscripts have come to light .\n", "2020-02-13 21:58:35,228 \tReference: Ho tloha mehleng ea Tischendorf , ho fumanoe libuka tse ngotsoeng ka letsoho tse eketsehileng .\n", "2020-02-13 21:58:35,228 \tHypothesis: Ho tloha mehleng ea Tischendorf , libuka tse ling tse ngotsoeng ka letsoho li ’ nile tsa khanya .\n", "2020-02-13 21:58:35,228 Validation result (greedy) at epoch 10, step 84000: bleu: 44.86, loss: 26323.2656, ppl: 2.8115, duration: 36.2019s\n", "2020-02-13 21:59:12,018 Epoch 10 Step: 84100 Batch Loss: 1.351267 Tokens per Sec: 6648, Lr: 0.000300\n", "2020-02-13 21:59:48,783 Epoch 10 Step: 84200 Batch Loss: 1.141725 Tokens per Sec: 6710, Lr: 0.000300\n", "2020-02-13 22:00:25,222 Epoch 10 Step: 84300 Batch Loss: 1.103337 Tokens per Sec: 6534, Lr: 0.000300\n", "2020-02-13 22:01:02,202 Epoch 10 Step: 84400 Batch Loss: 1.161196 Tokens per Sec: 6683, Lr: 0.000300\n", "2020-02-13 22:01:39,374 Epoch 10 Step: 84500 Batch Loss: 0.983973 Tokens per Sec: 6714, Lr: 0.000300\n", "Traceback (most recent call last):\n", " File \"/usr/lib/python3.6/runpy.py\", line 193, in _run_module_as_main\n", " \"__main__\", mod_spec)\n", " File \"/usr/lib/python3.6/runpy.py\", line 85, in _run_code\n", " exec(code, run_globals)\n", " File \"/content/joeynmt/joeynmt/__main__.py\", line 41, in \n", " main()\n", " File \"/content/joeynmt/joeynmt/__main__.py\", line 29, in main\n", " train(cfg_file=args.config_path)\n", " File \"/content/joeynmt/joeynmt/training.py\", line 600, in train\n", " trainer.train_and_validate(train_data=train_data, valid_data=dev_data)\n", " File \"/content/joeynmt/joeynmt/training.py\", line 300, in train_and_validate\n", " batch_loss = self._train_batch(batch, update=update)\n", " File \"/content/joeynmt/joeynmt/training.py\", line 463, in _train_batch\n", " self.optimizer.step()\n", " File \"/usr/local/lib/python3.6/dist-packages/torch/optim/adam.py\", line 95, in step\n", " exp_avg.mul_(beta1).add_(1 - beta1, grad)\n", "KeyboardInterrupt\n", "^C\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "og_MNJ8a2S6T", "colab_type": "code", "outputId": "af69bc2a-8d52-41be-e0e6-0c1be7ab62e8", "colab": { "base_uri": "https://localhost:8080/", "height": 73 } }, "source": [ "# Copy the created models from the notebook storage to google drive for persistant storage\n", "!mkdir \"$gdrive_path/models/${src}${tgt}_transformer\"\n", "!cp -r joeynmt/models/${src}${tgt}_transformer/* \"$gdrive_path/models/${src}${tgt}_transformer/\"\n", "!cp joeynmt/models/${src}${tgt}_transformer/best.ckpt \"$gdrive_path/models/${src}${tgt}_transformer/\"" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "mkdir: cannot create directory ‘/content/drive/My Drive/masakhane/en-st-baseline/models/enst_transformer’: File exists\n", "cp: cannot stat 'joeynmt/models/enst_transformer/*': No such file or directory\n", "cp: cannot stat 'joeynmt/models/enst_transformer/best.ckpt': No such file or directory\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "B4VAF06-Ga-O", "colab_type": "code", "outputId": "9b20f1d1-a18b-412d-a7eb-df12c9a38c13", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "# Output our validation accuracy\n", "! cat \"$gdrive_path/models/${src}${tgt}_transformer/validations.txt\"" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "Steps: 1000\tLoss: 108547.92969\tPPL: 70.99663\tbleu: 0.96149\tLR: 0.00030000\t*\n", "Steps: 2000\tLoss: 88209.55469\tPPL: 31.94299\tbleu: 1.72850\tLR: 0.00030000\t*\n", "Steps: 3000\tLoss: 76930.92188\tPPL: 20.51273\tbleu: 4.70271\tLR: 0.00030000\t*\n", "Steps: 4000\tLoss: 68434.78125\tPPL: 14.69351\tbleu: 9.21711\tLR: 0.00030000\t*\n", "Steps: 5000\tLoss: 61544.32812\tPPL: 11.21016\tbleu: 13.85765\tLR: 0.00030000\t*\n", "Steps: 6000\tLoss: 56446.87500\tPPL: 9.17650\tbleu: 17.98712\tLR: 0.00030000\t*\n", "Steps: 7000\tLoss: 52346.01562\tPPL: 7.81157\tbleu: 21.53790\tLR: 0.00030000\t*\n", "Steps: 8000\tLoss: 49132.64062\tPPL: 6.88551\tbleu: 25.71689\tLR: 0.00030000\t*\n", "Steps: 9000\tLoss: 46748.56250\tPPL: 6.27013\tbleu: 27.71073\tLR: 0.00030000\t*\n", "Steps: 10000\tLoss: 45489.00391\tPPL: 5.96754\tbleu: 29.18093\tLR: 0.00030000\t*\n", "Steps: 11000\tLoss: 42909.72656\tPPL: 5.39271\tbleu: 30.87249\tLR: 0.00030000\t*\n", "Steps: 12000\tLoss: 41262.10156\tPPL: 5.05484\tbleu: 32.44968\tLR: 0.00030000\t*\n", "Steps: 13000\tLoss: 40603.58984\tPPL: 4.92580\tbleu: 33.14003\tLR: 0.00030000\t*\n", "Steps: 14000\tLoss: 39439.69141\tPPL: 4.70573\tbleu: 34.34833\tLR: 0.00030000\t*\n", "Steps: 15000\tLoss: 38308.28906\tPPL: 4.50123\tbleu: 35.67213\tLR: 0.00030000\t*\n", "Steps: 16000\tLoss: 37757.96875\tPPL: 4.40500\tbleu: 35.98613\tLR: 0.00030000\t*\n", "Steps: 17000\tLoss: 36910.75000\tPPL: 4.26086\tbleu: 37.04409\tLR: 0.00030000\t*\n", "Steps: 18000\tLoss: 36366.78516\tPPL: 4.17080\tbleu: 37.13830\tLR: 0.00030000\t*\n", "Steps: 19000\tLoss: 35914.87500\tPPL: 4.09744\tbleu: 37.40406\tLR: 0.00030000\t*\n", "Steps: 20000\tLoss: 35548.44922\tPPL: 4.03890\tbleu: 38.02700\tLR: 0.00030000\t*\n", "Steps: 21000\tLoss: 34824.57031\tPPL: 3.92571\tbleu: 38.44264\tLR: 0.00030000\t*\n", "Steps: 22000\tLoss: 34300.15234\tPPL: 3.84569\tbleu: 38.65244\tLR: 0.00030000\t*\n", "Steps: 23000\tLoss: 33841.75391\tPPL: 3.77708\tbleu: 39.26034\tLR: 0.00030000\t*\n", "Steps: 24000\tLoss: 33616.39062\tPPL: 3.74380\tbleu: 39.15721\tLR: 0.00030000\t*\n", "Steps: 25000\tLoss: 33239.61328\tPPL: 3.68882\tbleu: 39.16063\tLR: 0.00030000\t*\n", "Steps: 26000\tLoss: 32730.25195\tPPL: 3.61576\tbleu: 39.96929\tLR: 0.00030000\t*\n", "Steps: 27000\tLoss: 32588.46094\tPPL: 3.59569\tbleu: 39.61640\tLR: 0.00030000\t*\n", "Steps: 28000\tLoss: 32438.31641\tPPL: 3.57455\tbleu: 39.98859\tLR: 0.00030000\t*\n", "Steps: 29000\tLoss: 32244.14453\tPPL: 3.54740\tbleu: 40.06382\tLR: 0.00030000\t*\n", "Steps: 30000\tLoss: 31913.40625\tPPL: 3.50162\tbleu: 40.68800\tLR: 0.00030000\t*\n", "Steps: 31000\tLoss: 31599.14648\tPPL: 3.45867\tbleu: 41.35574\tLR: 0.00030000\t*\n", "Steps: 32000\tLoss: 31655.83984\tPPL: 3.46638\tbleu: 40.88501\tLR: 0.00030000\t\n", "Steps: 33000\tLoss: 31182.29492\tPPL: 3.40252\tbleu: 41.50268\tLR: 0.00030000\t*\n", "Steps: 34000\tLoss: 31066.63867\tPPL: 3.38710\tbleu: 41.51353\tLR: 0.00030000\t*\n", "Steps: 35000\tLoss: 30806.52539\tPPL: 3.35268\tbleu: 41.58490\tLR: 0.00030000\t*\n", "Steps: 36000\tLoss: 30705.52930\tPPL: 3.33941\tbleu: 41.33901\tLR: 0.00030000\t*\n", "Steps: 37000\tLoss: 30573.38086\tPPL: 3.32212\tbleu: 41.69055\tLR: 0.00030000\t*\n", "Steps: 38000\tLoss: 30325.64648\tPPL: 3.28996\tbleu: 41.90099\tLR: 0.00030000\t*\n", "Steps: 39000\tLoss: 30216.73242\tPPL: 3.27592\tbleu: 41.69334\tLR: 0.00030000\t*\n", "Steps: 40000\tLoss: 29857.54688\tPPL: 3.23004\tbleu: 42.07589\tLR: 0.00030000\t*\n", "Steps: 41000\tLoss: 29925.39258\tPPL: 3.23865\tbleu: 41.75120\tLR: 0.00030000\t\n", "Steps: 42000\tLoss: 29745.07031\tPPL: 3.21580\tbleu: 42.11170\tLR: 0.00030000\t*\n", "Steps: 43000\tLoss: 29463.84570\tPPL: 3.18048\tbleu: 42.48200\tLR: 0.00030000\t*\n", "Steps: 44000\tLoss: 29451.08203\tPPL: 3.17889\tbleu: 42.75804\tLR: 0.00030000\t*\n", "Steps: 45000\tLoss: 29204.74023\tPPL: 3.14829\tbleu: 42.69795\tLR: 0.00030000\t*\n", "Steps: 46000\tLoss: 29253.82227\tPPL: 3.15436\tbleu: 42.50608\tLR: 0.00030000\t\n", "Steps: 47000\tLoss: 29091.80078\tPPL: 3.13435\tbleu: 43.27207\tLR: 0.00030000\t*\n", "Steps: 48000\tLoss: 28861.96094\tPPL: 3.10619\tbleu: 43.49844\tLR: 0.00030000\t*\n", "Steps: 49000\tLoss: 28771.90234\tPPL: 3.09523\tbleu: 42.83001\tLR: 0.00030000\t*\n", "Steps: 50000\tLoss: 28744.87500\tPPL: 3.09194\tbleu: 42.97435\tLR: 0.00030000\t*\n", "Steps: 51000\tLoss: 28496.42578\tPPL: 3.06192\tbleu: 43.09106\tLR: 0.00030000\t*\n", "Steps: 52000\tLoss: 28301.80859\tPPL: 3.03861\tbleu: 43.44813\tLR: 0.00030000\t*\n", "Steps: 53000\tLoss: 28258.18750\tPPL: 3.03341\tbleu: 43.66491\tLR: 0.00030000\t*\n", "Steps: 54000\tLoss: 28327.17773\tPPL: 3.04164\tbleu: 43.33189\tLR: 0.00030000\t\n", "Steps: 55000\tLoss: 28253.72852\tPPL: 3.03288\tbleu: 43.16155\tLR: 0.00030000\t*\n", "Steps: 56000\tLoss: 28331.01562\tPPL: 3.04210\tbleu: 43.09409\tLR: 0.00030000\t\n", "Steps: 57000\tLoss: 27946.97070\tPPL: 2.99656\tbleu: 43.34545\tLR: 0.00030000\t*\n", "Steps: 58000\tLoss: 27931.42188\tPPL: 2.99473\tbleu: 43.84953\tLR: 0.00030000\t*\n", "Steps: 59000\tLoss: 27766.70508\tPPL: 2.97543\tbleu: 43.65511\tLR: 0.00030000\t*\n", "Steps: 60000\tLoss: 27651.60938\tPPL: 2.96201\tbleu: 43.68342\tLR: 0.00030000\t*\n", "Steps: 61000\tLoss: 27738.05664\tPPL: 2.97208\tbleu: 43.69468\tLR: 0.00030000\t\n", "Steps: 62000\tLoss: 27610.03516\tPPL: 2.95718\tbleu: 44.72461\tLR: 0.00030000\t*\n", "Steps: 63000\tLoss: 27856.81445\tPPL: 2.98597\tbleu: 43.84341\tLR: 0.00030000\t\n", "Steps: 64000\tLoss: 27382.61719\tPPL: 2.93088\tbleu: 44.35961\tLR: 0.00030000\t*\n", "Steps: 65000\tLoss: 27388.63672\tPPL: 2.93158\tbleu: 44.25289\tLR: 0.00030000\t\n", "Steps: 66000\tLoss: 27272.58984\tPPL: 2.91825\tbleu: 44.52005\tLR: 0.00030000\t*\n", "Steps: 67000\tLoss: 27316.35742\tPPL: 2.92327\tbleu: 44.43330\tLR: 0.00030000\t\n", "Steps: 68000\tLoss: 27134.11914\tPPL: 2.90242\tbleu: 44.58064\tLR: 0.00030000\t*\n", "Steps: 69000\tLoss: 27141.83984\tPPL: 2.90330\tbleu: 43.97417\tLR: 0.00030000\t\n", "Steps: 70000\tLoss: 27172.28125\tPPL: 2.90677\tbleu: 44.21714\tLR: 0.00030000\t\n", "Steps: 71000\tLoss: 27257.70117\tPPL: 2.91654\tbleu: 44.12328\tLR: 0.00030000\t\n", "Steps: 72000\tLoss: 27107.95703\tPPL: 2.89944\tbleu: 44.35836\tLR: 0.00030000\t*\n", "Steps: 73000\tLoss: 26895.14062\tPPL: 2.87531\tbleu: 44.53364\tLR: 0.00030000\t*\n", "Steps: 74000\tLoss: 26977.70508\tPPL: 2.88465\tbleu: 44.61194\tLR: 0.00030000\t\n", "Steps: 75000\tLoss: 26818.14648\tPPL: 2.86663\tbleu: 44.61026\tLR: 0.00030000\t*\n", "Steps: 76000\tLoss: 26784.37109\tPPL: 2.86283\tbleu: 44.65337\tLR: 0.00030000\t*\n", "Steps: 77000\tLoss: 26617.19922\tPPL: 2.84410\tbleu: 44.97137\tLR: 0.00030000\t*\n", "Steps: 78000\tLoss: 26725.07422\tPPL: 2.85617\tbleu: 44.43874\tLR: 0.00030000\t\n", "Steps: 79000\tLoss: 26483.38477\tPPL: 2.82919\tbleu: 44.99577\tLR: 0.00030000\t*\n", "Steps: 80000\tLoss: 26543.19141\tPPL: 2.83585\tbleu: 44.92037\tLR: 0.00030000\t\n", "Steps: 81000\tLoss: 26605.94336\tPPL: 2.84284\tbleu: 44.88913\tLR: 0.00030000\t\n", "Steps: 82000\tLoss: 26773.83203\tPPL: 2.86165\tbleu: 44.84268\tLR: 0.00030000\t\n", "Steps: 83000\tLoss: 26619.63867\tPPL: 2.84437\tbleu: 45.20957\tLR: 0.00030000\t\n", "Steps: 84000\tLoss: 26323.26562\tPPL: 2.81146\tbleu: 44.85574\tLR: 0.00030000\t*\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "uk3m-OGJGkNk", "colab_type": "text" }, "source": [ "## Testing from gdrive" ] }, { "cell_type": "code", "metadata": { "id": "QawYr0cXxgaT", "colab_type": "code", "colab": {} }, "source": [ "! mkdir -p joeynmt/models/${src}${tgt}_transformer/\n", "! cp \"$gdrive_path/models/${src}${tgt}_transformer/best.ckpt\" \"joeynmt/models/${src}${tgt}_transformer/best.ckpt\"" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "XbwDg5_OzY_K", "colab_type": "code", "colab": {} }, "source": [ "# copy test files\n", "! cp \"$gdrive_path/test.$src\" ./\n", "! cp \"$gdrive_path/test.$tgt\" ./" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "y4OsAmelzbOR", "colab_type": "code", "colab": {} }, "source": [ "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$src < test.$src > test.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < test.$tgt > test.bpe.$tgt\n", "! cp test.* $data_path" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "NYhwX-SvGdbh", "colab_type": "code", "outputId": "4b0025f1-ae32-45d2-cfd5-d159e35dc28a", "colab": { "base_uri": "https://localhost:8080/", "height": 73 } }, "source": [ "# Test our model\n", "! cd joeynmt; python3 -m joeynmt test \"$gdrive_path/models/${src}${tgt}_transformer/config.yaml\"" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "2020-02-16 15:42:59,652 Hello! This is Joey-NMT.\n", "2020-02-16 15:44:07,041 dev bleu: 46.15 [Beam search decoding with beam size = 5 and alpha = 1.0]\n", "2020-02-16 15:44:51,075 test bleu: 41.23 [Beam search decoding with beam size = 5 and alpha = 1.0]\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "_Iymv7_jpXlK", "colab_type": "text" }, "source": [ "## Testing from Autshumato" ] }, { "cell_type": "code", "metadata": { "id": "rJz-TCs9parr", "colab_type": "code", "outputId": "1a30ad15-3bcb-460c-8941-52c3f03ff5eb", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "! wget https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator1.$src.txt\n", "! wget https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator1.$tgt.txt\n", "! wget https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator2.$tgt.txt\n", "! wget https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator3.$tgt.txt\n", "! wget https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator4.$tgt.txt" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "--2020-02-16 17:44:43-- https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator1.en.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 58665 (57K) [text/plain]\n", "Saving to: ‘translator1.en.txt’\n", "\n", "\rtranslator1.en.txt 0%[ ] 0 --.-KB/s \rtranslator1.en.txt 100%[===================>] 57.29K --.-KB/s in 0.008s \n", "\n", "2020-02-16 17:44:43 (7.17 MB/s) - ‘translator1.en.txt’ saved [58665/58665]\n", "\n", "--2020-02-16 17:44:48-- https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator1.st.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 65568 (64K) [text/plain]\n", "Saving to: ‘translator1.st.txt’\n", "\n", "translator1.st.txt 100%[===================>] 64.03K --.-KB/s in 0.008s \n", "\n", "2020-02-16 17:44:48 (7.57 MB/s) - ‘translator1.st.txt’ saved [65568/65568]\n", "\n", "--2020-02-16 17:44:53-- https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator2.st.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 64983 (63K) [text/plain]\n", "Saving to: ‘translator2.st.txt’\n", "\n", "translator2.st.txt 100%[===================>] 63.46K --.-KB/s in 0.008s \n", "\n", "2020-02-16 17:44:53 (8.25 MB/s) - ‘translator2.st.txt’ saved [64983/64983]\n", "\n", "--2020-02-16 17:44:56-- https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator3.st.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 64515 (63K) [text/plain]\n", "Saving to: ‘translator3.st.txt’\n", "\n", "translator3.st.txt 100%[===================>] 63.00K --.-KB/s in 0.008s \n", "\n", "2020-02-16 17:44:56 (7.94 MB/s) - ‘translator3.st.txt’ saved [64515/64515]\n", "\n", "--2020-02-16 17:45:01-- https://raw.githubusercontent.com/jasonrobwebster/autshumato-eval-bleu/master/data/processed/translator4.st.txt\n", "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n", "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 59463 (58K) [text/plain]\n", "Saving to: ‘translator4.st.txt’\n", "\n", "translator4.st.txt 100%[===================>] 58.07K --.-KB/s in 0.008s \n", "\n", "2020-02-16 17:45:01 (7.00 MB/s) - ‘translator4.st.txt’ saved [59463/59463]\n", "\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "colab_type": "code", "outputId": "8a7052d7-96c0-4bac-e4be-66808fb7d121", "id": "L-6PlW4s_hyI", "colab": { "base_uri": "https://localhost:8080/", "height": 221 } }, "source": [ "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$src < ./translator1.$src.txt > test.bpe.$src\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < ./translator1.$tgt.txt > translator1.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < ./translator2.$tgt.txt > translator2.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < ./translator3.$tgt.txt > translator3.bpe.$tgt\n", "! subword-nmt apply-bpe -c bpe.codes.$codes --vocabulary vocab.$tgt < ./translator4.$tgt.txt > translator4.bpe.$tgt\n", "\n", "! head -n 5 test.bpe.$src\n", "! echo \"\"\n", "! head -n 5 translator1.bpe.$tgt" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "South African Social Security Agency\n", "Custo@@ mer Care Char@@ ter\n", "Welcome to our Cli@@ ent Char@@ ter\n", "We want you , our cli@@ ent , to judge us according to the standard of service we set out in this char@@ ter .\n", "We will treat you with respect , and give you good service in accordance with the BA@@ THO PE@@ LE principles .\n", "\n", "Bo@@ e@@ me@@ di ba Afrika Bor@@ wa ya T@@ shi@@ rel@@ etso ba Set@@ j@@ ha@@ ba\n", "Tumellano ya Tlhokomelo ya Ba@@ reki\n", "Re a o amohela ho Tumellano ya rona ya Ba@@ reki\n", "Re batla hore wena , moreki wa rona , o re hlahlobe ho ya ka maemo a ts@@ he@@ bel@@ etso ao re a hlahisitseng ka hara tumellano ena .\n", "Re tla o sebeletsa ka tlhompho le ho o fa ts@@ he@@ bel@@ etso e lokileng ho k@@ ge@@ ma le maano a BATHO PELE .\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "InEcivYIpqyc", "colab_type": "code", "outputId": "8ea9f4e0-5f3a-498c-f2b6-183e1f189a1d", "colab": { "base_uri": "https://localhost:8080/", "height": 36 } }, "source": [ "! cd joeynmt; python3 -m joeynmt translate \"$gdrive_path/models/${src}${tgt}_transformer/config.yaml\" < ../test.bpe.$src > ../model.$tgt" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "2020-02-16 17:46:37,878 Hello! This is Joey-NMT.\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "xluBgF3DrViS", "colab_type": "code", "outputId": "01f313f4-5c19-4180-8b87-c37b2cc973f1", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "! echo \"==> Source <==\"\n", "! head -n 10 translator1.$src.txt\n", "! echo \"\"\n", "! head -n 10 *.$tgt.txt\n", "! echo \"\"\n", "! echo \"==> Model <==\"\n", "! head -n 10 model.$tgt" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "==> Source <==\n", "South African Social Security Agency\n", "Customer Care Charter\n", "Welcome to our Client Charter\n", "We want you , our client , to judge us according to the standard of service we set out in this charter .\n", "We will treat you with respect , and give you good service in accordance with the BATHO PELE principles .\n", "As part of our responsibility , we promise to deliver a world class service , and to give you accurate information , advice and assistance for all our services .\n", "Aim of the Charter\n", "This charter tells you what standard of service you can expect from the South African Social Security Agency ( SASSA ) .\n", "SASSA is an extension of a government delivery branch that administers the delivery of social grants to the citizens of South Africa .\n", "We promise\n", "\n", "==> translator1.st.txt <==\n", "Boemedi ba Afrika Borwa ya Tshireletso ba Setjhaba\n", "Tumellano ya Tlhokomelo ya Bareki\n", "Re a o amohela ho Tumellano ya rona ya Bareki\n", "Re batla hore wena , moreki wa rona , o re hlahlobe ho ya ka maemo a tshebeletso ao re a hlahisitseng ka hara tumellano ena .\n", "Re tla o sebeletsa ka tlhompho le ho o fa tshebeletso e lokileng ho kgema le maano a BATHO PELE .\n", "Re a itlama jwalokaha e le karolo ya boikarabelo ba rona ho fana ka tshebeletso ya maemo a phahameng lefatsheng , ho o fa dintlha tse nepahetseng , keletso le thuso ditshebeletsong tsa rona tsohle .\n", "Sepheo sa Tumellano\n", "Tumellano ena e o tsebisa ka maemo a tshebeletso eo o ka e lebellang ho Boemedi ba Afrika Borwa ba Tshireletso ya Setjhaba ( SASSA ) .\n", "SASSA ke katoloso ya karolo ya tshebetso ya mmuso e laolang phano ya ditjhelete tsa thuso ya setjhaba ho baahi ba Afrika Borwa .\n", "Re itlama\n", "\n", "==> translator2.st.txt <==\n", "Boeletsi ba Tshireletso ya Phedisano ya Afrika Borwa\n", "Tokomane ya Tlhokomelo ya Moreki\n", "O a amohelwa Tokomaneng ya rona ya Moreki\n", "Re batla hore wena moreki wa rona o re ahlole ho ya ka tshebeletso eo re e totobatsang tokomaneng ena .\n", "Re tla o hlompha mme re o fe tshebeletso e tswileng ka mahetla ho latela ditheo tsa BATHO PELE .\n", "Re itlama e le karolo ya boikarabelo ba rona ho fana ka tshebeletso e tswileng ka mahetla , re o fe lesedi le tsepameng , keletso mabapi le ditshebeletso tsohle tsa rona .\n", "Sepheo sa Tokomane\n", "Tokomane ena e o bolella ka maemo a tshebeletso eo o ka e lebellang ho tswa ho Boeletsi ba Tshireletso ya Phedisano ya Afrika Borwa ( SASSA ) .\n", "SASSA ke katoloso ya lekala la phano ya ditshebeletso le sebetsanang le phano ya dithuso ho baahi ba Afrika Borwa .\n", "Re a itlama\n", "\n", "==> translator3.st.txt <==\n", "Boemedi bo fanang ka Tshireletso ya Thekolohelo wa Afrika Borwa\n", "Tumellano ya tlhokomelo ya moreki\n", "O a amohelwa ho Tumellano ya rona ya Moreki\n", "Re a o hloka jwalo ka moreki wa rona hore o re ahlole ho latela boemo ba ditshebeletso jwalo ka ha di hlaha ka hare ho tumellano ena .\n", "Re tla sebetsa le wena ka hlompho mme re o fe tshebeletso e ntle e ikamahanyang le maano a BATHO PELE .\n", "Re a o tshepisa jwalo ka ha e le boikarabelo ba rona ho fana ka ditshebeletso tse tswileng matsoho , mme re ofe lesedi le nepahetseng , keletso mmoho le thuso ditshebeletsong tsohle tsa rona .\n", "Maikemisetso a Tumellano\n", "Tumellano ena e o bolella ka boemo ba ditshebeletso tseo o ka di lebellang ho tswa ho Boemedi ba Tshireletso ya Thekolohelo ya Afrika Borwa ( SASSA ) .\n", "SASSA ke katoloso ya letsoho la mmuso la ditshebeletso tse laolang phano ya dithuso tsa thekolohelo baahing ba Afrika Borwa .\n", "Re itlama\n", "\n", "==> translator4.st.txt <==\n", "Setheo sa Tshireletso ya Setjhaba sa Afrika Borwa\n", "Tjhata ya Tlhokomelo ya Bareki\n", "O amohetswe ho Tjhata ya rona ya Tlhokomelo\n", "Re batla wena , moreki wa rona , o re ahlole ho ya ka maemo a tshebeletso ao re a qollang tjhateng ena .\n", "Re tla o tshwara ka tlhompho mme re o fe tshebeletso tse nepahetseng ho latela metheo ya BATHO PELE .\n", "Re itlama jwalo ka boikarabello ba rona ho fana ka tshebeletso ya maemo a phahameng lefatshe ka bophara , re o fe lesedi le nepahetseng , keletso le thuso ditshebeletsong tsa rona kaofela .\n", "Sepheo sa Tjhata\n", "Tjhata e o bolella ka maemo a tshebeletso eo o ka e lebellang ho tswa ho Setheo sa Tshireletso ya Setjhaba sa Afrika Borwa ( SASSA ) .\n", "SASSA ke karolo ya letsoho le abang la mmuso le tsamaisang ho ajwa ha dithuso tsa setjhaba ho baahi ba Afrika Borwa .\n", "Re itlama\n", "\n", "==> Model <==\n", "Mokhatlo oa Tšireletso ea Sechaba oa Afrika Boroa\n", "Moreki oa Tlhokomelo\n", "Amohela Khakanyo ea Rōna e Tiileng\n", "Re batla hore uena , motho eo re mo batlang , u re ahlole ho ea ka tekanyetso ea tšebeletso eo re e behileng tlhokomelong ena ea molao .\n", "Re tla u tšoara ka tlhompho , ’ me re u fe tšebeletso e molemo tumellanong le melao - motheo ea BATHO .\n", "Re le karolo ea boikarabelo ba rōna , re tšepisa ho nehelana ka tšebeletso ea sehlopha sa lefatše , le ho u fa boitsebiso bo nepahetseng , keletso le thuso bakeng sa litšebeletso tsohle tsa rōna .\n", "Aim of the Charter\n", "Molao ona oa motheo o u bolella hore na ke tekanyetso efe ea tšebeletso eo u ka e lebellang ho Mokhatlo oa Tšireletso oa Afrika Boroa ( SASSA ) .\n", "SASSA ke katoloso ea lekala la ’ muso le tsamaisang batho ba fanang ka liphallelo ho baahi ba Afrika Boroa .\n", "Re tšepisa\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "HMG7Bc1UrGH9", "colab_type": "code", "colab": {} }, "source": [ "import re\n", "import codecs\n", "\n", "def load_all_translations(lang, translators=['translator1', 'translator2', 'translator3', 'translator4'], proc_dir='.'):\n", " \"\"\"Load all autshumato evaluation translations into a dictionary.\n", " \n", " Params\n", " ------\n", " \n", " lang (str):\n", " The ISO code language to load.\n", " \n", " Returns\n", " -------\n", " \n", " out (dict):\n", " A dictionary containing all translated lines from the Autshumato evaluation set\n", " for the given language. The key corresponds to a translator, and the value is a list\n", " containing the translation.\n", " \"\"\"\n", " out = {}\n", " for translator in translators:\n", " fp = f\"{translator}.bpe.{lang}\"\n", " fp = os.path.join(proc_dir, fp)\n", " with codecs.open(fp, 'r', encoding='utf-8') as f:\n", " lines = f.readlines()\n", " # strip the translation of any escape chars or whitespace\n", " out[translator] = list(map(lambda x: x.strip(), lines))\n", " out[translator] = [string.replace(\"@@ \", \"\") for string in out[translator]]\n", " return out" ], "execution_count": 0, "outputs": [] }, { "cell_type": "code", "metadata": { "id": "mOX1IwmmsZ3W", "colab_type": "code", "outputId": "4e13bb0d-9165-4678-82b3-f14e4c15a29a", "colab": { "base_uri": "https://localhost:8080/", "height": 110 } }, "source": [ "all_translations = load_all_translations(target_language)\n", "all_translations['translator1'][0:5]" ], "execution_count": 0, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['Boemedi ba Afrika Borwa ya Tshireletso ba Setjhaba',\n", " 'Tumellano ya Tlhokomelo ya Bareki',\n", " 'Re a o amohela ho Tumellano ya rona ya Bareki',\n", " 'Re batla hore wena , moreki wa rona , o re hlahlobe ho ya ka maemo a tshebeletso ao re a hlahisitseng ka hara tumellano ena .',\n", " 'Re tla o sebeletsa ka tlhompho le ho o fa tshebeletso e lokileng ho kgema le maano a BATHO PELE .']" ] }, "metadata": { "tags": [] }, "execution_count": 149 } ] }, { "cell_type": "code", "metadata": { "id": "GPTFe2g7stuI", "colab_type": "code", "outputId": "c165560d-7a17-4e9a-c5cd-d7b0300ce081", "colab": { "base_uri": "https://localhost:8080/", "height": 91 } }, "source": [ "print(len(all_translations['translator1']) == 500)\n", "print(len(all_translations['translator2']) == 500)\n", "print(len(all_translations['translator3']) == 500)\n", "print(len(all_translations['translator4']) == 500)" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "True\n", "True\n", "True\n", "True\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "e8WcC73wtMua", "colab_type": "code", "outputId": "26d16a84-041f-4226-d409-ffcdce365720", "colab": { "base_uri": "https://localhost:8080/", "height": 74 } }, "source": [ "refs = list(all_translations.values())\n", "with codecs.open(f'model.{target_language}', 'r', 'utf-8') as f:\n", " sys = f.readlines()\n", " sys = list(map(lambda x: x.strip(), sys))\n", "\n", "print(sys[0:5])\n", "print(refs[0][0:5])" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "['Mokhatlo oa Tšireletso ea Sechaba oa Afrika Boroa', 'Moreki oa Tlhokomelo', 'Amohela Khakanyo ea Rōna e Tiileng', 'Re batla hore uena , motho eo re mo batlang , u re ahlole ho ea ka tekanyetso ea tšebeletso eo re e behileng tlhokomelong ena ea molao .', 'Re tla u tšoara ka tlhompho , ’ me re u fe tšebeletso e molemo tumellanong le melao - motheo ea BATHO .']\n", "['Boemedi ba Afrika Borwa ya Tshireletso ba Setjhaba', 'Tumellano ya Tlhokomelo ya Bareki', 'Re a o amohela ho Tumellano ya rona ya Bareki', 'Re batla hore wena , moreki wa rona , o re hlahlobe ho ya ka maemo a tshebeletso ao re a hlahisitseng ka hara tumellano ena .', 'Re tla o sebeletsa ka tlhompho le ho o fa tshebeletso e lokileng ho kgema le maano a BATHO PELE .']\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "x5uk1oNBucp1", "colab_type": "code", "outputId": "c173769d-f1aa-46a5-9115-473aecd02df3", "colab": { "base_uri": "https://localhost:8080/", "height": 91 } }, "source": [ "import sacrebleu\n", "\n", "score = sacrebleu.corpus_bleu(sys, refs).score\n", "print(f\"Autshumato Test BLEU: {score}\")" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "WARNING:root:That's 100 lines that end in a tokenized period ('.')\n", "WARNING:root:It looks like you forgot to detokenize your test data, which may hurt your score.\n", "WARNING:root:If you insist your data is detokenized, or don't care, you can suppress this message with '--force'.\n" ], "name": "stderr" }, { "output_type": "stream", "text": [ "Autshumato Test BLEU: 12.182730696079144\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "x7WRLrR356in", "colab_type": "code", "colab": {} }, "source": [ "" ], "execution_count": 0, "outputs": [] } ] }