Repository: piskvorky/gensim

Issue:
Dockerfile doesn't build
#### Description
dockerfile doesn't build

#### Steps/Code/Corpus to Reproduce
cd docker
docker build .

#### Expected Results
build comples succesfully

#### Actual Results
```
Get:18 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8532 B]
Fetched 15.5 MB in 1s (10.1 MB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
E: Version '1.17.1-1ubuntu1.3' for 'wget' was not found
E: Version '2.23-0ubuntu9' for 'locales' was not found
The command '/bin/sh -c apt-get update     && apt-get install -y     ant=1.9.6-1ubuntu1     cmake=3.5.1-1ubuntu3     default-jdk=2:1.8-56ubuntu2     g++=4:5.3.1-1ubuntu1     git=1:2.7.4-0ubuntu1     libboost-all-dev=1.58.0.1ubuntu1     libgsl-dev=2.1+dfsg-2     mercurial=3.7.3-1ubuntu1     python3=3.5.1-3     python3-pip=8.1.1-2ubuntu0.4     python3-setuptools=20.7.0-1     python=2.7.11-1     python-pip=8.1.1-2ubuntu0.4     python-setuptools=20.7.0-1     unzip=6.0-20ubuntu1     wget=1.17.1-1ubuntu1.3     subversion=1.9.3-2ubuntu1.1     locales=2.23-0ubuntu9     libopenblas-dev=0.2.18-1ubuntu1     libboost-program-options-dev=1.58.0.1ubuntu1     zlib1g-dev=1:1.2.8.dfsg-2ubuntu4.1' returned a non-zero code: 100
```


#### Versions
develop branch @ 20181101_2219


Discussion:
Hy, 
did you confirm that the mentioned apps are installed on your machine?
What packages should be installed on my machine?
Do you mean on the host machine that's trying to build the docker Container? Why would that matter? 
Packages wget and locales have updated recently to versions wget 1.17.1-1ubuntu1.4 (https://packages.ubuntu.com/xenial/wget) and locales 2.23-0ubuntu10 (https://packages.ubuntu.com/xenial/locales). So of course everything is broken.

@adrianog you can get around by specifying current version of wget and locales.
[asking with a certain degree of ignorance]
Are we not able to pin to specific versions _that are also located at download urls that don't change_? So that the build doesn't break over time (it might become out of date, but it's always possible to rebuild the container).


So, I think this has the sense to unpin apt packages in `Dockerfile` to avoid this issue in the future (I already saw almost same issue some time ago).
I don't like unpinned packages as it makes the build less deterministic. I want the same output irrespective of when I run it, and if the location of the pinned packages has changed, so let it be (and fail) - update the dockerfile to the new location.

And of course when you have new / updated dependencies because Gensim requires a new version, update the dockerfile. But if I checkout the repo as of a certain date, it should still be self-consistent and build ok.

Why would the old package disappear? Can we not pin "to specific versions *that are also located at download **urls that don't change***"... That "don't change" might be impossible, I don't enough about apt repo lifecycles...


@adrianog 
> Why would the old package disappear?

I don't know too, not sure about apt repo :( maybe @horpto can give us an advice?

> I don't like unpinned packages as it makes the build less deterministic

For this one, we can unpin all stuff, build an image and push it to docker-registry: if you pull image instead of building your own - this will be deterministic.



> > I don't like unpinned packages as it makes the build less deterministic
> 
> For this one, we can unpin all stuff, build an image and push it to docker-registry: if you pull image instead of building your own - this will be deterministic.

It won't be possible to know what versions were used in building the container image just by looking at the dockerfile if you unpin. So I am definitively in favour of keeping everything pinned; at a conceptual level.

On a practical level, your suggestion makes perfect sense to move on from this.
Irrespective of the pinned vs non-pinned issue, it is a good idea in general for something (excellent) like Gensim to have an "official" image from RaRe-Technologies on the docker hub (so that people can just "docker run" easily).

When pushing, I suggest having two labels for each image pushed: one with the git commit sha, the other one with the numbered version. Avoid "latest".




@adrianog original intent of `Dockerfile` was related to "wrappers" (like mallet, wovpal, etc) & manually testing, it's a bit tricky to install all "deps" to run all tests.
Gensim itself can be easily installed from pip on linux/osx/windows, so, no reasons to worry about it too much.
This `Dockerfile` more for contributors (not for users). If you need "determinism" as user - you can easily create your own image-with-gensim.


> Gensim itself can be easily installed from pip on linux/osx/windows, so, no reasons to worry about it too much.

I would politely disagree. You could say this of a lot of software: does it mean there is no need to provide a docker image in those cases?

I think there is still value in producing an official image for users that "just works". The amount of dependencies (wrappers) available / officially tested in the container can vary, according to level of trickiness.
But note that, paradoxically, if it is especially tricky, it is especially important to have a container image that "just works" - so that users can run that rather than going through the tricky voyage themselves.
> does it mean there is no need to provide a docker image in those cases?

up to us, I think

> But note that, paradoxically, if it is especially tricky, it is especially important to have a container image that "just works" - so that users can run that rather than going through the tricky voyage themselves.

Yes, I definitely agree with this point
@mpenkov probably axe the whole Docker functionality – there's no one to maintain it.
Hi, I had run into the same issue with docker build earlier. I can, depending on your preferences:

A. Try to figure out what it would take to fix it. 
B. Straight up remove the docker folder. 

Also, I noticed that the pip install in the Dockerfile is independent of the `requirements_docs.txt` file. Should the Dockerfile not be using the same requirements file as the standard virtual environment? 
@FyzHsn yes, if you could remove Docker that'd be great, thanks.
@piskvorky Ok, I'll remove it.

Resolving patch:
diff --git a/docker/check_fast_version.py b/docker/check_fast_version.py
deleted file mode 100644
--- a/docker/check_fast_version.py
+++ /dev/null
@@ -1,10 +0,0 @@
-import sys
-
-try:
-    from gensim.models.word2vec_inner import FAST_VERSION
-
-    print('FAST_VERSION ok ! Retrieved with value ', FAST_VERSION)
-    sys.exit()
-except ImportError:
-    print('Failed... fall back to plain numpy (20-80x slower training than the above)')
-    sys.exit(-1)
diff --git a/gensim/models/wrappers/wordrank.py b/gensim/models/wrappers/wordrank.py
--- a/gensim/models/wrappers/wordrank.py
+++ b/gensim/models/wrappers/wordrank.py
@@ -41,11 +41,6 @@
     >>> model = Wordrank.train(path_to_wordrank_binary, corpus_file='text8', out_name='wr_model')
     >>>
     >>> print(model["hello"])  # prints vector for given words
-
-Warnings
---------
-Note that the wrapper might not work in a docker container for large datasets due to memory limits (caused by MPI).
-
 """
 
 from __future__ import division


Test patch: