Repository: docker/docker-py

Issue:
Support for .dockerignore
According to https://docs.docker.com/reference/builder/#dockerignore you can have a `.dockerignore` file that prevents items from being ADDed. Mine looks like this:

```
.git
```

And then in my Dockerfile I have 

```
ADD . /root/mycodes/
```

Unfortunately I still see `/root/mycodes/.git` in my container, which is sad because it's almost 300M. Is there any way to support this? 



Discussion:
@hamiltont Are you using `1.1.0` ?

Yes, and just running a `sudo docker build .` works as expected. Running the same build command from docker-py does not 

For reference: 

```
hamiltont@3:/localhdd$ sudo docker info
Containers: 6
Images: 37
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 49
Execution Driver: native-0.2
Kernel Version: 3.11.0-24-generic
WARNING: No swap limit support
hamiltont@3:/localhdd$ sudo docker version
Client version: 1.1.0
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): 79812e3
Server version: 1.1.0
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): 79812000
```

@hamiltont what are the permission of your `.dockerignore` file ? can you paste us a `ls -la` please ?

@hamiltont also, please paste us the output of `sudo docker -D build .`

```
hamiltont@3:/localhdd$ ls -la | grep dock
-rw-rw-r--   1 hamiltont hamiltont       5 Jul  7 23:26 .dockerignore
```

Here you go. It should not be relevant, but my cwd when I call c.build is different from the path I provide to c.build(path=foo).  

```
hamiltont@3:/localhdd$ sudo docker -D build . | tee output.log
[debug] archive.go:350 Skipping excluded path: .git
Sending build context to Docker daemon 263.8 MB
Sending build context to Docker daemon
Step 0 : FROM ubuntu:14.04
 ---> e54ca5efa2e9
Step 1 : RUN locale-gen en_US.UTF-8
 ---> Running in e5b6ad76d8e5
Generating locales...
  en_US.UTF-8... done
Generation complete.
 ---> 21733509e288
Removing intermediate container e5b6ad76d8e5
Step 2 : ENV LANG en_US.UTF-8
 ---> Running in aa17977bfe5a
<snip>
```

@hamiltont sorry to ask, but are you sure you test the right image ? 
We can see `[debug] archive.go:350 Skipping excluded path: .git` so it is skipped.

I mentioned above that it works correctly if I run `sudo build $MYPATH` myself, the problem is that when I run `c.build(path=mypath)` the `.dockerignore` file has no effect.

I wonder if [this](https://github.com/dotcloud/docker-py/blob/master/docker/utils/utils.py#L45) tar function is the culprit. Is it successfully including `.dockerignore` in the tarred context? 

EDIT: Don't think that is it, the `.dockerignore` file is still inside the container that I build with `docker-py`

@hamiltont oh sorry I thought we were in the docker repo, not docker-py :)

ping @shin- 

ha ha, I was wondering...

I'm thinking the issue is that the client needs to detect .dockerignore, build a list of excludes, and ensure the are not tarred into the context. docker probably does this in go, but docker-py probably needs to be trained to look for a .dockerignore file and react to it. tarfile has a filter option that would do nicely

Yes, I'm guessing this is implemented on the client side in Go, so it needs to be rewritten in Python. :)

There's an implementation here: https://github.com/orchardup/fig/pull/294


Resolving patch:
diff --git a/docker/client.py b/docker/client.py
--- a/docker/client.py
+++ b/docker/client.py
@@ -13,6 +13,7 @@
 #    limitations under the License.
 
 import json
+import os
 import re
 import shlex
 import struct
@@ -351,7 +352,12 @@ def build(self, path=None, tag=None, quiet=False, fileobj=None,
                               'git://', 'github.com/')):
             remote = path
         else:
-            context = utils.tar(path)
+            dockerignore = os.path.join(path, '.dockerignore')
+            exclude = None
+            if os.path.exists(dockerignore):
+                with open(dockerignore, 'r') as f:
+                    exclude = list(filter(bool, f.read().split('\n')))
+            context = utils.tar(path, exclude=exclude)
 
         if utils.compare_version('1.8', self._version) >= 0:
             stream = True
diff --git a/docker/utils/utils.py b/docker/utils/utils.py
--- a/docker/utils/utils.py
+++ b/docker/utils/utils.py
@@ -13,9 +13,11 @@
 #    limitations under the License.
 
 import io
+import os
 import tarfile
 import tempfile
 from distutils.version import StrictVersion
+from fnmatch import fnmatch
 
 import requests
 import six
@@ -42,10 +44,29 @@ def mkbuildcontext(dockerfile):
     return f
 
 
-def tar(path):
+def fnmatch_any(relpath, patterns):
+    return any([fnmatch(relpath, pattern) for pattern in patterns])
+
+
+def tar(path, exclude=None):
     f = tempfile.NamedTemporaryFile()
     t = tarfile.open(mode='w', fileobj=f)
-    t.add(path, arcname='.')
+    for dirpath, dirnames, filenames in os.walk(path):
+        relpath = os.path.relpath(dirpath, path)
+        if relpath == '.':
+            relpath = ''
+        if exclude is None:
+            fnames = filenames
+        else:
+            dirnames[:] = [d for d in dirnames
+                           if not fnmatch_any(os.path.join(relpath, d),
+                                              exclude)]
+            fnames = [name for name in filenames
+                      if not fnmatch_any(os.path.join(relpath, name),
+                                         exclude)]
+        for name in fnames:
+            arcname = os.path.join(relpath, name)
+            t.add(os.path.join(path, arcname), arcname=arcname)
     t.close()
     f.seek(0)
     return f


Test patch:
diff --git a/tests/integration_test.py b/tests/integration_test.py
--- a/tests/integration_test.py
+++ b/tests/integration_test.py
@@ -17,6 +17,7 @@
 import json
 import io
 import os
+import shutil
 import signal
 import tempfile
 import unittest
@@ -24,6 +25,8 @@
 import docker
 import six
 
+from tests.test import Cleanup
+
 # FIXME: missing tests for
 # export; history; import_image; insert; port; push; tag; get; load
 
@@ -820,6 +823,43 @@ def runTest(self):
         self.assertEqual(logs.find('HTTP code: 403'), -1)
 
 
+class TestBuildWithDockerignore(Cleanup, BaseTestCase):
+    def runTest(self):
+        if self.client._version < 1.8:
+            return
+
+        base_dir = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, base_dir)
+
+        with open(os.path.join(base_dir, 'Dockerfile'), 'w') as f:
+            f.write("\n".join([
+                'FROM busybox',
+                'MAINTAINER docker-py',
+                'ADD . /test',
+                'RUN ls -A /test',
+            ]))
+
+        with open(os.path.join(base_dir, '.dockerignore'), 'w') as f:
+            f.write("\n".join([
+                'node_modules',
+                '',  # empty line
+            ]))
+
+        with open(os.path.join(base_dir, 'not-ignored'), 'w') as f:
+            f.write("this file should not be ignored")
+
+        subdir = os.path.join(base_dir, 'node_modules', 'grunt-cli')
+        os.makedirs(subdir)
+        with open(os.path.join(subdir, 'grunt'), 'w') as f:
+            f.write("grunt")
+
+        stream = self.client.build(path=base_dir, stream=True)
+        logs = ''
+        for chunk in stream:
+            logs += chunk
+        self.assertFalse('node_modules' in logs)
+        self.assertTrue('not-ignored' in logs)
+
 #######################
 #  PY SPECIFIC TESTS  #
 #######################
diff --git a/tests/test.py b/tests/test.py
--- a/tests/test.py
+++ b/tests/test.py
@@ -17,7 +17,10 @@
 import io
 import json
 import os
+import shutil
 import signal
+import sys
+import tarfile
 import tempfile
 import unittest
 import gzip
@@ -58,9 +61,34 @@ def fake_resp(url, data=None, **kwargs):
     docker.client.DEFAULT_DOCKER_API_VERSION)
 
 
+class Cleanup(object):
+    if sys.version_info < (2, 7):
+        # Provide a basic implementation of addCleanup for Python < 2.7
+        def __init__(self, *args, **kwargs):
+            super(Cleanup, self).__init__(*args, **kwargs)
+            self._cleanups = []
+
+        def tearDown(self):
+            super(Cleanup, self).tearDown()
+            ok = True
+            while self._cleanups:
+                fn, args, kwargs = self._cleanups.pop(-1)
+                try:
+                    fn(*args, **kwargs)
+                except KeyboardInterrupt:
+                    raise
+                except:
+                    ok = False
+            if not ok:
+                raise
+
+        def addCleanup(self, function, *args, **kwargs):
+            self._cleanups.append((function, args, kwargs))
+
+
 @mock.patch.multiple('docker.Client', get=fake_request, post=fake_request,
                      put=fake_request, delete=fake_request)
-class DockerClientTest(unittest.TestCase):
+class DockerClientTest(Cleanup, unittest.TestCase):
     def setUp(self):
         self.client = docker.Client()
         # Force-clear authconfig to avoid tampering with the tests
@@ -1350,11 +1378,13 @@ def test_build_container_custom_context_gzip(self):
 
     def test_load_config_no_file(self):
         folder = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, folder)
         cfg = docker.auth.load_config(folder)
         self.assertTrue(cfg is not None)
 
     def test_load_config(self):
         folder = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, folder)
         f = open(os.path.join(folder, '.dockercfg'), 'w')
         auth_ = base64.b64encode(b'sakuya:izayoi').decode('ascii')
         f.write('auth = {0}\n'.format(auth_))
@@ -1369,6 +1399,26 @@ def test_load_config(self):
         self.assertEqual(cfg['email'], 'sakuya@scarlet.net')
         self.assertEqual(cfg.get('auth'), None)
 
+    def test_tar_with_excludes(self):
+        base = tempfile.mkdtemp()
+        self.addCleanup(shutil.rmtree, base)
+        for d in ['test/foo', 'bar']:
+            os.makedirs(os.path.join(base, d))
+            for f in ['a.txt', 'b.py', 'other.png']:
+                with open(os.path.join(base, d, f), 'w') as f:
+                    f.write("content")
+
+        for exclude, names in (
+            (['*.py'], ['bar/a.txt', 'bar/other.png',
+                        'test/foo/a.txt', 'test/foo/other.png']),
+            (['*.png', 'bar'], ['test/foo/a.txt', 'test/foo/b.py']),
+            (['test/foo', 'a.txt'], ['bar/a.txt', 'bar/b.py',
+                                     'bar/other.png']),
+        ):
+            archive = docker.utils.tar(base, exclude=exclude)
+            tar = tarfile.open(fileobj=archive)
+            self.assertEqual(sorted(tar.getnames()), names)
+
 
 if __name__ == '__main__':
     unittest.main()