Repository: python-poetry/poetry

Issue:
Building sdist is slow with wide exclude pattern
<!--
  Hi there! Thank you for wanting to make Poetry better.

  Before you submit this; let's make sure of a few things.
  Please make sure the following boxes are ticked if they are correct.
  If not, please try and fulfill these first.
-->

<!-- Checked checkbox should look like this: [x] -->
- [x] I have searched the [issues](https://github.com/sdispater/poetry/issues) of this repo and believe that this is not a duplicate.

## Issue

I'm working on a project with the following `pyproject.toml`: https://gist.github.com/Lothiraldan/2c0b5ce0171e8450490e3b493e7c2960 and I want to ship a React project inside my package.

Include and exclude are working great, far easier to configure than `MANIFEST.IN` IMHO.

My issue is that the builder code is walking all of the directories and for each file check that it's not in the excluded list. One of my exclude pattern is `"balto/web_interfaces/balto_react/node_modules/**/*"` which generates a lot of matching files. The length of the excluded file list is `28761` in my case.

This makes the following line https://github.com/sdispater/poetry/blob/master/poetry/masonry/builders/sdist.py#L281 quite slow. A build takes about 4 minutes on my laptop.

Here is a `py-spy` dump of the process:
```
Collecting samples from 'pid: 31302' (python v3.6.6)
Total Samples 5100
GIL: 0.00%, Active: 95.50%, Threads: 1

  %Own   %Total  OwnTime  TotalTime  Function (filename:line)                                                                                                                                                        
 38.00%  51.00%   10.55s    14.14s   __eq__ (4/python3.6/pathlib.py:736)
 29.00%  93.00%    6.64s    24.87s   <listcomp> (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/poetry/masonry/builders/sdist.py:281)
 16.50%  16.50%    4.38s     4.38s   _cparts (4/python3.6/pathlib.py:728)
  7.50%  11.00%    2.81s     3.65s   __eq__ (4/python3.6/pathlib.py:734)
  1.50%   1.50%   0.015s    0.015s   run (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/cleo/application.py:104)
  1.00%   1.00%   0.130s    0.130s   _cparts (4/python3.6/pathlib.py:724)
  1.00%   2.00%   0.315s    0.435s   __eq__ (4/python3.6/pathlib.py:733)
  0.50%   0.50%   0.025s    0.035s   parse_parts (4/python3.6/pathlib.py:87)
  0.50%   0.50%   0.165s    0.180s   wrapped (4/python3.6/pathlib.py:387)
  0.00%   1.00%   0.000s    0.355s   find_packages (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/poetry/masonry/builders/sdist.py:277)
  0.00%   0.00%   0.030s    0.030s   _get_sep (4/python3.6/posixpath.py:45)
  0.00%  94.00%   0.000s    25.36s   execute (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/cleo/commands/command.py:107)
  0.00%   0.00%   0.010s    0.075s   <listcomp> (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/poetry/masonry/builders/sdist.py:276)
  0.00%   0.00%   0.025s    0.025s   _select_from (4/python3.6/pathlib.py:529)
  0.00%  94.00%   0.000s    25.36s   do_run (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/cleo/application.py:197)
  0.00%  94.00%   0.000s    25.36s   build (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/poetry/masonry/builder.py:21)
  0.00%  94.00%   0.000s    25.36s   do_run (/home/lothiraldan/.local/pipx/venvs/poetry/lib64/python3.6/site-packages/poetry/console/application.py:88)

Press Control-C to quit, or ? for help.
```

I have some ideas about how to make it faster, I will send some patches if that's okay.


Discussion:

Resolving patch:
diff --git a/poetry/masonry/builders/builder.py b/poetry/masonry/builders/builder.py
--- a/poetry/masonry/builders/builder.py
+++ b/poetry/masonry/builders/builder.py
@@ -63,7 +63,10 @@ def find_excluded_files(self):  # type: () -> list
 
             result.append(file)
 
-        return result
+        # The list of excluded files might be big and we will do a lot
+        # containment check (x in excluded).
+        # Returning a set make those tests much much faster.
+        return set(result)
 
     def find_files_to_add(self, exclude_build=True):  # type: () -> list
         """


Test patch: