Repository: PyCQA/pyflakes (python)

Issue:
Please also check python scripts that are not named *.py

*Original report by [morph-debian](https://launchpad.net/~morph-debian) on [Launchpad](https://bugs.launchpad.net/bugs/970465):*

------------------------------------

Hello,
as reported on Debian at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653890:

"""
It would be great if pyflakes could detect files that are python scripts
but are not named *.py. It would have to read the first two bytes of
each file to check for scripts and then read the rest of the line to
determine if it is a python script. It would need to process stuff like:

#!/usr/bin/python
#!/usr/local/bin/python
#!/usr/bin/env python
#!/usr/bin/python2.6
#!/usr/bin/python3.2
"""

Regards,
Sandro


Pull request description:
Find Python files without extensions
Previously, only files ending in `.py` would be detected as Python files while recursing a directory. This   pull request allows for detecting Python files by also looking for shebangs. I'm using the same technique that I've used in `autopep8` for years.

I timed this against the ~7000 files in `/opt/local/Library/Frameworks/Python.framework/Versions/3.6/`.

Three timings before this change:

```
0m23.091s
0m22.792s
0m22.745s
```

Three timings after this change:

```
0m23.046s
0m23.111s
0m22.860s
```

This closes #149.

Patch:
diff --git a/README.rst b/README.rst
index e84d334..aeb15f9 100644
--- a/README.rst
+++ b/README.rst
@@ -72,7 +72,7 @@ rebase your commits for you.
 
 All changes should include tests and pass flake8_.
 
-.. image:: https://api.travis-ci.org/PyCQA/pyflakes.svg
+.. image:: https://api.travis-ci.org/PyCQA/pyflakes.svg?branch=master
    :target: https://travis-ci.org/PyCQA/pyflakes
    :alt: Build status
 
diff --git a/pyflakes/api.py b/pyflakes/api.py
index a535bff..e30f920 100644
--- a/pyflakes/api.py
+++ b/pyflakes/api.py
@@ -5,6 +5,7 @@ from __future__ import with_statement
 
 import sys
 import os
+import re
 import _ast
 
 from pyflakes import checker, __version__
@@ -13,6 +14,9 @@ from pyflakes import reporter as modReporter
 __all__ = ['check', 'checkPath', 'checkRecursive', 'iterSourceCode', 'main']
 
 
+PYTHON_SHEBANG_REGEX = re.compile(br'^#!.*\bpython[23]?\b\s*$')
+
+
 def check(codeString, filename, reporter=None):
     """
     Check the Python source given by C{codeString} for flakes.
@@ -108,6 +112,25 @@ def checkPath(filename, reporter=None):
     return check(codestr, filename, reporter)
 
 
+def isPythonFile(filename):
+    """Return True if filename points to a Python file."""
+    if filename.endswith('.py'):
+        return True
+
+    max_bytes = 128
+
+    try:
+        with open(filename, 'rb') as f:
+            text = f.read(max_bytes)
+            if not text:
+                return False
+    except IOError:
+        return False
+
+    first_line = text.splitlines()[0]
+    return PYTHON_SHEBANG_REGEX.match(first_line)
+
+
 def iterSourceCode(paths):
     """
     Iterate over all Python source files in C{paths}.
@@ -120,8 +143,9 @@ def iterSourceCode(paths):
         if os.path.isdir(path):
             for dirpath, dirnames, filenames in os.walk(path):
                 for filename in filenames:
-                    if filename.endswith('.py'):
-                        yield os.path.join(dirpath, filename)
+                    full_path = os.path.join(dirpath, filename)
+                    if isPythonFile(full_path):
+                        yield full_path
         else:
             yield path
 


Test patch:
diff --git a/pyflakes/test/test_api.py b/pyflakes/test/test_api.py
index 51b0027..a1e0be2 100644
--- a/pyflakes/test/test_api.py
+++ b/pyflakes/test/test_api.py
@@ -187,6 +187,22 @@ class TestIterSourceCode(TestCase):
             sorted(iterSourceCode([self.tempdir])),
             sorted([apath, bpath, cpath]))
 
+    def test_shebang(self):
+        """
+        Find Python files that don't end with `.py`, but contain a Python
+        shebang.
+        """
+        apath = os.path.join(self.tempdir, 'a')
+        fd = open(apath, 'w')
+        fd.write('#!/usr/bin/env python\n')
+        fd.close()
+
+        self.makeEmptyFile('b')
+
+        self.assertEqual(
+            list(iterSourceCode([self.tempdir])),
+            list([apath]))
+
     def test_multipleDirectories(self):
         """
         L{iterSourceCode} can be given multiple directories.  It will recurse