Repository: google/yapf
Pull request #851 (2020-08-04 02:40:44)

Adds walrus test and python 3.7/3.8 tox definitions
Give a mouse a cookie...

I was looking at this issue: https://github.com/google/yapf/issues/787
Was curious if lib2to3 supported the walrus operator since the bug was reported (it does: https://bugs.python.org/issue36541#msg365718)

Added a test on the thing, which necessitated adding 3.8 to tox.ini.

Seemed silly to have 3.8 in there but not 3.7, so added that, too.

Also, interestingly enough, the lib2to3 walrus operator support includes 3.7, even though it wasn't introduced until 3.8. I don't think that's a compelling reason to support it in yapf, though.

coveralls: 
[![Coverage Status](https://coveralls.io/builds/32518316/badge)](https://coveralls.io/builds/32518316)

Coverage decreased (-0.05%) to 96.267% when pulling **55274a2c6d145e03b0e304ae47dae74078da7b87 on brianmego:add_walrus_test** into **51ffe2d07930a509ecb2ef454a7b251eeeac0a59 on google:master**.


Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
new file mode 100644
index 0000000..3bd65c2
--- /dev/null
+++ b/.pre-commit-config.yaml
@@ -0,0 +1,30 @@
+# File introduces automated checks triggered on git events
+# to enable run `pip install pre-commit && pre-commit install`
+
+repos:
+  - repo: local
+    hooks:
+      - id: yapf
+        name: yapf
+        language: python
+        entry: yapf
+        args: [-i, -vv]
+        types: [python]
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+      - id: trailing-whitespace
+      - id: check-docstring-first
+      - id: check-json
+      - id: check-added-large-files
+      - id: check-yaml
+      - id: debug-statements
+      - id: requirements-txt-fixer
+      - id: check-merge-conflict
+      - id: double-quote-string-fixer
+      - id: end-of-file-fixer
+      - id: sort-simple-yaml
+  - repo: meta
+    hooks:
+      - id: check-hooks-apply
+      - id: check-useless-excludes
diff --git a/.pre-commit-config.yml b/.pre-commit-config.yml
new file mode 100644
index 0000000..3bd65c2
--- /dev/null
+++ b/.pre-commit-config.yml
@@ -0,0 +1,30 @@
+# File introduces automated checks triggered on git events
+# to enable run `pip install pre-commit && pre-commit install`
+
+repos:
+  - repo: local
+    hooks:
+      - id: yapf
+        name: yapf
+        language: python
+        entry: yapf
+        args: [-i, -vv]
+        types: [python]
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+      - id: trailing-whitespace
+      - id: check-docstring-first
+      - id: check-json
+      - id: check-added-large-files
+      - id: check-yaml
+      - id: debug-statements
+      - id: requirements-txt-fixer
+      - id: check-merge-conflict
+      - id: double-quote-string-fixer
+      - id: end-of-file-fixer
+      - id: sort-simple-yaml
+  - repo: meta
+    hooks:
+      - id: check-hooks-apply
+      - id: check-useless-excludes
diff --git a/.pre-commit-hooks.yaml b/.pre-commit-hooks.yaml
new file mode 100644
index 0000000..3eba1f2
--- /dev/null
+++ b/.pre-commit-hooks.yaml
@@ -0,0 +1,9 @@
+# File configures YAPF to be used as a git hook with https://github.com/pre-commit/pre-commit
+
+- id: yapf
+  name: yapf
+  description: "A formatter for Python files."
+  entry: yapf
+  args: [-i] #inplace
+  language: python
+  types: [python]
diff --git a/.pre-commit-hooks.yml b/.pre-commit-hooks.yml
new file mode 100644
index 0000000..3eba1f2
--- /dev/null
+++ b/.pre-commit-hooks.yml
@@ -0,0 +1,9 @@
+# File configures YAPF to be used as a git hook with https://github.com/pre-commit/pre-commit
+
+- id: yapf
+  name: yapf
+  description: "A formatter for Python files."
+  entry: yapf
+  args: [-i] #inplace
+  language: python
+  types: [python]
diff --git a/CHANGELOG b/CHANGELOG
index 0078296..6b10382 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -2,6 +2,15 @@
 # All notable changes to this project will be documented in this file.
 # This project adheres to [Semantic Versioning](http://semver.org/).
 
+## [0.31.0] UNRELEASED
+### Added
+- Add 'BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES' to support setting
+  a custom number of blank lines between top-level imports and variable
+  definitions.
+- Ignore end of line `# copybara:` directives when checking line length.
+### Fixed
+- Exclude directories on Windows.
+
 ## [0.30.0] 2020-04-23
 ### Added
 - Added `SPACES_AROUND_LIST_DELIMITERS`, `SPACES_AROUND_DICT_DELIMITERS`,
diff --git a/CONTRIBUTORS b/CONTRIBUTORS
index 284b13f..054ef26 100644
--- a/CONTRIBUTORS
+++ b/CONTRIBUTORS
@@ -14,3 +14,4 @@ Eli Bendersky <eliben@google.com>
 Sam Clegg <sbc@google.com>
 Łukasz Langa <ambv@fb.com>
 Oleg Butuzov <butuzov@made.ua>
+Mauricio Herrera Cuadra <mauricio@arareko.net>
diff --git a/README.rst b/README.rst
index e2db751..e013c52 100644
--- a/README.rst
+++ b/README.rst
@@ -37,8 +37,6 @@ The ultimate goal is that the code YAPF produces is as good as the code that a
 programmer would write if they were following the style guide. It takes away
 some of the drudgery of maintaining your code.
 
-Try out YAPF with this `online demo <https://yapf.now.sh>`_.
-
 .. footer::
 
     YAPF is not an official Google product (experimental or otherwise), it is
@@ -321,6 +319,42 @@ The ``in_place`` argument saves the reformatted code back to the file:
     >>> print(open("foo.py").read())  # contents of file (now fixed)
     a == b
 
+Formatting diffs
+================
+
+Options::
+
+    usage: yapf-diff [-h] [-i] [-p NUM] [--regex PATTERN] [--iregex PATTERN][-v]
+                     [--style STYLE] [--binary BINARY]
+
+    This script reads input from a unified diff and reformats all the changed
+    lines. This is useful to reformat all the lines touched by a specific patch.
+    Example usage for git/svn users:
+
+      git diff -U0 --no-color --relative HEAD^ | yapf-diff -i
+      svn diff --diff-cmd=diff -x-U0 | yapf-diff -p0 -i
+
+    It should be noted that the filename contained in the diff is used
+    unmodified to determine the source file to update. Users calling this script
+    directly should be careful to ensure that the path in the diff is correct
+    relative to the current working directory.
+
+    optional arguments:
+      -h, --help            show this help message and exit
+      -i, --in-place        apply edits to files instead of displaying a diff
+      -p NUM, --prefix NUM  strip the smallest prefix containing P slashes
+      --regex PATTERN       custom pattern selecting file paths to reformat
+                            (case sensitive, overrides -iregex)
+      --iregex PATTERN      custom pattern selecting file paths to reformat
+                            (case insensitive, overridden by -regex)
+      -v, --verbose         be more verbose, ineffective without -i
+      --style STYLE         specify formatting style: either a style name (for
+                            example "pep8" or "google"), or the name of a file
+                            with style settings. The default is pep8 unless a
+                            .style.yapf or setup.cfg file located in one of the
+                            parent directories of the source file (or current
+                            directory for stdin)
+      --binary BINARY       location of binary to use for yapf
 
 Knobs
 =====
@@ -401,6 +435,10 @@ Knobs
         class Bar:
             pass
 
+``BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES``
+    Sets the number of desired blank lines between top-level imports and
+    variable definitions. Useful for compatibility with tools like isort.
+
 ``COALESCE_BRACKETS``
     Do not split consecutive brackets. Only relevant when
     ``DEDENT_CLOSING_BRACKETS`` or ``INDENT_CLOSING_BRACKETS``
@@ -567,7 +605,7 @@ Knobs
         [1, 2]
 
     will be formatted as:
-    
+
     .. code-block:: python
 
         [ 1, 2 ]
@@ -665,16 +703,16 @@ Knobs
     ``b`` in this code:
 
     .. code-block:: python
-      
+
       abcdef(
           aReallyLongThing: int,
           b: [Int,
               Int])
-   
+
     With the new knob this is split as:
 
     .. code-block:: python
-      
+
       abcdef(
           aReallyLongThing: int,
           b: [Int, Int])
diff --git a/setup.py b/setup.py
index 21e9470..fe2a046 100644
--- a/setup.py
+++ b/setup.py
@@ -65,7 +65,10 @@ with codecs.open('README.rst', 'r', 'utf-8') as fd:
           'Topic :: Software Development :: Quality Assurance',
       ],
       entry_points={
-          'console_scripts': ['yapf = yapf:run_main'],
+          'console_scripts': [
+              'yapf = yapf:run_main',
+              'yapf-diff = yapf.third_party.yapf_diff.yapf_diff:main',
+          ],
       },
       cmdclass={
           'test': RunTests,
diff --git a/tox.ini b/tox.ini
index dca30d2..f42b884 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
 [tox]
-envlist=py27,py34,py35,py36
+envlist=py27,py34,py35,py36,py37,py38
 
 [testenv]
 commands=
diff --git a/yapf/__init__.py b/yapf/__init__.py
index f03c932..3aca8bd 100644
--- a/yapf/__init__.py
+++ b/yapf/__init__.py
@@ -57,7 +57,8 @@ def main(argv):
   Raises:
     YapfError: if none of the supplied files were Python files.
   """
-  args = _ParseArguments(argv)
+  parser = _BuildParser()
+  args = parser.parse_args(argv[1:])
   if args.version:
     print('yapf {}'.format(__version__))
     return 0
@@ -65,7 +66,7 @@ def main(argv):
   style_config = args.style
 
   if args.style_help:
-    print_help(args)
+    _PrintHelp(args)
     return 0
 
   if args.lines and len(args.files) > 1:
@@ -136,7 +137,7 @@ def main(argv):
   return 1 if changed and (args.diff or args.quiet) else 0
 
 
-def print_help(args):
+def _PrintHelp(args):
   """Prints the help menu."""
 
   if args.style is None and not args.no_local_style:
@@ -268,15 +269,11 @@ def _GetLines(line_strings):
   return lines
 
 
-def _ParseArguments(argv):
-  """Parse the command line arguments.
-
-  Arguments:
-    argv: command-line arguments, such as sys.argv (including the program name
-      in argv[0]).
+def _BuildParser():
+  """Constructs the parser for the command line arguments.
 
   Returns:
-    An object containing the arguments used to invoke the program.
+    An ArgumentParser instance for the CLI.
   """
   parser = argparse.ArgumentParser(description='Formatter for Python code.')
   parser.add_argument(
@@ -357,7 +354,7 @@ def _ParseArguments(argv):
 
   parser.add_argument(
       'files', nargs='*', help='reads from stdin when no files are specified.')
-  return parser.parse_args(argv[1:])
+  return parser
 
 
 def run_main():  # pylint: disable=invalid-name
diff --git a/yapf/third_party/__init__.py b/yapf/third_party/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/yapf/third_party/yapf_diff/LICENSE b/yapf/third_party/yapf_diff/LICENSE
new file mode 100644
index 0000000..f9dc506
--- /dev/null
+++ b/yapf/third_party/yapf_diff/LICENSE
@@ -0,0 +1,219 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+    1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+    2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+    3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+    4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+    5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+    6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+    7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+    8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+    9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+    END OF TERMS AND CONDITIONS
+
+    APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+    Copyright [yyyy] [name of copyright owner]
+
+    Licensed under the Apache License, Version 2.0 (the "License");
+    you may not use this file except in compliance with the License.
+    You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+
+
+--- LLVM Exceptions to the Apache 2.0 License ----
+
+As an exception, if, as a result of your compiling your source code, portions
+of this Software are embedded into an Object form of such source code, you
+may redistribute such embedded portions in such Object form without complying
+with the conditions of Sections 4(a), 4(b) and 4(d) of the License.
+
+In addition, if you combine or link compiled forms of this Software with
+software that is licensed under the GPLv2 ("Combined Software") and if a
+court of competent jurisdiction determines that the patent provision (Section
+3), the indemnity provision (Section 9) or other Section of the License
+conflicts with the conditions of the GPLv2, you may retroactively and
+prospectively choose to deem waived or otherwise exclude such Section(s) of
+the License, but only in their entirety and only with respect to the Combined
+Software.
+
diff --git a/yapf/third_party/yapf_diff/__init__.py b/yapf/third_party/yapf_diff/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/yapf/third_party/yapf_diff/yapf_diff.py b/yapf/third_party/yapf_diff/yapf_diff.py
new file mode 100644
index 0000000..3377231
--- /dev/null
+++ b/yapf/third_party/yapf_diff/yapf_diff.py
@@ -0,0 +1,146 @@
+# Modified copy of clang-format-diff.py that works with yapf.
+#
+# Licensed under the Apache License, Version 2.0 (the "License") with LLVM
+# Exceptions; you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://llvm.org/LICENSE.txt
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""
+This script reads input from a unified diff and reformats all the changed
+lines. This is useful to reformat all the lines touched by a specific patch.
+Example usage for git/svn users:
+
+  git diff -U0 --no-color --relative HEAD^ | yapf-diff -i
+  svn diff --diff-cmd=diff -x-U0 | yapf-diff -p0 -i
+
+It should be noted that the filename contained in the diff is used unmodified
+to determine the source file to update. Users calling this script directly
+should be careful to ensure that the path in the diff is correct relative to the
+current working directory.
+"""
+from __future__ import absolute_import, division, print_function
+
+import argparse
+import difflib
+import re
+import subprocess
+import sys
+
+if sys.version_info.major >= 3:
+  from io import StringIO
+else:
+  from io import BytesIO as StringIO
+
+
+def main():
+  parser = argparse.ArgumentParser(
+      description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+  parser.add_argument(
+      '-i',
+      '--in-place',
+      action='store_true',
+      default=False,
+      help='apply edits to files instead of displaying a diff')
+  parser.add_argument(
+      '-p',
+      '--prefix',
+      metavar='NUM',
+      default=1,
+      help='strip the smallest prefix containing P slashes')
+  parser.add_argument(
+      '--regex',
+      metavar='PATTERN',
+      default=None,
+      help='custom pattern selecting file paths to reformat '
+      '(case sensitive, overrides -iregex)')
+  parser.add_argument(
+      '--iregex',
+      metavar='PATTERN',
+      default=r'.*\.(py)',
+      help='custom pattern selecting file paths to reformat '
+      '(case insensitive, overridden by -regex)')
+  parser.add_argument(
+      '-v',
+      '--verbose',
+      action='store_true',
+      help='be more verbose, ineffective without -i')
+  parser.add_argument(
+      '--style',
+      help='specify formatting style: either a style name (for '
+      'example "pep8" or "google"), or the name of a file with '
+      'style settings. The default is pep8 unless a '
+      '.style.yapf or setup.cfg file located in one of the '
+      'parent directories of the source file (or current '
+      'directory for stdin)')
+  parser.add_argument(
+      '--binary', default='yapf', help='location of binary to use for yapf')
+  args = parser.parse_args()
+
+  # Extract changed lines for each file.
+  filename = None
+  lines_by_file = {}
+  for line in sys.stdin:
+    match = re.search(r'^\+\+\+\ (.*?/){%s}(\S*)' % args.prefix, line)
+    if match:
+      filename = match.group(2)
+    if filename is None:
+      continue
+
+    if args.regex is not None:
+      if not re.match('^%s$' % args.regex, filename):
+        continue
+    else:
+      if not re.match('^%s$' % args.iregex, filename, re.IGNORECASE):
+        continue
+
+    match = re.search(r'^@@.*\+(\d+)(,(\d+))?', line)
+    if match:
+      start_line = int(match.group(1))
+      line_count = 1
+      if match.group(3):
+        line_count = int(match.group(3))
+      if line_count == 0:
+        continue
+      end_line = start_line + line_count - 1
+      lines_by_file.setdefault(filename, []).extend(
+          ['--lines', str(start_line) + '-' + str(end_line)])
+
+  # Reformat files containing changes in place.
+  for filename, lines in lines_by_file.items():
+    if args.in_place and args.verbose:
+      print('Formatting {}'.format(filename))
+    command = [args.binary, filename]
+    if args.in_place:
+      command.append('-i')
+    command.extend(lines)
+    if args.style:
+      command.extend(['--style', args.style])
+    p = subprocess.Popen(
+        command,
+        stdout=subprocess.PIPE,
+        stderr=None,
+        stdin=subprocess.PIPE,
+        universal_newlines=True)
+    stdout, stderr = p.communicate()
+    if p.returncode != 0:
+      sys.exit(p.returncode)
+
+    if not args.in_place:
+      with open(filename) as f:
+        code = f.readlines()
+      formatted_code = StringIO(stdout).readlines()
+      diff = difflib.unified_diff(code, formatted_code, filename, filename,
+                                  '(before formatting)', '(after formatting)')
+      diff_string = ''.join(diff)
+      if len(diff_string) > 0:
+        sys.stdout.write(diff_string)
+
+
+if __name__ == '__main__':
+  main()
diff --git a/yapf/yapflib/file_resources.py b/yapf/yapflib/file_resources.py
index 50b4687..4c6bf38 100644
--- a/yapf/yapflib/file_resources.py
+++ b/yapf/yapflib/file_resources.py
@@ -154,6 +154,7 @@ def _FindPythonFiles(filenames, recursive, exclude):
   """Find all Python files."""
   if exclude and any(e.startswith('./') for e in exclude):
     raise errors.YapfError("path in '--exclude' should not start with ./")
+  exclude = exclude and [e.rstrip("/" + os.path.sep) for e in exclude]
 
   python_files = []
   for filename in filenames:
@@ -186,10 +187,10 @@ def _FindPythonFiles(filenames, recursive, exclude):
 
 def IsIgnored(path, exclude):
   """Return True if filename matches any patterns in exclude."""
-  path = path.lstrip('/')
-  while path.startswith('./'):
+  path = path.lstrip(os.path.sep)
+  while path.startswith('.' + os.path.sep):
     path = path[2:]
-  return any(fnmatch.fnmatch(path, e.rstrip('/')) for e in exclude)
+  return any(fnmatch.fnmatch(path, e.rstrip(os.path.sep)) for e in exclude)
 
 
 def IsPythonFile(filename):
diff --git a/yapf/yapflib/format_decision_state.py b/yapf/yapflib/format_decision_state.py
index 577ae10..dbe3d81 100644
--- a/yapf/yapflib/format_decision_state.py
+++ b/yapf/yapflib/format_decision_state.py
@@ -741,8 +741,8 @@ class FormatDecisionState(object):
 
     # Calculate the penalty for overflowing the column limit.
     penalty = 0
-    if (not current.is_pylint_comment and not current.is_pytype_comment and
-        self.column > self.column_limit):
+    if (not current.is_pylint_comment and not current.is_pytype_comment
+        and not current.is_copybara_comment and self.column > self.column_limit):
       excess_characters = self.column - self.column_limit
       penalty += style.Get('SPLIT_PENALTY_EXCESS_CHARACTER') * excess_characters
 
diff --git a/yapf/yapflib/format_token.py b/yapf/yapflib/format_token.py
index 0ed983f..92c26ee 100644
--- a/yapf/yapflib/format_token.py
+++ b/yapf/yapflib/format_token.py
@@ -378,3 +378,8 @@ class FormatToken(object):
   def is_pytype_comment(self):
     return self.is_comment and re.match(r'#.*\bpytype:\s*(disable|enable)=',
                                         self.value)
+
+  @property
+  def is_copybara_comment(self):
+    return self.is_comment and re.match(r'#.*\bcopybara:(strip|insert|replace)',
+                                        self.value)
diff --git a/yapf/yapflib/py3compat.py b/yapf/yapflib/py3compat.py
index 5373b01..7c90523 100644
--- a/yapf/yapflib/py3compat.py
+++ b/yapf/yapflib/py3compat.py
@@ -20,6 +20,8 @@ import sys
 
 PY3 = sys.version_info[0] >= 3
 PY36 = sys.version_info[0] >= 3 and sys.version_info[1] >= 6
+PY37 = sys.version_info[0] >= 3 and sys.version_info[1] >= 7
+PY38 = sys.version_info[0] >= 3 and sys.version_info[1] >= 8
 
 if PY3:
   StringIO = io.StringIO
diff --git a/yapf/yapflib/reformatter.py b/yapf/yapflib/reformatter.py
index 7ec9888..cd940bb 100644
--- a/yapf/yapflib/reformatter.py
+++ b/yapf/yapflib/reformatter.py
@@ -115,10 +115,6 @@ def _RetainHorizontalSpacing(uwline):
 
 def _RetainRequiredVerticalSpacing(cur_uwline, prev_uwline, lines):
   """Retain all vertical spacing between lines."""
-  if cur_uwline.disable and (not prev_uwline or prev_uwline.disable):
-    # If both lines are disabled we aren't allowed to reformat anything.
-    lines = set()
-
   prev_tok = None
   if prev_uwline is not None:
     prev_tok = prev_uwline.last
@@ -160,7 +156,7 @@ def _RetainRequiredVerticalSpacingBetweenTokens(cur_tok, prev_tok, lines):
   if cur_tok.is_comment and not prev_tok.is_comment:
     # Don't adjust between a comment and non-comment.
     pass
-  elif lines and (cur_lineno in lines or prev_lineno in lines):
+  elif lines and lines.intersection(range(prev_lineno, cur_lineno + 1)):
     desired_newlines = cur_tok.whitespace_prefix.count('\n')
     whitespace_lines = range(prev_lineno + 1, cur_lineno)
     deletable_lines = len(lines.intersection(whitespace_lines))
@@ -263,7 +259,8 @@ def _CanPlaceOnSingleLine(uwline):
   indent_amt = style.Get('INDENT_WIDTH') * uwline.depth
   last = uwline.last
   last_index = -1
-  if last.is_pylint_comment or last.is_pytype_comment:
+  if (last.is_pylint_comment or last.is_pytype_comment
+      or last.is_copybara_comment):
     last = last.previous_token
     last_index = -2
   if last is None:
@@ -650,6 +647,14 @@ def _CalculateNumberOfNewlines(first_token, indent_depth, prev_uwline,
     # The docstring shouldn't have a newline before it.
     return NO_BLANK_LINES
 
+  if first_token.is_name and not indent_depth:
+    if (prev_uwline.first.value == 'from' or
+        prev_uwline.first.value == 'import'):
+      # Support custom number of blank lines between top-level imports and
+      # variable definitions.
+      return 1 + style.Get(
+          'BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES')
+
   prev_last_token = prev_uwline.last
   if prev_last_token.is_docstring:
     if (not indent_depth and first_token.value in {'class', 'def', 'async'}):
diff --git a/yapf/yapflib/style.py b/yapf/yapflib/style.py
index 7cb3180..b01c231 100644
--- a/yapf/yapflib/style.py
+++ b/yapf/yapflib/style.py
@@ -104,6 +104,9 @@ _STYLE_HELP = dict(
     BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION=textwrap.dedent("""\
       Number of blank lines surrounding top-level function and class
       definitions."""),
+    BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES=textwrap.dedent("""\
+      Number of blank lines between top-level imports and variable
+      definitions."""),
     COALESCE_BRACKETS=textwrap.dedent("""\
       Do not split consecutive brackets. Only relevant when
       dedent_closing_brackets is set. For example:
@@ -419,6 +422,7 @@ def CreatePEP8Style():
       BLANK_LINE_BEFORE_CLASS_DOCSTRING=False,
       BLANK_LINE_BEFORE_MODULE_DOCSTRING=False,
       BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION=2,
+      BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES=1,
       COALESCE_BRACKETS=False,
       COLUMN_LIMIT=79,
       CONTINUATION_ALIGN_STYLE='SPACE',
@@ -606,6 +610,7 @@ _STYLE_OPTION_VALUE_CONVERTER = dict(
     BLANK_LINE_BEFORE_CLASS_DOCSTRING=_BoolConverter,
     BLANK_LINE_BEFORE_MODULE_DOCSTRING=_BoolConverter,
     BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION=int,
+    BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES=int,
     COALESCE_BRACKETS=_BoolConverter,
     COLUMN_LIMIT=int,
     CONTINUATION_ALIGN_STYLE=_ContinuationAlignStyleStringConverter,
diff --git a/yapf/yapflib/unwrapped_line.py b/yapf/yapflib/unwrapped_line.py
index 38501c0..475bd6e 100644
--- a/yapf/yapflib/unwrapped_line.py
+++ b/yapf/yapflib/unwrapped_line.py
@@ -366,9 +366,9 @@ def _SpaceRequiredBetween(left, right, is_line_disabled):
     if lval == '**' or rval == '**':
       # Space around the "power" operator.
       return style.Get('SPACES_AROUND_POWER_OPERATOR')
-    # Enforce spaces around binary operators except the blacklisted ones.
-    blacklist = style.Get('NO_SPACES_AROUND_SELECTED_BINARY_OPERATORS')
-    if lval in blacklist or rval in blacklist:
+    # Enforce spaces around binary operators except the blocked ones.
+    block_list = style.Get('NO_SPACES_AROUND_SELECTED_BINARY_OPERATORS')
+    if lval in block_list or rval in block_list:
       return False
     if style.Get('ARITHMETIC_PRECEDENCE_INDICATION'):
       if _PriorityIndicatingNoSpace(left) or _PriorityIndicatingNoSpace(right):


Test patch:
diff --git a/yapftests/file_resources_test.py b/yapftests/file_resources_test.py
index c91d93c..2bf68ab 100644
--- a/yapftests/file_resources_test.py
+++ b/yapftests/file_resources_test.py
@@ -152,8 +152,8 @@ class GetCommandLineFilesTest(unittest.TestCase):
     self.old_dir = os.getcwd()
 
   def tearDown(self):  # pylint: disable=g-missing-super-call
-    shutil.rmtree(self.test_tmpdir)
     os.chdir(self.old_dir)
+    shutil.rmtree(self.test_tmpdir)
 
   def _make_test_dir(self, name):
     fullpath = os.path.normpath(os.path.join(self.test_tmpdir, name))
@@ -313,7 +313,8 @@ class GetCommandLineFilesTest(unittest.TestCase):
                                                'test2/testinner/',
                                            ]))
 
-    self.assertEqual(found, ['test3/foo/bar/bas/xxx/testfile3.py'])
+    self.assertEqual(
+        found, ['test3/foo/bar/bas/xxx/testfile3.py'.replace("/", os.path.sep)])
 
     found = sorted(
         file_resources.GetCommandLineFiles(['.'],
@@ -323,7 +324,8 @@ class GetCommandLineFilesTest(unittest.TestCase):
                                                'test3',
                                            ]))
 
-    self.assertEqual(found, ['./test2/testinner/testfile2.py'])
+    self.assertEqual(
+        found, ['./test2/testinner/testfile2.py'.replace("/", os.path.sep)])
 
   def test_find_with_excluded_current_dir(self):
     with self.assertRaises(errors.YapfError):
@@ -386,7 +388,7 @@ class IsIgnoredTest(unittest.TestCase):
 
   def test_trailing_slash(self):
     self.assertTrue(file_resources.IsIgnored('z', ['z']))
-    self.assertTrue(file_resources.IsIgnored('z', ['z/']))
+    self.assertTrue(file_resources.IsIgnored('z', ['z' + os.path.sep]))
 
 
 class BufferedByteStream(object):
diff --git a/yapftests/reformatter_basic_test.py b/yapftests/reformatter_basic_test.py
index 50cfdbc..a67e4c4 100644
--- a/yapftests/reformatter_basic_test.py
+++ b/yapftests/reformatter_basic_test.py
@@ -253,6 +253,95 @@ class BasicReformatterTest(yapf_test_helper.YAPFTest):
     uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
     self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
 
+  def testBlankLinesBetweenTopLevelImportsAndVariables(self):
+    unformatted_code = textwrap.dedent("""\
+        import foo as bar
+        VAR = 'baz'
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        import foo as bar
+
+        VAR = 'baz'
+        """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
+
+    unformatted_code = textwrap.dedent("""\
+        import foo as bar
+
+        VAR = 'baz'
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        import foo as bar
+
+
+        VAR = 'baz'
+        """)
+    try:
+      style.SetGlobalStyle(
+          style.CreateStyleFromConfig(
+              '{based_on_style: yapf, blank_lines_between_top_level_imports_and_variables: 2}'
+          ))
+      uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+      self.assertCodeEqual(expected_formatted_code,
+                           reformatter.Reformat(uwlines))
+    finally:
+      style.SetGlobalStyle(style.CreateYapfStyle())
+
+    unformatted_code = textwrap.dedent("""\
+        import foo as bar
+        # Some comment
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        import foo as bar
+        # Some comment
+        """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
+
+    unformatted_code = textwrap.dedent("""\
+        import foo as bar
+        class Baz():
+          pass
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        import foo as bar
+
+
+        class Baz():
+          pass
+        """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
+
+    unformatted_code = textwrap.dedent("""\
+        import foo as bar
+        def foobar():
+          pass
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        import foo as bar
+
+
+        def foobar():
+          pass
+        """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
+
+    unformatted_code = textwrap.dedent("""\
+        def foobar():
+          from foo import Bar
+          Bar.baz()
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        def foobar():
+          from foo import Bar
+          Bar.baz()
+        """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected_formatted_code, reformatter.Reformat(uwlines))
+
   def testBlankLinesAtEndOfFile(self):
     unformatted_code = textwrap.dedent("""\
         def foobar(): # foo
@@ -3033,6 +3122,19 @@ my_dict = {
     finally:
       style.SetGlobalStyle(style.CreateYapfStyle())
 
+  @unittest.skipUnless(py3compat.PY38, 'Requires Python 3.8')
+  def testWalrus(self):
+    unformatted_code = textwrap.dedent("""\
+      if (x  :=  len([1]*1000)>100):
+        print(f'{x} is pretty big' )
+    """)
+    expected = textwrap.dedent("""\
+      if (x := len([1] * 1000) > 100):
+        print(f'{x} is pretty big')
+    """)
+    uwlines = yapf_test_helper.ParseAndUnwrap(unformatted_code)
+    self.assertCodeEqual(expected, reformatter.Reformat(uwlines))
+
 
 if __name__ == '__main__':
   unittest.main()
diff --git a/yapftests/reformatter_buganizer_test.py b/yapftests/reformatter_buganizer_test.py
index 653e885..e96f6da 100644
--- a/yapftests/reformatter_buganizer_test.py
+++ b/yapftests/reformatter_buganizer_test.py
@@ -166,7 +166,10 @@ class _():
   def _():
     X = (
         _ares_label_prefix +
-        'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')  # pylint: disable=line-too-long
+        'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'  # pylint: disable=line-too-long
+        'PyTypePyTypePyTypePyTypePyTypePyTypePyTypePyTypePyTypePyTypePyTypePyTypePyType'  # pytype: disable=attribute-error
+        'CopybaraCopybaraCopybaraCopybaraCopybaraCopybaraCopybaraCopybaraCopybara'  # copybara:strip
+    )
 """
     uwlines = yapf_test_helper.ParseAndUnwrap(code)
     self.assertCodeEqual(code, reformatter.Reformat(uwlines))
diff --git a/yapftests/yapf_test.py b/yapftests/yapf_test.py
index 23c4fca..e3e3df3 100644
--- a/yapftests/yapf_test.py
+++ b/yapftests/yapf_test.py
@@ -1016,6 +1016,29 @@ x = {
         expected_formatted_code,
         extra_options=['--lines', '4-7'])
 
+  def testRetainVerticalFormattingBetweenDisabledLines(self):
+    unformatted_code = textwrap.dedent("""\
+        class A(object):
+            def aaaaaaaaaaaaa(self):
+                pass
+
+
+            def bbbbbbbbbbbbb(self):  # 5
+                pass
+        """)
+    expected_formatted_code = textwrap.dedent("""\
+        class A(object):
+            def aaaaaaaaaaaaa(self):
+                pass
+
+            def bbbbbbbbbbbbb(self):  # 5
+                pass
+        """)
+    self.assertYapfReformats(
+        unformatted_code,
+        expected_formatted_code,
+        extra_options=['--lines', '4-4'])
+
   def testFormatLinesSpecifiedInMiddleOfExpression(self):
     unformatted_code = textwrap.dedent("""\
         class A(object):