Package: whitespace-format
whitespace-formatLinter and formatter for source code files and text files.The purpose of this tool is to normalize source code files (e.g. Python, Java,
C/C++, Ruby, Go, JavaScript, etc.) and text files (HTML, JSON, YAML, CSV,
MarkDown, LaTeX) before checking them into a version control system.The features include:Auto-detection of new line markers (Linux\n, Windows\r\n, Mac\r).Add a new line marker at the end of the file if it is missing.Make new line markers consistent.Remove empty lines at the end of the file.Remove whitespace at the end of each line.Replace tabs with spaces.Remove/replace non-standard whitespace characters.The formatting changes areidempotent, i.e., running the tool
second time (with the same parameters) has no effect.Installationpipinstallwhitespace-formatInstallation requires Python 3.7.5 or higher.UsageA sample command that formats source code files:whitespace-format\--exclude".git/|.idea/|.pyc$"\--new-line-markerlinux\--normalize-new-line-markers\foo.txtmy_project/The command above formatsfoo.txtand all files containedmy_project/and
its subdirectories. Files that contain.git/or.idea/in their (relative)
path are excluded. For example, files inmy_project/.git/and files inmy_project/.idea/are excluded. Likewise, files ending with*.pycare
excluded.If you want only know if any changeswould bemade, add--check-onlyoption:whitespace-format\--exclude".git/|.idea/|.pyc$"\--check-only\--new-line-markerlinux\--normalize-new-line-markers\foo.txtmy_project/This command can be used as a validation step before checking the source files
into a version control system. The command outputs non-zero exit code if any
of the files would be formatted.Options--check-only-- Do not format files. Only report which files would be formatted.--follow-symlinks-- Follow symbolic links when searching for files.--exclude=REGEX-- Regular expression that specifies which files to exclude.
The regular expression is evaluated on the path of each file.--verbose-- Print more messages than normally.--quiet-- Do not print any messages, except for errors when reading or writing files.Formatting options--add-new-line-marker-at-end-of-file-- Add missing new line marker at end of each file.--remove-new-line-marker-from-end-of-file-- Remove all new line marker(s) from the end of each file.
This option is ignored when--add-new-line-marker-at-end-of-fileis used.
Empty lines at the end of the file are removed.--normalize-new-line-markers-- Make new line markers consistent in each file
by replacing\\r\\n,\\n, and\rwith a consistent new line marker.--remove-trailing-whitespace-- Remove whitespace at the end of each line.--remove-trailing-empty-lines-- Remove empty lines at the end of each file.--new-line-marker=MARKER-- This option specifies what new line marker to use.MARKERmust be one of the following:auto-- Use new line marker that is the most common in each individual file.
If no new line marker is present in the file, Linux\nis used.
This is the default option.linux-- Use Linux new line marker\\n.mac-- Use Mac new line marker\\r.windows-- Use Windows new line marker\\r\\n.--encoding-- Text encoding for both reading and writing files. Default encoding isutf-8.
List of supported encodings can be found athttps://docs.python.org/3/library/codecs.html#standard-encodingsNote that input files can contain an arbitrary mix of new line markers\n,\r,\r\neven within the same file. The option--new-line-markerspecifies the character that should be in the formatted file.An opinionated combination of options is:whitespace-format\--new-line-marker=linux\--add-new-line-marker-at-end-of-file\--normalize-new-line-markers\--remove-trailing-whitespace\--remove-trailing-empty-lines\foo.txtmy_project/This should work well for common programming languages (e.g. Python, Java,
C/C++, JavaScript) and common text file formats (e.g. CSV, LaTeX, JSON, YAML,
HTML, MarkDown).Empty filesThere are separate options for handling empty files and files consisting of
whitespace characters only:--normalize-empty-files=MODE--normalize-whitespace-only-files=MODEwhereMODEis one of the following:ignore-- Leave the file as is. This is the default option.empty-- Replace the file with an empty file.one-line-- Replace each file with a file consisting of a single new line marker.Depending on the mode, an empty file or a whitespace-only file will be either
ignored, replaced by a zero-byte file, or replaced by a file consisting of
single end of line marker.If--normalize-whitespace-only-filesis set toempty,--normalize-empty-files settingset toemptyas well. In other words,
combination--normalize-whitespace-only-files=emptyand--normalize-empty-files=one-lineis not allowed, since it would lead to
behavior that is not idempotent.Special characters--replace-tabs-with-spaces=N-- Replace tabs with spaces.
Where isNis the number of spaces. IfNis negative, tabs are not replaced.
Default value is-1.--normalize-non-standard-whitespace=MODE-- Replace or remove
non-standard whitespace characters (\vand\f).MODEmust be one of the following:ignore-- Leave\vandfas is. This is the default option.replace-- Replace any occurrence of\vor\fwith a single space.remove-- Remove all occurrences of\vand\fLicenseMITMacOS development setupMake sure you havebrewpackage manager installed.Installpyenv,pyenv-virtualenvandpoetry:brewinstallpyenv
brewinstallpyenv-virtualenv
brewinstallpoetryCreate Python virtual environment with the correct Python version:makeinstall-python
makecreate-environmentAdd the following lines to.zshrcor.bash_profileand restart the terminal:# Pyenv settingsexportPYENV_ROOT="$HOME/.pyenv"exportPATH="$PYENV_ROOT/bin:$PATH"eval"$(pyenvinit--path)"eval"$(pyenvvirtualenv-init-)"Install all dependenciesmakeinstall-dependeciesIf you need to delete the Python virtual environment, you can do so with the
commandmake delete-environment.Running unit tests and code checksIf you make code change, run unit tests and code checks with the command:makecleanwhitespace-format-checkisort-checkblack-checkflake8pydocstylepylintmypytestcoverageEach make target runs different checks:cleandeletes temporary fileswhitespace-format-checkrunswhitespace-formatchecker on all filesisort-checkrunsisortchecker of imports in*.pyfilesblack-checkrunsblackcode format checker on*.pyfilesflake8runsflake8code style checker on*.pyfilespydocstylerunspydocstyledocstring checker on*.pyfilespylintrunspylintcode checker on*.pyfilesmypyrunsmypytype checker on*.pyfilestestruns unit testscoveragegenerates code coverage reportYou can automatically format code with the command:makeisort-formatblack-formatwhitespace-formatModifying dependenciesThe list of Python packages that this project depends on is specified inpyproject.tomland inpoetry.lockfiles. The filepyproject.tomlcan be
edited by humans. The filepoetry.lockis automatically generated bypoetry.Install a development dependency with the command:poetryadd--dev<some_new_python_tool>Install a new production dependency with the command:poetryadd<some_python_library>Manual modification ofpyproject.tomlInstead of usingpoetry addcommand, you can editpyproject.tomlfile. Then,
regeneratepoetry.lockfile with the command:poetrylockor the command:poetrylock--no-updateThe latter command does not update already locked packages.Fixing broken Python environmentIf your Python virtual environment becomes broken or polluted with unnecessary
packages, delete it, recreate it from scratch and install dependencies a fresh
with the following commands:makedelete-environment
makecreate-environment
makeinstall-dependencies