Repository: astral-sh/ruff
Pull request #17220: [`pyupgrade`] Preserve parenthesis when fixing native literals containing newlines (`UP018`)
Author: login: maxmynter
Labels: name: bug
description: Something isn't working
name: fixes
description: Related to suggested fixes for violations
Created: 2025-04-05T14:27:39Z | Merged: 2025-04-24T06:48:02Z

<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

Closes #17124 

## Summary
Add a check whether a native literal contains a newline (e.g. `int(+\n1)`) in which case we must preserve the parenthesis to not break syntax.

Note: An alternative could have been stripping the newline, but I think if it's there it's there for a reason; if not other formatting rules can take care of it.


## Test Plan
Extended the existing snapshot test by the affected case.

Linked issue #17124: UP018 fix introduces a syntax error when the argument contains a newline
### Summary

The fix for [`native-literals` (UP018)](https://docs.astral.sh/ruff/rules/native-literals/) can introduce a syntax error when the argument contains a newline between a unary `+` or `-` and a number literal. It should wrap the expression with parentheses as needed to avoid the error.
```console
$ cat >up018.py <<'# EOF'
int(-
1)
# EOF

$ ruff --isolated check up018.py --select UP018 --diff 2>&1 | grep error:
error: Fix introduced a syntax error. Reverting all changes.
```

### Version

ruff 0.11.2 (4773878ee 2025-03-21)

Diff:
diff --git a/.gitattributes b/.gitattributes
index ebc35c67454587..966aa80ecf018b 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -12,6 +12,12 @@ crates/ruff_python_parser/resources/invalid/re_lexing/line_continuation_windows_
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_windows_eol.py text eol=crlf
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_mac_eol.py text eol=cr
 
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py text eol=cr
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_LF.py text eol=lf
+
+crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_CR.py text eol=cr
+crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_LF.py text eol=lf
+
 crates/ruff_python_parser/resources/inline linguist-generated=true
 
 ruff.schema.json -diff linguist-generated=true text=auto eol=lf
diff --git a/crates/ruff_linter/resources/test/fixtures/pyupgrade/.editorconfig b/crates/ruff_linter/resources/test/fixtures/pyupgrade/.editorconfig
new file mode 100644
index 00000000000000..048d4deec14e7b
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/pyupgrade/.editorconfig
@@ -0,0 +1,5 @@
+# These rules test for reformatting with specific line endings.
+# Using a formatter fixes that and breaks the tests
+[UP018_{CR,LF}.py]
+generated_code = true
+ij_formatter_enabled = false
diff --git a/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_CR.py b/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_CR.py
new file mode 100644
index 00000000000000..7535093fa14876
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_CR.py
@@ -0,0 +1 @@
+# Keep parenthesis around preserved CRint(-    1)int(+    1)
diff --git a/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_LF.py b/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_LF.py
new file mode 100644
index 00000000000000..3a2d43335e505a
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/pyupgrade/UP018_LF.py
@@ -0,0 +1,7 @@
+# Keep parentheses around preserved \n 
+
+int(-
+    1)
+
+int(+
+    1)
diff --git a/crates/ruff_linter/resources/test/fixtures/ruff/.editorconfig b/crates/ruff_linter/resources/test/fixtures/ruff/.editorconfig
new file mode 100644
index 00000000000000..bf4811bd1a7a2a
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/ruff/.editorconfig
@@ -0,0 +1,5 @@
+# These rules test for reformatting with specific line endings.
+# Using a formatter fixes that and breaks the tests
+[RUF046_{CR,LF}.py]
+generated_code = true
+ij_formatter_enabled = false
diff --git a/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py b/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py
new file mode 100644
index 00000000000000..9723c347dcf0c9
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py
@@ -0,0 +1 @@
+int(-    1) # Carriage return as newline
\ No newline at end of file
diff --git a/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_LF.py b/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_LF.py
new file mode 100644
index 00000000000000..53a879a56e7148
--- /dev/null
+++ b/crates/ruff_linter/resources/test/fixtures/ruff/RUF046_LF.py
@@ -0,0 +1,3 @@
+# \n as newline
+int(-
+    1)
diff --git a/crates/ruff_linter/src/rules/pyupgrade/mod.rs b/crates/ruff_linter/src/rules/pyupgrade/mod.rs
index 822e680acfceb4..6a7003cabc1ab1 100644
--- a/crates/ruff_linter/src/rules/pyupgrade/mod.rs
+++ b/crates/ruff_linter/src/rules/pyupgrade/mod.rs
@@ -36,6 +36,8 @@ mod tests {
     #[test_case(Rule::LRUCacheWithMaxsizeNone, Path::new("UP033_1.py"))]
     #[test_case(Rule::LRUCacheWithoutParameters, Path::new("UP011.py"))]
     #[test_case(Rule::NativeLiterals, Path::new("UP018.py"))]
+    #[test_case(Rule::NativeLiterals, Path::new("UP018_CR.py"))]
+    #[test_case(Rule::NativeLiterals, Path::new("UP018_LF.py"))]
     #[test_case(Rule::NonPEP585Annotation, Path::new("UP006_0.py"))]
     #[test_case(Rule::NonPEP585Annotation, Path::new("UP006_1.py"))]
     #[test_case(Rule::NonPEP585Annotation, Path::new("UP006_2.py"))]
diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/native_literals.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/native_literals.rs
index a1d8bf31deb379..14e31351ca614c 100644
--- a/crates/ruff_linter/src/rules/pyupgrade/rules/native_literals.rs
+++ b/crates/ruff_linter/src/rules/pyupgrade/rules/native_literals.rs
@@ -4,6 +4,7 @@ use std::str::FromStr;
 use ruff_diagnostics::{AlwaysFixableViolation, Applicability, Diagnostic, Edit, Fix};
 use ruff_macros::{derive_message_formats, ViolationMetadata};
 use ruff_python_ast::{self as ast, Expr, Int, LiteralExpressionRef, OperatorPrecedence, UnaryOp};
+use ruff_source_file::find_newline;
 use ruff_text_size::{Ranged, TextRange};
 
 use crate::checkers::ast::Checker;
@@ -244,6 +245,9 @@ pub(crate) fn native_literals(
             let arg_code = checker.locator().slice(arg);
 
             let content = match (parent_expr, literal_type, has_unary_op) {
+                // Expressions including newlines must be parenthesised to be valid syntax
+                (_, _, true) if find_newline(arg_code).is_some() => format!("({arg_code})"),
+
                 // Attribute access on an integer requires the integer to be parenthesized to disambiguate from a float
                 // Ex) `(7).denominator` is valid but `7.denominator` is not
                 // Note that floats do not have this problem
diff --git a/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_CR.py.snap b/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_CR.py.snap
new file mode 100644
index 00000000000000..6dae74bfe53b66
--- /dev/null
+++ b/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_CR.py.snap
@@ -0,0 +1,22 @@
+---
+source: crates/ruff_linter/src/rules/pyupgrade/mod.rs
+---
+UP018_CR.py:2:1: UP018 [*] Unnecessary `int` call (rewrite as a literal)
+  |
+1 | # Keep parenthesis around preserved CRint(-    1)int(+    1)
+  |                                       ^^^^^^^^^^^ UP018
+  |
+  = help: Replace with integer literal
+
+ℹ Safe fix
+1 1 | # Keep parenthesis around preserved CR2   |-int(-  2 |+(-3 3 |     1)4 4 | int(+5 5 |     1)
+
+UP018_CR.py:4:1: UP018 [*] Unnecessary `int` call (rewrite as a literal)
+  |
+2 | int(-    1)int(+    1)
+  |            ^^^^^^^^^^^ UP018
+  |
+  = help: Replace with integer literal
+
+ℹ Safe fix
+1 1 | # Keep parenthesis around preserved CR2 2 | int(-3 3 |     1)4   |-int(+  4 |+(+5 5 |     1)
diff --git a/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_LF.py.snap b/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_LF.py.snap
new file mode 100644
index 00000000000000..d3a530d273c713
--- /dev/null
+++ b/crates/ruff_linter/src/rules/pyupgrade/snapshots/ruff_linter__rules__pyupgrade__tests__UP018_LF.py.snap
@@ -0,0 +1,41 @@
+---
+source: crates/ruff_linter/src/rules/pyupgrade/mod.rs
+---
+UP018_LF.py:3:1: UP018 [*] Unnecessary `int` call (rewrite as a literal)
+  |
+1 |   # Keep parentheses around preserved \n 
+2 |
+3 | / int(-
+4 | |     1)
+  | |______^ UP018
+5 |
+6 |   int(+
+  |
+  = help: Replace with integer literal
+
+ℹ Safe fix
+1 1 | # Keep parentheses around preserved \n 
+2 2 | 
+3   |-int(-
+  3 |+(-
+4 4 |     1)
+5 5 | 
+6 6 | int(+
+
+UP018_LF.py:6:1: UP018 [*] Unnecessary `int` call (rewrite as a literal)
+  |
+4 |       1)
+5 |
+6 | / int(+
+7 | |     1)
+  | |______^ UP018
+  |
+  = help: Replace with integer literal
+
+ℹ Safe fix
+3 3 | int(-
+4 4 |     1)
+5 5 | 
+6   |-int(+
+  6 |+(+
+7 7 |     1)
diff --git a/crates/ruff_linter/src/rules/ruff/mod.rs b/crates/ruff_linter/src/rules/ruff/mod.rs
index 75c59307957f84..b5006db9997195 100644
--- a/crates/ruff_linter/src/rules/ruff/mod.rs
+++ b/crates/ruff_linter/src/rules/ruff/mod.rs
@@ -84,6 +84,8 @@ mod tests {
     #[test_case(Rule::UnnecessaryNestedLiteral, Path::new("RUF041.py"))]
     #[test_case(Rule::UnnecessaryNestedLiteral, Path::new("RUF041.pyi"))]
     #[test_case(Rule::UnnecessaryCastToInt, Path::new("RUF046.py"))]
+    #[test_case(Rule::UnnecessaryCastToInt, Path::new("RUF046_CR.py"))]
+    #[test_case(Rule::UnnecessaryCastToInt, Path::new("RUF046_LF.py"))]
     #[test_case(Rule::NeedlessElse, Path::new("RUF047_if.py"))]
     #[test_case(Rule::NeedlessElse, Path::new("RUF047_for.py"))]
     #[test_case(Rule::NeedlessElse, Path::new("RUF047_while.py"))]
diff --git a/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap
new file mode 100644
index 00000000000000..61b9115357d0a2
--- /dev/null
+++ b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap
@@ -0,0 +1,12 @@
+---
+source: crates/ruff_linter/src/rules/ruff/mod.rs
+---
+RUF046_CR.py:1:1: RUF046 [*] Value being cast to `int` is already an integer
+  |
+1 | int(-    1) # Carriage return as newline
+  | ^^^^^^^^^^^ RUF046
+  |
+  = help: Remove unnecessary `int` call
+
+ℹ Safe fix
+1   |-int(-  1 |+(-2 2 |     1) # Carriage return as newline
diff --git a/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_LF.py.snap b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_LF.py.snap
new file mode 100644
index 00000000000000..97d44c191d4df6
--- /dev/null
+++ b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_LF.py.snap
@@ -0,0 +1,17 @@
+---
+source: crates/ruff_linter/src/rules/ruff/mod.rs
+---
+RUF046_LF.py:2:1: RUF046 [*] Value being cast to `int` is already an integer
+  |
+1 |   # \n as newline
+2 | / int(-
+3 | |     1)
+  | |______^ RUF046
+  |
+  = help: Remove unnecessary `int` call
+
+ℹ Safe fix
+1 1 | # \n as newline
+2   |-int(-
+  2 |+(-
+3 3 |     1)


Reviews:
user: login: dhruvmanila
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T20:05:52Z
user: login: maxmynter
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-11T09:16:48Z
user: login: dhruvmanila
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-21T19:07:13Z
user: login: maxmynter
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-22T16:12:56Z
user: login: MichaReiser
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-23T06:38:27Z
user: login: maxmynter
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-23T22:20:53Z

Review comments:
diff_hunk: @@ -161,6 +161,9 @@ async def f():
 int(round(1,
           0))
 
+int(-
+1) # Carriage return as newline
path: crates/ruff_linter/resources/test/fixtures/ruff/RUF046.py
user: login: dhruvmanila
body: This does not seem to have a carriage return character as newline. It might have been stripped by your either / git. You'd need to create a separate file for this test case and include it in https://github.com/astral-sh/ruff/blob/main/.gitattributes similar to how other files are included there.
created_at: 2025-04-10T20:05:52Z
author_association: MEMBER
diff_hunk: @@ -161,6 +161,9 @@ async def f():
 int(round(1,
           0))
 
+int(-
+1) # Carriage return as newline
path: crates/ruff_linter/resources/test/fixtures/ruff/RUF046.py
user: login: maxmynter
body: Dang. Mus've been git or pre-commit hooks because i was aware of editor autoformatting and checked it before committing.
created_at: 2025-04-11T09:16:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -12,6 +12,16 @@ crates/ruff_python_parser/resources/invalid/re_lexing/line_continuation_windows_
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_windows_eol.py text eol=crlf
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_mac_eol.py text eol=cr
 
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py text eol=cr
+crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap text eol=cr
path: .gitattributes
user: login: dhruvmanila
body: Do we need to include the snapshot files in here? I don't think so considering that other snapshot files aren't included and there are `.editorconfig` files present in both `ruff` and `pyupgrade` snapshot directories.
created_at: 2025-04-21T19:07:13Z
author_association: MEMBER
diff_hunk: @@ -12,6 +12,16 @@ crates/ruff_python_parser/resources/invalid/re_lexing/line_continuation_windows_
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_windows_eol.py text eol=crlf
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_mac_eol.py text eol=cr
 
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py text eol=cr
+crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap text eol=cr
path: .gitattributes
user: login: maxmynter
body: I think we need them. 

I added both mechanisms for specific reasons:
-  `.editorconfig` (added in [67348be](https://github.com/astral-sh/ruff/pull/17220/commits/67348be1ca770d5158181769bf969b1302336f95)): Prevents editors from automatically reformatting the test fixtures when opened
- `.gitattributes`: Ensures Git maintains specific line endings (CR/LF) during checkout/staging irrespective of OS

This way i want to avoid (1) tests from passing incorrectly (if both files get normalized) or (2) breaking unexpectedly (if only one gets normalized).

In [5edfdc8](https://github.com/astral-sh/ruff/pull/17220/commits/5edfdc8f8c6b61d376a687a08e8d9675f3001803) I wanted to add tests for CR and LF lineendings but local normalization made tests passing as both were normalized thus not testing for CRs.

The other test cases don't seem to be sensitive to the specific line endings and should be fine as long as both fixture and snapshot are regularized on checkout by git. But fixing only the fixture, not the snapshot could make tests flaky.

Or am I missing something?
created_at: 2025-04-22T16:12:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -12,6 +12,16 @@ crates/ruff_python_parser/resources/invalid/re_lexing/line_continuation_windows_
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_windows_eol.py text eol=crlf
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_mac_eol.py text eol=cr
 
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py text eol=cr
+crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap text eol=cr
path: .gitattributes
user: login: MichaReiser
body: Looking at the crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap snapshot, it does seem to render the `\r` differently (`^M`). That's why I think adding the snapshot here isn't needed.
created_at: 2025-04-23T06:38:27Z
author_association: MEMBER
diff_hunk: @@ -12,6 +12,16 @@ crates/ruff_python_parser/resources/invalid/re_lexing/line_continuation_windows_
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_windows_eol.py text eol=crlf
 crates/ruff_python_parser/resources/invalid/re_lex_logical_token_mac_eol.py text eol=cr
 
+crates/ruff_linter/resources/test/fixtures/ruff/RUF046_CR.py text eol=cr
+crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF046_RUF046_CR.py.snap text eol=cr
path: .gitattributes
user: login: maxmynter
body: Allright, addressed in [005a528](https://github.com/astral-sh/ruff/pull/17220/commits/005a5288cf8426765d8249eac0c22f5435eb38f9).
created_at: 2025-04-23T22:20:53Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-04-05T14:35:03Z
body: <!-- generated-comment ecosystem -->
## `ruff-ecosystem` results
### Linter (stable)
✅ ecosystem check detected no linter changes.

### Linter (preview)
✅ ecosystem check detected no linter changes.



author_association: CONTRIBUTOR
user: login: dscorbett
created_at: 2025-04-07T12:22:57Z
body: Does `arg_code.contains('\n')` work when the newline is a carriage return? For example:
```console
$ printf 'int(-\r1)\r' | ruff --isolated check - --select UP018 --diff 2>&1 | grep error:
error: Fix introduced a syntax error. Reverting all changes.
```
If not, see #15230 and #15703 for ways to detect all newlines.
author_association: NONE
user: login: maxmynter
created_at: 2025-04-11T10:23:25Z
body: I'm unsure how the testfiles will behave when running on different machines and I couldn't test as I have only access to windows machines. So let me know if my approach doesn't work.

For the rules `RUF046` and `UP018` i've added 2 test fixtures each; one with `cr` the other with `lr` newlines. All are persisted in `.gitattributes`. 

Alternatively, we could merge those and use `test eol=keep` but that runs at the risk of devs running into formatting errors when they edit the fixtures. 
author_association: CONTRIBUTOR
user: login: MichaReiser
created_at: 2025-04-11T11:54:39Z
body: > that runs at the risk of devs running into formatting errors when they edit the fixtures.

That's a problem that exists anyway. Not when commiting, but it means they may accidentally change the line ending locally which might be confusing when debugging a test. You can create an `.editorconfig` file to disable formatting in editors that support and respect `.editorconfig`

https://github.com/astral-sh/ruff/blob/46ab9dec181f7736a0d32be4bad59d00228c1819/crates/ruff_linter/resources/test/fixtures/pycodestyle/.editorconfig#L9-L8
author_association: MEMBER
user: login: maxmynter
created_at: 2025-04-12T11:28:05Z
body: I added the `.editorconfig`s in [67348be](https://github.com/astral-sh/ruff/pull/17220/commits/67348be1ca770d5158181769bf969b1302336f95).

----

I think it makes sense to keep the separate files for the `cr` and `lf` cases. That way we enforce the desired line endings via `.gitattributes`, revert any unwanted formatting changes and the tests fail if functionality breaks.

With `eol=keep` we would keep accidental changes to the file. If these changes slip in, they may easily happen with the snapshot, too. 

I've had some issues getting the linebreak's to render in different editors.

What do you think? 
author_association: CONTRIBUTOR