Repository: psf/black

Issue:
String processing doesn't respect existing code formatting for f-strings
**Describe the bug**

There is some issue with how Black splits the string if at least one of them is an f-string.

**To Reproduce**

For example:
1. Take this file:
```py
message = (
    f"1. Go to Google Developers Console and log in with your Google account."
    "(https://console.developers.google.com/)"
    "2. You should be prompted to create a new project (name does not matter)."
    "3. Click on Enable APIs and Services at the top."
    "4. In the list of APIs choose or search for YouTube Data API v3 and "
    "click on it. Choose Enable."
    "5. Click on Credentials on the left navigation bar."
    "6. Click on Create Credential at the top."
    '7. At the top click the link for "API key".'
    "8. No application restrictions are needed. Click Create at the bottom."
    "9. You now have a key to add to `{prefix}set api youtube api_key`"
)
```
2. Run _Black_ on it with `--experimental-string-processing` flag
3. See rather bad formatting even though the original respected line length:
```py
message = (
    f"1. Go to Google Developers Console and log in with your Google account."
    f"(https://console.developers.google.com/"
    f")2. You should be prompted to create a new project (name does not matter"
    f").3. Click on Enable APIs and Services at the t"
    f"op.4. In the list of APIs choose or search for YouTube Data API v3 "
    f"and click on it. Choose En"
    f"able.5. Click on Credentials on the left navigatio"
    f"n bar.6. Click on Create Credential at t"
    f'he top.7. At the top click the link for "A'
    f'PI key".8. No application restrictions are needed. Click Create at th'
    f"e bottom.9. You now have a key to add to `{{prefix}}set api youtube api_key`"
)
```
I'm also not sure why it splits in the middle of the word?

**Expected behavior**

No 

**Environment (please complete the following information):**

- Version: main (cf75673 at the time of testing)
- OS and Python version: Windows 10/Python 3.8.9

**Does this bug also happen on main?**

Yes

**Additional context**

N/A



Discussion:
I'm only seeing this with  --experimental-string-processing
Aaaaand, of course, I forgot to mention that in the description, sorry. I updated the issue description appropriately.
I'm looking into this. Note that this only occurs when you prefix one or more (but not all) of the substrings with a redundant `f` (i.e. one of the substrings is an f-string containing no f-expression) and none of the other substrings are valid f-strings (i.e. f-strings that actually contain f-expressions).

Resolving patch:
diff --git a/src/black/trans.py b/src/black/trans.py
--- a/src/black/trans.py
+++ b/src/black/trans.py
@@ -1151,21 +1151,32 @@ def more_splits_should_be_made() -> bool:
 
             # --- Construct `next_value`
             next_value = rest_value[:break_idx] + QUOTE
+
+            # HACK: The following 'if' statement is a hack to fix the custom
+            # breakpoint index in the case of either: (a) substrings that were
+            # f-strings but will have the 'f' prefix removed OR (b) substrings
+            # that were not f-strings but will now become f-strings because of
+            # redundant use of the 'f' prefix (i.e. none of the substrings
+            # contain f-expressions but one or more of them had the 'f' prefix
+            # anyway; in which case, we will prepend 'f' to _all_ substrings).
+            #
+            # There is probably a better way to accomplish what is being done
+            # here...
+            #
+            # If this substring is an f-string, we _could_ remove the 'f'
+            # prefix, and the current custom split did NOT originally use a
+            # prefix...
             if (
-                # Are we allowed to try to drop a pointless 'f' prefix?
-                drop_pointless_f_prefix
-                # If we are, will we be successful?
-                and next_value != self._normalize_f_string(next_value, prefix)
+                next_value != self._normalize_f_string(next_value, prefix)
+                and use_custom_breakpoints
+                and not csplit.has_prefix
             ):
-                # If the current custom split did NOT originally use a prefix,
-                # then `csplit.break_idx` will be off by one after removing
+                # Then `csplit.break_idx` will be off by one after removing
                 # the 'f' prefix.
-                break_idx = (
-                    break_idx + 1
-                    if use_custom_breakpoints and not csplit.has_prefix
-                    else break_idx
-                )
+                break_idx += 1
                 next_value = rest_value[:break_idx] + QUOTE
+
+            if drop_pointless_f_prefix:
                 next_value = self._normalize_f_string(next_value, prefix)
 
             # --- Construct `next_leaf`


Test patch:
diff --git a/tests/data/long_strings__regression.py b/tests/data/long_strings__regression.py
--- a/tests/data/long_strings__regression.py
+++ b/tests/data/long_strings__regression.py
@@ -445,6 +445,45 @@ def _legacy_listen_examples():
 )
 assert str(suffix_arr) in "['$', 'angaroo$', 'angrykangaroo$', 'aroo$', 'garoo$', 'grykangaroo$', 'kangaroo$', 'ngaroo$', 'ngrykangaroo$', 'o$', 'oo$', 'roo$', 'rykangaroo$', 'ykangaroo$']"
 assert str(suffix_arr) not in "['$', 'angaroo$', 'angrykangaroo$', 'aroo$', 'garoo$', 'grykangaroo$', 'kangaroo$', 'ngaroo$', 'ngrykangaroo$', 'o$', 'oo$', 'roo$', 'rykangaroo$', 'ykangaroo$']"
+message = (
+    f"1. Go to Google Developers Console and log in with your Google account."
+    "(https://console.developers.google.com/)"
+    "2. You should be prompted to create a new project (name does not matter)."
+    "3. Click on Enable APIs and Services at the top."
+    "4. In the list of APIs choose or search for YouTube Data API v3 and "
+    "click on it. Choose Enable."
+    "5. Click on Credentials on the left navigation bar."
+    "6. Click on Create Credential at the top."
+    '7. At the top click the link for "API key".'
+    "8. No application restrictions are needed. Click Create at the bottom."
+    "9. You now have a key to add to `{prefix}set api youtube api_key`"
+)
+message = (
+    f"1. Go to Google Developers Console and log in with your Google account."
+    "(https://console.developers.google.com/)"
+    "2. You should be prompted to create a new project (name does not matter)."
+    f"3. Click on Enable APIs and Services at the top."
+    "4. In the list of APIs choose or search for YouTube Data API v3 and "
+    "click on it. Choose Enable."
+    f"5. Click on Credentials on the left navigation bar."
+    "6. Click on Create Credential at the top."
+    '7. At the top click the link for "API key".'
+    "8. No application restrictions are needed. Click Create at the bottom."
+    "9. You now have a key to add to `{prefix}set api youtube api_key`"
+)
+message = (
+    f"1. Go to Google Developers Console and log in with your Google account."
+    "(https://console.developers.google.com/)"
+    "2. You should be prompted to create a new project (name does not matter)."
+    f"3. Click on Enable APIs and Services at the top."
+    "4. In the list of APIs choose or search for YouTube Data API v3 and "
+    "click on it. Choose Enable."
+    f"5. Click on Credentials on the left navigation bar."
+    "6. Click on Create Credential at the top."
+    '7. At the top click the link for "API key".'
+    "8. No application restrictions are needed. Click Create at the bottom."
+    f"9. You now have a key to add to `{prefix}set api youtube api_key`"
+)
 
 # output
 
@@ -1002,3 +1041,42 @@ def _legacy_listen_examples():
     " 'kangaroo$', 'ngaroo$', 'ngrykangaroo$', 'o$', 'oo$', 'roo$',"
     " 'rykangaroo$', 'ykangaroo$']"
 )
+message = (
+    f"1. Go to Google Developers Console and log in with your Google account."
+    f"(https://console.developers.google.com/)"
+    f"2. You should be prompted to create a new project (name does not matter)."
+    f"3. Click on Enable APIs and Services at the top."
+    f"4. In the list of APIs choose or search for YouTube Data API v3 and "
+    f"click on it. Choose Enable."
+    f"5. Click on Credentials on the left navigation bar."
+    f"6. Click on Create Credential at the top."
+    f'7. At the top click the link for "API key".'
+    f"8. No application restrictions are needed. Click Create at the bottom."
+    f"9. You now have a key to add to `{{prefix}}set api youtube api_key`"
+)
+message = (
+    f"1. Go to Google Developers Console and log in with your Google account."
+    f"(https://console.developers.google.com/)"
+    f"2. You should be prompted to create a new project (name does not matter)."
+    f"3. Click on Enable APIs and Services at the top."
+    f"4. In the list of APIs choose or search for YouTube Data API v3 and "
+    f"click on it. Choose Enable."
+    f"5. Click on Credentials on the left navigation bar."
+    f"6. Click on Create Credential at the top."
+    f'7. At the top click the link for "API key".'
+    f"8. No application restrictions are needed. Click Create at the bottom."
+    f"9. You now have a key to add to `{{prefix}}set api youtube api_key`"
+)
+message = (
+    "1. Go to Google Developers Console and log in with your Google account."
+    "(https://console.developers.google.com/)"
+    "2. You should be prompted to create a new project (name does not matter)."
+    "3. Click on Enable APIs and Services at the top."
+    "4. In the list of APIs choose or search for YouTube Data API v3 and "
+    "click on it. Choose Enable."
+    "5. Click on Credentials on the left navigation bar."
+    "6. Click on Create Credential at the top."
+    '7. At the top click the link for "API key".'
+    "8. No application restrictions are needed. Click Create at the bottom."
+    f"9. You now have a key to add to `{prefix}set api youtube api_key`"
+)