Repository: microsoft/vscode (TypeScript)
Date: 2021-05-07 14:22:47+00:00
Task type: Bug Fix

Issue:
VSCode fails to recognize URLs containing the characters 【】
<!-- ⚠️⚠️ Do Not Delete This! bug_report_template ⚠️⚠️ -->
<!-- Please read our Rules of Conduct: https://opensource.microsoft.com/codeofconduct/ -->
<!-- 🕮 Read our guide about submitting issues: https://github.com/microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions -->
<!-- 🔎 Search existing issues to avoid creating duplicates. -->
<!-- 🧪 Test using the latest Insiders build to see if your issue has already been fixed: https://code.visualstudio.com/insiders/ -->
<!-- 💡 Instead of creating your report here, use 'Report Issue' from the 'Help' menu in VS Code to pre-fill useful information. -->
- VS Code Version: 1.55.2
- OS Version: Windows 10 18363.1440

Steps to Reproduce:

1. enter a URL containing 【 or 】, for example: https://zh.wikipedia.org/wiki/【我推的孩子】

VSCode usually can recognize URLs with non-alphanumeric characters, for example, the URL https://zh.wikipedia.org/wiki/新加坡 can be properly recognized and you can ctrl+click to open it. 

However, URLs containing 【 or 】, like the above example I gave, cannot be properly recognized in VSCode.

I assume this is a bug or an oversight rather than a design decision, as I can't see any reason to exclude 【 or 】 in URLs. While I write this issue I notice that Github actually recognizes such URLs with no problem, so I guess it's not outrageous to ask VSCode to support them too.

<!-- 🔧 Launch with `code --disable-extensions` to check. -->
Does this issue occur when all extensions are disabled?: Yes

<!-- 🪓 If you answered No above, use 'Help: Start Extension Bisect' from Command Palette to try to identify the cause. -->
<!-- 📣 Issues caused by an extension need to be reported directly to the extension publisher. The 'Help > Report Issue' dialog can assist with this. -->


VSCode fails to recognize URLs containing the characters 【】
<!-- ⚠️⚠️ Do Not Delete This! bug_report_template ⚠️⚠️ -->
<!-- Please read our Rules of Conduct: https://opensource.microsoft.com/codeofconduct/ -->
<!-- 🕮 Read our guide about submitting issues: https://github.com/microsoft/vscode/wiki/Submitting-Bugs-and-Suggestions -->
<!-- 🔎 Search existing issues to avoid creating duplicates. -->
<!-- 🧪 Test using the latest Insiders build to see if your issue has already been fixed: https://code.visualstudio.com/insiders/ -->
<!-- 💡 Instead of creating your report here, use 'Report Issue' from the 'Help' menu in VS Code to pre-fill useful information. -->
- VS Code Version: 1.55.2
- OS Version: Windows 10 18363.1440

Steps to Reproduce:

1. enter a URL containing 【 or 】, for example: https://zh.wikipedia.org/wiki/【我推的孩子】

VSCode usually can recognize URLs with non-alphanumeric characters, for example, the URL https://zh.wikipedia.org/wiki/新加坡 can be properly recognized and you can ctrl+click to open it. 

However, URLs containing 【 or 】, like the above example I gave, cannot be properly recognized in VSCode.

I assume this is a bug or an oversight rather than a design decision, as I can't see any reason to exclude 【 or 】 in URLs. While I write this issue I notice that Github actually recognizes such URLs with no problem, so I guess it's not outrageous to ask VSCode to support them too.

<!-- 🔧 Launch with `code --disable-extensions` to check. -->
Does this issue occur when all extensions are disabled?: Yes

<!-- 🪓 If you answered No above, use 'Help: Start Extension Bisect' from Command Palette to try to identify the cause. -->
<!-- 📣 Issues caused by an extension need to be reported directly to the extension publisher. The 'Help > Report Issue' dialog can assist with this. -->




Issue discussion:
@alexdima in [this commit](https://github.com/microsoft/vscode/commit/7018ffce0bf2873bfd71ead5612b974b475bcffc#diff-47ef45cdeef86c6a54aa67b5c25a970fea5fa4e134160e377aefe0e589906ca9R51) "【" was explicitly marked as url termination character. I don't know why though. Shall I just remove it?
@hediet Thanks for pointing out the place! This list seems to contain a lot of punctuation marks and many of them may be a little questionable. For instance `‘“〈《「『【〔（［｛｢｣｝］）〕】』」》〉”’` are all just variations of parentheses or quotation marks. They are often used as part of an URL rather than to terminate an URL. But since there are so many of them I don't know what is the best way to handle it. Some people may have been relying on this behavior and removing some characters could break it. Maybe you can make it customizable but then that would add to your workload.

I think it's fine to change things and remove "【" / "】" as force termination characters. Especially if there are things like wikipedia links with them. Actually, the best change would be to do some logic similar as we do for "(" / ")", where we actually tweak behavior for ")" based on the presence of "(" inside the link or not.
@alexdima in [this commit](https://github.com/microsoft/vscode/commit/7018ffce0bf2873bfd71ead5612b974b475bcffc#diff-47ef45cdeef86c6a54aa67b5c25a970fea5fa4e134160e377aefe0e589906ca9R51) "【" was explicitly marked as url termination character. I don't know why though. Shall I just remove it?
@hediet Thanks for pointing out the place! This list seems to contain a lot of punctuation marks and many of them may be a little questionable. For instance `‘“〈《「『【〔（［｛｢｣｝］）〕】』」》〉”’` are all just variations of parentheses or quotation marks. They are often used as part of an URL rather than to terminate an URL. But since there are so many of them I don't know what is the best way to handle it. Some people may have been relying on this behavior and removing some characters could break it. Maybe you can make it customizable but then that would add to your workload.

I think it's fine to change things and remove "【" / "】" as force termination characters. Especially if there are things like wikipedia links with them. Actually, the best change would be to do some logic similar as we do for "(" / ")", where we actually tweak behavior for ")" based on the presence of "(" inside the link or not.

Solution patch:
diff --git a/src/vs/editor/common/modes/linkComputer.ts b/src/vs/editor/common/modes/linkComputer.ts
--- a/src/vs/editor/common/modes/linkComputer.ts
+++ b/src/vs/editor/common/modes/linkComputer.ts
@@ -154,7 +154,7 @@ function getClassifier(): CharacterClassifier<CharacterClass> {
 	if (_classifier === null) {
 		_classifier = new CharacterClassifier<CharacterClass>(CharacterClass.None);
 
-		const FORCE_TERMINATION_CHARACTERS = ' \t<>\'\"、。｡､，．：；‘“〈《「『【〔（［｛｢｣｝］）〕】』」》〉”’｀～…';
+		const FORCE_TERMINATION_CHARACTERS = ' \t<>\'\"、。｡､，．：；‘〈「『〔（［｛｢｣｝］）〕』」〉’｀～…';
 		for (let i = 0; i < FORCE_TERMINATION_CHARACTERS.length; i++) {
 			_classifier.set(FORCE_TERMINATION_CHARACTERS.charCodeAt(i), CharacterClass.ForceTermination);
 		}


Test patch:
diff --git a/src/vs/editor/test/common/modes/linkComputer.test.ts b/src/vs/editor/test/common/modes/linkComputer.test.ts
--- a/src/vs/editor/test/common/modes/linkComputer.test.ts
+++ b/src/vs/editor/test/common/modes/linkComputer.test.ts
@@ -230,4 +230,25 @@ suite('Editor Modes - Link Computer', () => {
 			'    http://tree-mark.chips.jp/レーズン＆ベリーミックス    '
 		);
 	});
+
+	test('issue #121438: Link detection stops at【...】', () => {
+		assertLink(
+			'aa  https://zh.wikipedia.org/wiki/【我推的孩子】 aa',
+			'    https://zh.wikipedia.org/wiki/【我推的孩子】   '
+		);
+	});
+
+	test('issue #121438: Link detection stops at《...》', () => {
+		assertLink(
+			'aa  https://zh.wikipedia.org/wiki/《新青年》编辑部旧址 aa',
+			'    https://zh.wikipedia.org/wiki/《新青年》编辑部旧址   '
+		);
+	});
+
+	test('issue #121438: Link detection stops at “...”', () => {
+		assertLink(
+			'aa  https://zh.wikipedia.org/wiki/“常凯申”误译事件 aa',
+			'    https://zh.wikipedia.org/wiki/“常凯申”误译事件   '
+		);
+	});
 });