Repository: RocketChat/Rocket.Chat.ReactNative (language: typescript)

Issue:
Real Name search doesn't work in Add Users form
- Your Rocket.Chat Experimental app version: 4.7.0.17188
- Your Rocket.Chat server version: 3.1.1
- Device (or Simulator) you're running with: Xiaomi Mi 9 SE

Search for users works perfectly in Directory Search. It does search by username and which is really much important for me and my users it does search by Real Name (which in my deployment almost always Cyrillic and not English names)
Here is example of Cyrillic search in mobile application on open.rocket.chat 
![Screenshot_2020-05-30-20-49-56-608_chat rocket reactnative](https://user-images.githubusercontent.com/4023037/83330123-5b23a900-a2b7-11ea-9ef2-451156d9574f.jpg)

And here is an example of how Cyrillic search doesn't work in Add Users form (when private chat owner need to add users to his chat)
![Screenshot_2020-05-30-20-50-18-213_chat rocket reactnative](https://user-images.githubusercontent.com/4023037/83330133-65de3e00-a2b7-11ea-8c07-36faddc3af7e.jpg)
So, please, fix that search form and search in mobile application will be consistent and convenient for users.


Discussion hints:
Thanks for opening this @ankar84, we'll try to add this as a fix at our 4.8.0 release, this will probably be launched at end of June.
If someone from community want to fix this, feel free to open a Pull Request. 😀 
is there any news?
The same problem is found in chat search...
@ankar84  @djorkaeffalexandre @diegolmello  Hey I am new here can I work on this?
> @ankar84 @giterium can i work on this?

I'm not a developer, to my deep regret.
@refactor-droidyy sure :)
Has a pull request been made for this issue, and is it closed, or else can I work on it ?
@diegolmello hi! Can you take a look at that issue?
@djorkaeffalexandre told me that it's spotlight related task (idk what it actually is)
> Has a pull request been made for this issue, and is it closed, or else can I work on it ?

Please work on that issue if you can. There is no ready pr for that.
@ankar84 I created the following user:
```
real name: d098
username: diego.mello1409
```

and searched by real name
<img src="https://user-images.githubusercontent.com/804994/93091182-2542de80-f674-11ea-8b6a-86168ea3e227.png" width="350" />

The issue is just the duplicated username?
Or is there something else?
> The issue is just the duplicated username?
> Or is there something else?

No.
Issue is about search users with non English real name (Cyrillic as example) in add users and magnifying glass tool.
Directory search works fine on Cyrillic Real Name (first screen) but when you try to add user to some channel by Cyrillic Real Name - search results are not correct. And magnifying glass too search by Cyrillic Real Name not correct.
Can you create a reproducible example on open server?
> Can you create a reproducible example on open server?

Yes. All screens from open.rocket.chat
Try to find `Антон` in Directory search and with magnifying glass tool.
Directory search
![IMG_20200914_203853](https://user-images.githubusercontent.com/4023037/93093303-bb095880-f6ca-11ea-9856-9d35f495d3e4.jpg)

Magnifying glass tool
![Screenshot_2020-09-14-20-39-04-581_chat rocket reactnative](https://user-images.githubusercontent.com/4023037/93093267-b2188700-f6ca-11ea-9d43-04cc5e87a21c.jpg)

Web version works perfect for both search.

Pull request: [FIX] Local database searches using non-latin characters
<!-- This is a pull request template, you do not need to uncomment or remove the comments, they won't show up in the PR text. -->

## Proposed changes
<!-- Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue below. -->
Queries using LIKE clause with user input aren't safe by default, so we need to sanitize them first.
WatermelonDB default sanitizer works for latin alphabet only.
This PR adds support for other alphabets.

## Issue(s)	
<!-- Link the issues being closed by or related to this PR. For example, you can use #594 if this PR closes issue number 594 -->
Closes #2158 
Related to https://github.com/Nozbe/WatermelonDB/issues/734

## How to test or reproduce
<!-- Mention how you would reproduce the bug if not mentioned on the issue page already. Also mention which screens are going to have the changes if applicable -->

## Screenshots

## Types of changes
<!-- What types of changes does your code introduce to Rocket.Chat? -->
<!-- Put an `x` in the boxes that apply -->

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] Improvement (non-breaking change which improves a current function)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Documentation update (if none of the other choices apply)

## Checklist
<!-- Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code. -->

- [x] I have read the [CONTRIBUTING](https://github.com/RocketChat/Rocket.Chat/blob/develop/.github/CONTRIBUTING.md#contributing-to-rocketchat) doc
- [x] I have signed the [CLA](https://cla-assistant.io/RocketChat/Rocket.Chat.ReactNative)
- [x] Lint and unit tests pass locally with my changes
- [x] I have added tests that prove my fix is effective or that my feature works (if applicable)
- [ ] I have added necessary documentation (if applicable)
- [ ] Any dependent changes have been merged and published in downstream modules

## Further comments
<!-- If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc... -->


Test command: NODE_OPTIONS='--experimental-vm-modules --max-old-space-size=4096' npx --no-install jest app/lib/database/utils.test.js --passWithNoTests 2>&1

Fix patch:
diff --git a/app/containers/MessageBox/index.js b/app/containers/MessageBox/index.js
index 8bd71f70499..f5c16bf88b2 100644
--- a/app/containers/MessageBox/index.js
+++ b/app/containers/MessageBox/index.js
@@ -46,6 +46,7 @@ import CommandsPreview from './CommandsPreview';
 import { getUserSelector } from '../../selectors/login';
 import Navigation from '../../lib/Navigation';
 import { withActionSheet } from '../ActionSheet';
+import { sanitizeLikeString } from '../../lib/database/utils';
 
 const imagePickerConfig = {
 	cropping: true,
@@ -491,8 +492,9 @@ class MessageBox extends Component {
 		const db = database.active;
 		if (keyword) {
 			const customEmojisCollection = db.collections.get('custom_emojis');
+			const likeString = sanitizeLikeString(keyword);
 			let customEmojis = await customEmojisCollection.query(
-				Q.where('name', Q.like(`${ Q.sanitizeLikeString(keyword) }%`))
+				Q.where('name', Q.like(`${ likeString }%`))
 			).fetch();
 			customEmojis = customEmojis.slice(0, MENTIONS_COUNT_TO_DISPLAY);
 			const filteredEmojis = emojis.filter(emoji => emoji.indexOf(keyword) !== -1).slice(0, MENTIONS_COUNT_TO_DISPLAY);
@@ -504,8 +506,9 @@ class MessageBox extends Component {
 	getSlashCommands = debounce(async(keyword) => {
 		const db = database.active;
 		const commandsCollection = db.collections.get('slash_commands');
+		const likeString = sanitizeLikeString(keyword);
 		const commands = await commandsCollection.query(
-			Q.where('id', Q.like(`${ Q.sanitizeLikeString(keyword) }%`))
+			Q.where('id', Q.like(`${ likeString }%`))
 		).fetch();
 		this.setState({ mentions: commands || [] });
 	}, 300)
@@ -734,8 +737,9 @@ class MessageBox extends Component {
 			const db = database.active;
 			const commandsCollection = db.collections.get('slash_commands');
 			const command = message.replace(/ .*/, '').slice(1);
+			const likeString = sanitizeLikeString(command);
 			const slashCommand = await commandsCollection.query(
-				Q.where('id', Q.like(`${ Q.sanitizeLikeString(command) }%`))
+				Q.where('id', Q.like(`${ likeString }%`))
 			).fetch();
 			if (slashCommand.length > 0) {
 				logEvent(events.COMMAND_RUN);
diff --git a/app/lib/database/utils.js b/app/lib/database/utils.js
index 8e580030488..6c5d86b5e7d 100644
--- a/app/lib/database/utils.js
+++ b/app/lib/database/utils.js
@@ -1,1 +1,7 @@
+import XRegExp from 'xregexp';
+
+// Matches letters from any alphabet and numbers
+const likeStringRegex = new XRegExp('[^\\p{L}\\p{Nd}]', 'g');
+export const sanitizeLikeString = str => str.replace(likeStringRegex, '_');
+
 export const sanitizer = r => r;
diff --git a/app/lib/rocketchat.js b/app/lib/rocketchat.js
index 483f183bdd3..5e60b09fa58 100644
--- a/app/lib/rocketchat.js
+++ b/app/lib/rocketchat.js
@@ -56,6 +56,7 @@ import { useSsl } from '../utils/url';
 import UserPreferences from './userPreferences';
 import { Encryption } from './encryption';
 import EventEmitter from '../utils/events';
+import { sanitizeLikeString } from './database/utils';
 
 const TOKEN_KEY = 'reactnativemeteor_usertoken';
 const CURRENT_SERVER = 'currentServer';
@@ -517,8 +518,12 @@ const RocketChat = {
 		}
 
 		const db = database.active;
+		const likeString = sanitizeLikeString(searchText);
 		let data = await db.collections.get('subscriptions').query(
-			Q.where('name', Q.like(`%${ Q.sanitizeLikeString(searchText) }%`))
+			Q.or(
+				Q.where('name', Q.like(`%${ likeString }%`)),
+				Q.where('fname', Q.like(`%${ likeString }%`))
+			)
 		).fetch();
 
 		if (filterUsers && !filterRooms) {
diff --git a/app/views/NewServerView/index.js b/app/views/NewServerView/index.js
index 8b2047acfb4..5d5e4704404 100644
--- a/app/views/NewServerView/index.js
+++ b/app/views/NewServerView/index.js
@@ -30,6 +30,7 @@ import { CloseModalButton } from '../../containers/HeaderButton';
 import { showConfirmationAlert } from '../../utils/info';
 import database from '../../lib/database';
 import ServerInput from './ServerInput';
+import { sanitizeLikeString } from '../../lib/database/utils';
 
 const styles = StyleSheet.create({
 	title: {
@@ -138,10 +139,11 @@ class NewServerView extends React.Component {
 				Q.experimentalSortBy('updated_at', Q.desc),
 				Q.experimentalTake(3)
 			];
+			const likeString = sanitizeLikeString(text);
 			if (text) {
 				whereClause = [
 					...whereClause,
-					Q.where('url', Q.like(`%${ Q.sanitizeLikeString(text) }%`))
+					Q.where('url', Q.like(`%${ likeString }%`))
 				];
 			}
 			const serversHistory = await serversHistoryCollection.query(...whereClause).fetch();
diff --git a/app/views/SearchMessagesView/index.js b/app/views/SearchMessagesView/index.js
index db7d7013f00..607711eb650 100644
--- a/app/views/SearchMessagesView/index.js
+++ b/app/views/SearchMessagesView/index.js
@@ -22,6 +22,7 @@ import { getUserSelector } from '../../selectors/login';
 import SafeAreaView from '../../containers/SafeAreaView';
 import { CloseModalButton } from '../../containers/HeaderButton';
 import database from '../../lib/database';
+import { sanitizeLikeString } from '../../lib/database/utils';
 
 class SearchMessagesView extends React.Component {
 	static navigationOptions = ({ navigation, route }) => {
@@ -83,12 +84,13 @@ class SearchMessagesView extends React.Component {
 		if (this.encrypted) {
 			const db = database.active;
 			const messagesCollection = db.collections.get('messages');
+			const likeString = sanitizeLikeString(searchText);
 			return messagesCollection
 				.query(
 					// Messages of this room
 					Q.where('rid', this.rid),
 					// Message content is like the search text
-					Q.where('msg', Q.like(`%${ Q.sanitizeLikeString(searchText) }%`))
+					Q.where('msg', Q.like(`%${ likeString }%`))
 				)
 				.fetch();
 		}
diff --git a/app/views/ShareListView/index.js b/app/views/ShareListView/index.js
index 250498e4996..1f94ee7439b 100644
--- a/app/views/ShareListView/index.js
+++ b/app/views/ShareListView/index.js
@@ -26,6 +26,7 @@ import { animateNextTransition } from '../../utils/layoutAnimation';
 import { withTheme } from '../../theme';
 import SafeAreaView from '../../containers/SafeAreaView';
 import RocketChat from '../../lib/rocketchat';
+import { sanitizeLikeString } from '../../lib/database/utils';
 
 const permission = {
 	title: I18n.t('Read_External_Permission'),
@@ -195,13 +196,14 @@ class ShareListView extends React.Component {
 			Q.experimentalSortBy('room_updated_at', Q.desc)
 		];
 		if (text) {
+			const likeString = sanitizeLikeString(text);
 			return db.collections
 				.get('subscriptions')
 				.query(
 					...defaultWhereClause,
 					Q.or(
-						Q.where('name', Q.like(`%${ Q.sanitizeLikeString(text) }%`)),
-						Q.where('fname', Q.like(`%${ Q.sanitizeLikeString(text) }%`))
+						Q.where('name', Q.like(`%${ likeString }%`)),
+						Q.where('fname', Q.like(`%${ likeString }%`))
 					)
 				).fetch();
 		}
diff --git a/package.json b/package.json
index 82aadc5e967..b47d79dab39 100644
--- a/package.json
+++ b/package.json
@@ -123,7 +123,8 @@
     "semver": "7.3.2",
     "ua-parser-js": "^0.7.21",
     "url-parse": "^1.4.7",
-    "use-deep-compare-effect": "^1.3.1"
+    "use-deep-compare-effect": "^1.3.1",
+    "xregexp": "^4.3.0"
   },
   "devDependencies": {
     "@babel/core": "^7.8.4",
diff --git a/yarn.lock b/yarn.lock
index ee38acfdd37..bc63f35ec84 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -16109,7 +16109,7 @@ xregexp@4.1.1:
   resolved "https://registry.yarnpkg.com/xregexp/-/xregexp-4.1.1.tgz#eb8a032aa028d403f7b1b22c47a5f16c24b21d8d"
   integrity sha512-QJ1gfSUV7kEOLfpKFCjBJRnfPErUzkNKFMso4kDSmGpp3x6ZgkyKf74inxI7PnnQCFYq5TqYJCd7DrgDN8Q05A==
 
-xregexp@^4.2.4:
+xregexp@^4.2.4, xregexp@^4.3.0:
   version "4.3.0"
   resolved "https://registry.yarnpkg.com/xregexp/-/xregexp-4.3.0.tgz#7e92e73d9174a99a59743f67a4ce879a04b5ae50"
   integrity sha512-7jXDIFXh5yJ/orPn4SXjuVrWWoi4Cr8jfV1eHv9CixKSbU+jY4mxfrBwAuDvupPNKpMUY+FeIqsVw/JLT9+B8g==


Test patch:
diff --git a/app/lib/database/utils.test.js b/app/lib/database/utils.test.js
new file mode 100644
index 00000000000..dd1e3996e1e
--- /dev/null
+++ b/app/lib/database/utils.test.js
@@ -0,0 +1,37 @@
+/* eslint-disable no-undef */
+import * as utils from './utils';
+
+describe('sanitizeLikeStringTester', () => {
+	// example chars that shouldn't return
+	const disallowedChars = ',./;[]!@#$%^&*()_-=+~';
+
+	const sanitizeLikeStringTester = str => expect(utils.sanitizeLikeString(`${ str }${ disallowedChars }`)).toBe(`${ str }${ '_'.repeat(disallowedChars.length) }`);
+
+	// Testing a couple of different alphabets
+	test('render test (latin)', () => {
+		sanitizeLikeStringTester('test123');
+	});
+
+	test('render test (arabic)', () => {
+		sanitizeLikeStringTester('اختبار123');
+	});
+
+	test('render test (russian)', () => {
+		sanitizeLikeStringTester('тест123');
+	});
+
+	test('render test (chinese trad)', () => {
+		sanitizeLikeStringTester('測試123');
+	});
+
+	test('render test (japanese)', () => {
+		sanitizeLikeStringTester('テスト123');
+	});
+});
+
+describe('sanitizer', () => {
+	test('render the same result', () => {
+		const content = { a: true };
+		expect(utils.sanitizer(content)).toBe(content);
+	});
+});