Repository: spyder-ide/spyder

Issue:
Files shown without hits when maximum number of results reached
<!--- **PLEASE READ:** When submitting here, please ensure you've completed the following checklist and checked the boxes to confirm. Issue reports without it may be closed. Thanks! --->

### Issue Report Checklist

* [x] Searched the [issues page](https://github.com/spyder-ide/spyder/issues?q=is%3Aissue) for similar reports
* [x] Read the relevant sections of the [Spyder Troubleshooting Guide](https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ) and followed its advice
* [x] Reproduced the issue after updating with ``conda update spyder`` (or ``pip``, if not using Anaconda)
* [ ] Could not reproduce inside ``jupyter qtconsole`` (if console-related)
* [ ] Tried basic troubleshooting (if a bug/error)
    * [ ] Restarted Spyder
    * [ ] Reset preferences with ``spyder --reset``
    * [ ] Reinstalled the latest version of [Anaconda](https://www.anaconda.com/download/)
    * [ ] Tried the other applicable steps from the Troubleshooting Guide
* [x] Completed the **Problem Description**, **Steps to Reproduce** and **Version** sections below


## Problem Description
Minor issue that doesn't really affect anything, but still something that one may want to fix.

When searching using the Find pane and reaching the maximum number of hits, sometimes a file is added to the list that do not have any hits. I've also seen cases where two files without hits are added. Probably some counter is a bit off, so the files are added although the maximum number of results is already reached.

![image](https://user-images.githubusercontent.com/8114497/130298961-92ff9a9a-86a4-4239-81ad-111b7cab4006.png)



### What steps reproduce the problem?

1. Search for something quite common in a directory
2. Have a bit of "luck"

### What is the expected output? What do you see instead?
Files without any visible hits should not be in the search results. The files that have no search results, do indeed have hits, just that they do not show up as the maximum number is reached.


### Paste Traceback/Error Below (if applicable)
<!--- Copy from error dialog or View > Panes > Internal Console --->



## Versions
<!--- You can get this information from Help > About Spyder...
or (if Spyder won't launch) the "conda list" command
from the Anaconda Prompt/Terminal/command line. --->

* Spyder version: 5.0.5 None
* Python version: 3.9.5 64-bit
* Qt version: 5.9.7
* PyQt5 version: 5.9.2
* Operating System: Windows 10

### Dependencies
<!--- Please go to the menu entry Help > Dependencies,
press the Copy to clipboard button and paste below --->

```

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  1.6.0 (OK)
cookiecutter >=1.6.0          :  1.7.2 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0               :  7.26.0 (OK)
jedi =0.17.2                  :  0.17.2 (OK)
jsonschema >=3.2.0            :  3.2.0 (OK)
keyring >=17.0.0              :  23.0.1 (OK)
nbconvert >=4.0               :  6.1.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
paramiko >=2.4.0              :  2.7.2 (OK)
parso =0.7.0                  :  0.7.0 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.8.0 (OK)
pygments >=2.0                :  2.9.0 (OK)
pylint >=1.0                  :  2.9.6 (OK)
pyls >=0.36.2;<1.0.0          :  0.36.2 (OK)
pyls_black >=0.4.6            :  0.4.6 (OK)
pyls_spyder >=0.3.2;<0.4.0    :  0.3.2 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.1.10 (OK)
qtawesome >=1.0.2             :  1.0.2 (OK)
qtconsole >=5.1.0             :  5.1.0 (OK)
qtpy >=1.5.0                  :  1.9.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=39.0.0           :  52.0.0.post20210125 (OK)
sphinx >=0.6.6                :  4.0.2 (OK)
spyder_kernels >=2.0.4;<2.1.0 :  2.0.5 (OK)
textdistance >=4.2.0          :  4.2.1 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  2.1.3 (OK)
zmq >=17                      :  20.0.0 (OK)

# Optional:
cython >=0.21                 :  None (NOK)
matplotlib >=2.0.0            :  None (OK)
numpy >=1.7                   :  1.20.3 (OK)
pandas >=1.1.1                :  None (NOK)
scipy >=0.17.0                :  1.6.2 (OK)
sympy >=0.7.3                 :  1.9.dev (OK)

```



Discussion:
Hi @oscargus,

Thanks for opening this issue, if you could find a way to have the systematic way of reproducing this bug it will be great. We will work on this for one of our future releases. 

@ccordoba12 have you encounter this issue while doing the improvements for the files pane?
I'd say that it is systematic in the sense that it always behave like this, but to actually trigger it require some specific searches.

However, searching for `class` in the current master spyder source directory (commit bb2ed9a6c90df30f370f7f67413d31b363cbfe15 ) gives eight files at the end without any hits. Even if you manage to add a few classes before you can try it, something like that is quite likely to trigger it. Now I was using the current master version of Spyder, so I can confirm that it is still there.
I'll give it a try! Thanks :)
> @ccordoba12 have you encounter this issue while doing the improvements for the files pane?

I haven't seen this error but I usually don't go until the end of the list.

> However, searching for class in the current master spyder source directory (commit bb2ed9a ) gives eight files at the end without any hits

Thanks @oscargus! I could reproduce it with your help and will try to fix it in our 5.2.0 version, to be released at the end of September.

Resolving patch:
diff --git a/spyder/plugins/findinfiles/widgets/main_widget.py b/spyder/plugins/findinfiles/widgets/main_widget.py
--- a/spyder/plugins/findinfiles/widgets/main_widget.py
+++ b/spyder/plugins/findinfiles/widgets/main_widget.py
@@ -589,7 +589,12 @@ def start(self):
         # Start
         self.running = True
         self.start_spinner()
-        self.search_thread = SearchThread(None, search_text, self.text_color)
+        self.search_thread = SearchThread(
+            None,
+            search_text,
+            self.text_color,
+            self.get_conf('max_results')
+        )
         self.search_thread.sig_finished.connect(self._handle_search_complete)
         self.search_thread.sig_file_match.connect(
             self.result_browser.append_file_result
@@ -630,10 +635,16 @@ def set_max_results(self, value=None):
             dialog.setWindowTitle(_('Max results'))
             dialog.setLabelText(_('Set maximum number of results: '))
             dialog.setInputMode(QInputDialog.IntInput)
-            dialog.setIntRange(1, 10000)
             dialog.setIntStep(1)
             dialog.setIntValue(self.get_conf('max_results'))
 
+            # In order to show the right number of results when max_results is
+            # reached, we can't allow users to introduce less than 2 in this
+            # dialog. Since that value seems a bit arbitrary, we decided to set
+            # it to 5.
+            # See spyder-ide/spyder#16256
+            dialog.setIntRange(5, 10000)
+
             # Connect slot
             dialog.intValueSelected.connect(
                 lambda value: self.set_conf('max_results', value))
diff --git a/spyder/plugins/findinfiles/widgets/results_browser.py b/spyder/plugins/findinfiles/widgets/results_browser.py
--- a/spyder/plugins/findinfiles/widgets/results_browser.py
+++ b/spyder/plugins/findinfiles/widgets/results_browser.py
@@ -11,8 +11,8 @@
 
 # Third party imports
 from qtpy.QtCore import QPoint, QSize, Qt, Signal, Slot
-from qtpy.QtGui import (QAbstractTextDocumentLayout, QColor, QBrush,
-                        QFontMetrics, QPalette, QTextDocument)
+from qtpy.QtGui import (QAbstractTextDocumentLayout, QColor, QFontMetrics,
+                        QTextDocument)
 from qtpy.QtWidgets import (QApplication, QStyle, QStyledItemDelegate,
                             QStyleOptionViewItem, QTreeWidgetItem)
 
@@ -260,13 +260,20 @@ def clear_title(self, search_text):
     def append_file_result(self, filename):
         """Real-time update of file items."""
         if len(self.data) < self.max_results:
-            self.files[filename] = item = FileMatchItem(
-                self,
-                self.path,
-                filename,
-                self.sorting,
-                self.text_color
-            )
+            # Catch any error while creating file items.
+            # Fixes spyder-ide/spyder#17443
+            try:
+                item = FileMatchItem(
+                    self,
+                    self.path,
+                    filename,
+                    self.sorting,
+                    self.text_color
+                )
+            except Exception:
+                return
+
+            self.files[filename] = item
 
             item.setExpanded(True)
             self.num_files += 1
diff --git a/spyder/plugins/findinfiles/widgets/search_thread.py b/spyder/plugins/findinfiles/widgets/search_thread.py
--- a/spyder/plugins/findinfiles/widgets/search_thread.py
+++ b/spyder/plugins/findinfiles/widgets/search_thread.py
@@ -58,12 +58,14 @@ class SearchThread(QThread):
     power = 0       # 0**1 = 1
     max_power = 9   # 2**9 = 512
 
-    def __init__(self, parent, search_text, text_color):
+    def __init__(self, parent, search_text, text_color, max_results=1000):
         super().__init__(parent)
-        self.mutex = QMutex()
-        self.stopped = None
         self.search_text = search_text
         self.text_color = text_color
+        self.max_results = max_results
+
+        self.mutex = QMutex()
+        self.stopped = None
         self.pathlist = None
         self.total_matches = None
         self.error_flag = None
@@ -80,6 +82,7 @@ def __init__(self, parent, search_text, text_color):
         self.num_files = 0
         self.files = []
         self.partial_results = []
+        self.total_items = 0
 
     def initialize(self, path, is_file, exclude,
                    texts, text_re, case_sensitive):
@@ -286,16 +289,18 @@ def process_results(self):
         items = []
         num_matches = self.total_matches
         for result in self.partial_results:
-            filename, lineno, colno, match_end, line = result
-
-            if filename not in self.files:
-                self.files.append(filename)
-                self.sig_file_match.emit(filename)
-                self.num_files += 1
-
-            line = self.truncate_result(line, colno, match_end)
-            item = (filename, lineno, colno, line, match_end)
-            items.append(item)
+            if self.total_items < self.max_results:
+                filename, lineno, colno, match_end, line = result
+
+                if filename not in self.files:
+                    self.files.append(filename)
+                    self.sig_file_match.emit(filename)
+                    self.num_files += 1
+
+                line = self.truncate_result(line, colno, match_end)
+                item = (filename, lineno, colno, line, match_end)
+                items.append(item)
+                self.total_items += 1
 
         # Process title
         title = "'%s' - " % self.search_text


Test patch:
diff --git a/spyder/plugins/findinfiles/tests/test_plugin.py b/spyder/plugins/findinfiles/tests/test_plugin.py
--- a/spyder/plugins/findinfiles/tests/test_plugin.py
+++ b/spyder/plugins/findinfiles/tests/test_plugin.py
@@ -19,10 +19,8 @@
 from spyder.plugins.findinfiles.plugin import FindInFiles
 from spyder.plugins.findinfiles.widgets.combobox import SELECT_OTHER
 
+
 LOCATION = osp.realpath(osp.join(os.getcwd(), osp.dirname(__file__)))
-NONASCII_DIR = osp.join(LOCATION, "èáïü Øαôå 字分误")
-if not osp.exists(NONASCII_DIR):
-    os.makedirs(NONASCII_DIR)
 
 
 @pytest.fixture
@@ -39,13 +37,14 @@ def findinfiles(qtbot):
 
 # ---- Tests for FindInFiles plugin
 @pytest.mark.order(1)
-def test_closing_plugin(findinfiles, qtbot, mocker):
+def test_closing_plugin(findinfiles, qtbot, mocker, tmpdir):
     """
     Test that the external paths listed in the combobox are saved and loaded
     correctly from the spyder config file.
     """
     path_selection_combo = findinfiles.get_widget().path_selection_combo
     path_selection_combo.clear_external_paths()
+    nonascii_dir = str(tmpdir.mkdir("èáïü Øαôå 字分误"))
     assert path_selection_combo.get_external_paths() == []
 
     # Add external paths to the path_selection_combo.
@@ -53,7 +52,7 @@ def test_closing_plugin(findinfiles, qtbot, mocker):
         LOCATION,
         osp.dirname(LOCATION),
         osp.dirname(osp.dirname(LOCATION)),
-        NONASCII_DIR,
+        nonascii_dir,
     ]
     for external_path in expected_results:
         mocker.patch(
diff --git a/spyder/plugins/findinfiles/widgets/tests/test_widgets.py b/spyder/plugins/findinfiles/widgets/tests/test_widgets.py
--- a/spyder/plugins/findinfiles/widgets/tests/test_widgets.py
+++ b/spyder/plugins/findinfiles/widgets/tests/test_widgets.py
@@ -20,6 +20,7 @@
 from qtpy.QtWidgets import QMessageBox
 
 # Local imports
+from spyder.config.base import running_in_ci
 from spyder.config.manager import CONF
 from spyder.plugins.findinfiles.widgets.main_widget import FindInFilesWidget
 from spyder.plugins.findinfiles.widgets.combobox import (
@@ -27,6 +28,7 @@
     SELECT_OTHER)
 from spyder.plugins.findinfiles.widgets.search_thread import SearchThread
 from spyder.utils.palette import QStylePalette, SpyderPalette
+from spyder.utils.stylesheet import APP_STYLESHEET
 
 
 LOCATION = osp.realpath(osp.join(os.getcwd(), osp.dirname(__file__)))
@@ -84,6 +86,7 @@ def teardown():
         widget.setup()
 
     widget.resize(640, 480)
+    widget.setStyleSheet(str(APP_STYLESHEET))
     qtbot.addWidget(widget)
     widget.show()
     return widget
@@ -310,6 +313,41 @@ def test_exclude_regexp_error(findinfiles, qtbot):
     assert findinfiles.REGEX_ERROR in tooltip
 
 
+@flaky(max_runs=5)
+@pytest.mark.skipif(not running_in_ci(), reason="Only works on CIs")
+@pytest.mark.skipif(os.name == 'nt', reason="Fails on Windows")
+def test_no_empty_file_items(findinfiles, qtbot):
+    """
+    Test that a search that hits the max number of results doesn't generate
+    empty file items.
+
+    This is a regression test for issue spyder-ide/spyder#16256
+    """
+    findinfiles.set_search_text("spam")
+    findinfiles.set_directory(osp.join(LOCATION, "data"))
+    findinfiles.set_conf('max_results', 6)
+
+    with qtbot.waitSignal(findinfiles.sig_max_results_reached):
+        findinfiles.find()
+
+    # Assert the results are the expected ones to reproduce the bug this test
+    # tries to catch. In other words, this is here to prevent future changes to
+    # our test files that would render this test useless.
+    results = {
+        'spam.py': [(2, 7), (5, 1), (7, 12)],
+        'spam.txt': [(1, 0), (1, 5), (3, 22)]
+    }
+    assert process_search_results(findinfiles.result_browser.data) == results
+
+    # Assert that the files with results are exactly the same as those
+    # displayed in the results browser.
+    files_with_results = set(
+        [v[0] for v in findinfiles.result_browser.data.values()]
+    )
+    displayed_files = set(findinfiles.result_browser.files.keys())
+    assert files_with_results == displayed_files
+
+
 # ---- Tests for SearchInComboBox
 
 def test_add_external_paths(searchin_combobox, mocker):