Repository: evennia/evennia

Issue:
[BUG] CmdFind - Runaway memory usage with improper syntax
#### Describe the bug
Was dealing with a "bug" where the `find` command was returning seemingly random results and/or dumping the entire contents of the database. Like #1994 this caused the entire queryset to be loaded into memory before rendering to the user.

It turns out the syntax I was using was incorrect but it was accepted anyway, which led to these side effects.

Ex: `find/startswith = searchterm`

Given that I did not provide a lhs and the rhs should only be a dbref/range, validating both before executing any queries would be beneficial.

#### Environment, Evennia version, OS etc
Dev
[BUG] CmdFind - Runaway memory usage with improper syntax
#### Describe the bug
Was dealing with a "bug" where the `find` command was returning seemingly random results and/or dumping the entire contents of the database. Like #1994 this caused the entire queryset to be loaded into memory before rendering to the user.

It turns out the syntax I was using was incorrect but it was accepted anyway, which led to these side effects.

Ex: `find/startswith = searchterm`

Given that I did not provide a lhs and the rhs should only be a dbref/range, validating both before executing any queries would be beneficial.

#### Environment, Evennia version, OS etc
Dev


Discussion:
@strikaco Btw, it's worth to stick with `master` branch for bugs like this; it is still the most quickly updating branch at this time. 
@Griatch Thanks for the pointer, but I'm a bit confused. I was under the impression `develop` was the bleeding-edge branch. Should PRs for things like this be submitted against `develop` or `master`?
Generally, `master` is always for bug fixes, `develop` is for completely new features. This one definitely belongs in `master`. 

Usually though, once develop merges into master (a new release), work continues on master for a while since there are usually bugs popping up that needs to be fixed. Once switching to develop, most work will happen on that. At the moment I've not really started up working on new stuff in `develop` branch and is working on stuff in master, so it's the fastest updating branch right now. 
@strikaco Btw, it's worth to stick with `master` branch for bugs like this; it is still the most quickly updating branch at this time. 
@Griatch Thanks for the pointer, but I'm a bit confused. I was under the impression `develop` was the bleeding-edge branch. Should PRs for things like this be submitted against `develop` or `master`?
Generally, `master` is always for bug fixes, `develop` is for completely new features. This one definitely belongs in `master`. 

Usually though, once develop merges into master (a new release), work continues on master for a while since there are usually bugs popping up that needs to be fixed. Once switching to develop, most work will happen on that. At the moment I've not really started up working on new stuff in `develop` branch and is working on stuff in master, so it's the fastest updating branch right now. 

Resolving patch:
diff --git a/evennia/commands/default/building.py b/evennia/commands/default/building.py
--- a/evennia/commands/default/building.py
+++ b/evennia/commands/default/building.py
@@ -3,7 +3,7 @@
 """
 import re
 from django.conf import settings
-from django.db.models import Q
+from django.db.models import Q, Min, Max
 from evennia.objects.models import ObjectDB
 from evennia.locks.lockhandler import LockException
 from evennia.commands.cmdhandler import get_and_merge_cmdsets
@@ -13,6 +13,7 @@
     class_from_module,
     get_all_typeclasses,
     variable_from_module,
+    dbref,
 )
 from evennia.utils.eveditor import EvEditor
 from evennia.utils.evmore import EvMore
@@ -2768,7 +2769,7 @@ def func(self):
         caller = self.caller
         switches = self.switches
 
-        if not self.args:
+        if not self.args or (not self.lhs and not self.rhs):
             caller.msg("Usage: find <string> [= low [-high]]")
             return
 
@@ -2778,18 +2779,40 @@ def func(self):
             switches.append("loc")
 
         searchstring = self.lhs
-        low, high = 1, ObjectDB.objects.all().order_by("-id")[0].id
+        
+        try:
+            # Try grabbing the actual min/max id values by database aggregation
+            qs = ObjectDB.objects.values('id').aggregate(low=Min('id'), high=Max('id'))
+            low, high = sorted(qs.values())
+            if not (low and high):
+                raise ValueError(f"{self.__class__.__name__}: Min and max ID not returned by aggregation; falling back to queryset slicing.")
+        except Exception as e:
+            logger.log_trace(e)
+            # If that doesn't work for some reason (empty DB?), guess the lower 
+            # bound and do a less-efficient query to find the upper.
+            low, high = 1, ObjectDB.objects.all().order_by("-id").first().id
+            
         if self.rhs:
-            if "-" in self.rhs:
-                # also support low-high syntax
-                limlist = [part.lstrip("#").strip() for part in self.rhs.split("-", 1)]
-            else:
-                # otherwise split by space
-                limlist = [part.lstrip("#") for part in self.rhs.split(None, 1)]
-            if limlist and limlist[0].isdigit():
-                low = max(low, int(limlist[0]))
-            if len(limlist) > 1 and limlist[1].isdigit():
-                high = min(high, int(limlist[1]))
+            try:
+                # Check that rhs is either a valid dbref or dbref range
+                bounds = tuple(sorted(dbref(x, False) for x in re.split('[-\s]+', self.rhs.strip())))
+                
+                # dbref() will return either a valid int or None
+                assert bounds
+                # None should not exist in the bounds list
+                assert None not in bounds
+                
+                low = bounds[0]
+                if len(bounds) > 1:
+                    high = bounds[-1]
+                    
+            except AssertionError:
+                caller.msg("Invalid dbref range provided (not a number).")
+                return
+            except IndexError as e:
+                logger.log_err(f"{self.__class__.__name__}: Error parsing upper and lower bounds of query.")
+                logger.log_trace(e)
+                
         low = min(low, high)
         high = max(low, high)
 
@@ -2801,7 +2824,6 @@ def func(self):
             restrictions = ", %s" % (", ".join(self.switches))
 
         if is_dbref or is_account:
-
             if is_dbref:
                 # a dbref search
                 result = caller.search(searchstring, global_search=True, quiet=True)
@@ -2838,7 +2860,7 @@ def func(self):
                     )
         else:
             # Not an account/dbref search but a wider search; build a queryset.
-            # Searchs for key and aliases
+            # Searches for key and aliases
             if "exact" in switches:
                 keyquery = Q(db_key__iexact=searchstring, id__gte=low, id__lte=high)
                 aliasquery = Q(
@@ -2866,53 +2888,43 @@ def func(self):
                     id__lte=high,
                 )
 
-            results = ObjectDB.objects.filter(keyquery | aliasquery).distinct()
-            nresults = results.count()
-
-            if nresults:
-                # convert result to typeclasses.
-                results = [result for result in results]
-                if "room" in switches:
-                    results = [
-                        obj for obj in results if inherits_from(obj, ROOM_TYPECLASS)
-                    ]
-                if "exit" in switches:
-                    results = [
-                        obj for obj in results if inherits_from(obj, EXIT_TYPECLASS)
-                    ]
-                if "char" in switches:
-                    results = [
-                        obj for obj in results if inherits_from(obj, CHAR_TYPECLASS)
-                    ]
-                nresults = len(results)
+            # Keep the initial queryset handy for later reuse
+            result_qs = ObjectDB.objects.filter(keyquery | aliasquery).distinct()
+            nresults = result_qs.count()
+            
+            # Use iterator to minimize memory ballooning on large result sets
+            results = result_qs.iterator()
+            
+            # Check and see if type filtering was requested; skip it if not
+            if any(x in switches for x in ("room", "exit", "char")):
+                obj_ids = set()
+                for obj in results:
+                    if ("room" in switches and inherits_from(obj, ROOM_TYPECLASS)) \
+                    or ("exit" in switches and inherits_from(obj, EXIT_TYPECLASS)) \
+                    or ("char" in switches and inherits_from(obj, CHAR_TYPECLASS)):
+                        obj_ids.add(obj.id)
+                
+                # Filter previous queryset instead of requesting another
+                filtered_qs = result_qs.filter(id__in=obj_ids).distinct()
+                nresults = filtered_qs.count()
+                
+                # Use iterator again to minimize memory ballooning
+                results = filtered_qs.iterator()
 
             # still results after type filtering?
             if nresults:
-                if nresults > 1:
-                    string = "|w%i Matches|n(#%i-#%i%s):" % (
-                        nresults,
-                        low,
-                        high,
-                        restrictions,
-                    )
-                    for res in results:
-                        string += "\n   |g%s - %s|n" % (
-                            res.get_display_name(caller),
-                            res.path,
-                        )
-                else:
-                    string = "|wOne Match|n(#%i-#%i%s):" % (low, high, restrictions)
-                    string += "\n   |g%s - %s|n" % (
-                        results[0].get_display_name(caller),
-                        results[0].path,
-                    )
-                    if "loc" in self.switches and nresults == 1 and results[0].location:
-                        string += " (|wlocation|n: |g{}|n)".format(
-                            results[0].location.get_display_name(caller)
-                        )
+                if nresults > 1: header = f'{nresults} Matches'
+                else: header = 'One Match'
+                
+                string = f"|w{header}|n(#{low}-#{high}{restrictions}):"
+                res = None
+                for res in results:
+                    string += f"\n   |g{res.get_display_name(caller)} - {res.path}|n"
+                if "loc" in self.switches and nresults == 1 and res and getattr(res, 'location', None):
+                    string += f" (|wlocation|n: |g{res.location.get_display_name(caller)}|n)"
             else:
-                string = "|wMatch|n(#%i-#%i%s):" % (low, high, restrictions)
-                string += "\n   |RNo matches found for '%s'|n" % searchstring
+                string = f"|wNo Matches|n(#{low}-#{high}{restrictions}):"
+                string += f"\n   |RNo matches found for '{searchstring}'|n"
 
         # send result
         caller.msg(string.strip())
@@ -3695,4 +3707,4 @@ def _search_show_prototype(query, prototypes=None):
             for obj in spawner.spawn(prototype):
                 self.caller.msg("Spawned %s." % obj.get_display_name(self.caller))
         except RuntimeError as err:
-            caller.msg(err)
+            caller.msg(err)
\ No newline at end of file


Test patch:
diff --git a/evennia/commands/default/tests.py b/evennia/commands/default/tests.py
--- a/evennia/commands/default/tests.py
+++ b/evennia/commands/default/tests.py
@@ -1210,11 +1210,32 @@ def test_find(self):
         self.call(building.CmdFind(), self.char1.dbref, "Exact dbref match")
         self.call(building.CmdFind(), "*TestAccount", "Match")
 
-        self.call(building.CmdFind(), "/char Obj")
-        self.call(building.CmdFind(), "/room Obj")
-        self.call(building.CmdFind(), "/exit Obj")
+        self.call(building.CmdFind(), "/char Obj", "No Matches")
+        self.call(building.CmdFind(), "/room Obj", "No Matches")
+        self.call(building.CmdFind(), "/exit Obj", "No Matches")
         self.call(building.CmdFind(), "/exact Obj", "One Match")
-
+        
+        # Test multitype filtering
+        with mock.patch('evennia.commands.default.building.CHAR_TYPECLASS', 'evennia.objects.objects.DefaultCharacter'):
+            self.call(building.CmdFind(), "/char/room Obj", "No Matches")
+            self.call(building.CmdFind(), "/char/room/exit Char", "2 Matches")
+            self.call(building.CmdFind(), "/char/room/exit/startswith Cha", "2 Matches")
+        
+        # Test null search
+        self.call(building.CmdFind(), "=", "Usage: ")
+        
+        # Test bogus dbref range with no search term
+        self.call(building.CmdFind(), "= obj", "Invalid dbref range provided (not a number).")
+        self.call(building.CmdFind(), "= #1a", "Invalid dbref range provided (not a number).")
+        
+        # Test valid dbref ranges with no search term
+        self.call(building.CmdFind(), "=#1", "7 Matches(#1-#7)")
+        self.call(building.CmdFind(), "=1-2", "2 Matches(#1-#2):")
+        self.call(building.CmdFind(), "=1 - 2", "2 Matches(#1-#2):")
+        self.call(building.CmdFind(), "=1- #2", "2 Matches(#1-#2):")
+        self.call(building.CmdFind(), "=1-#2", "2 Matches(#1-#2):")
+        self.call(building.CmdFind(), "=#1-2", "2 Matches(#1-#2):")
+        
     def test_script(self):
         self.call(building.CmdScript(), "Obj = ", "No scripts defined on Obj")
         self.call(
@@ -1679,4 +1700,4 @@ def test_channelcommand(self, mock_channeldb):
         mock_channeldb.objects.get_channel = mock.MagicMock(return_value=channel)
 
         self.call(syscommands.SystemSendToChannel(), "public:Hello")
-        channel.msg.assert_called()
+        channel.msg.assert_called()
\ No newline at end of file