Repository: evennia/evennia

Issue:
`@spawn` traceback within evennia.prototypes.prototypes.protfunc_parser
#### Brief summary of issue / Description of requested feature:

Based on https://github.com/evennia/evennia/wiki/evennia.prototypes.protfuncs, it seems like all `protofunc`s are preceeded by `$` character. However, prototype values without preceding `$` are getting treated as `$protofunc`.

#### Steps to reproduce the issue / Reasons for adding feature:

1. `@spawn {'key':'stone(#1234)'}`


#### Error output / Expected result of feature

```
18-10-24 01:46:49 [::] Traceback (most recent call last):
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/commands/cmdhandler.py", line 594, in _run_command
18-10-24 01:46:49 [::]     _progressive_cmd_run(cmd, ret)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/commands/cmdhandler.py", line 193, in _progressive_cmd_run
18-10-24 01:46:49 [::]     value = generator.next()
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/commands/default/building.py", line 3098, in func
18-10-24 01:46:49 [::]     for obj in spawner.spawn(prototype):
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/prototypes/spawner.py", line 694, in spawn
18-10-24 01:46:49 [::]     create_kwargs["db_key"] = init_spawn_value(val, str)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/prototypes/prototypes.py", line 735, in init_spawn_value
18-10-24 01:46:49 [::]     value = protfunc_parser(value)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/prototypes/prototypes.py", line 583, in protfunc_parser
18-10-24 01:46:49 [::]     stacktrace=stacktrace, testing=testing, **kwargs)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/utils/inlinefuncs.py", line 425, in parse_inlinefunc
18-10-24 01:46:49 [::]     retval = "".join(_run_stack(item) for item in stack)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/utils/inlinefuncs.py", line 425, in <genexpr>
18-10-24 01:46:49 [::]     retval = "".join(_run_stack(item) for item in stack)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/utils/inlinefuncs.py", line 423, in _run_stack
18-10-24 01:46:49 [::]     retval = "" if strip else func(*args, **kwargs)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/prototypes/protfuncs.py", line 315, in obj
18-10-24 01:46:49 [::]     obj = _obj_search(return_list=False, *args, **kwargs)
18-10-24 01:46:49 [::]   File "/usr/src/evennia/evennia/prototypes/protfuncs.py", line 295, in _obj_search
18-10-24 01:46:49 [::]     raise ValueError("$obj: Query '{}' gave no matches.".format(query))
18-10-24 01:46:49 [::] ValueError: $obj: Query '#1234' gave no matches.
18-10-24 01:46:49 [EE] User input was: '@spawn {'key':'stone(#1234)'}'.
```

#### Extra information, such as Evennia revision/repo/branch, operating system and ideas for how to solve / implement:

It seems like the parsing and its regular expression is incorrect?

**Code lifted from `evennia.prototypes.prototypes.protfunc_parser`**

```python
import re
_RE_DBREF = re.compile(r"(?<!\$obj\()(#[0-9]+)")
v = 'stone(#1234)'
_RE_DBREF.sub('$obj(\\1)', v)
'stone($obj(#1234))'
 ```




Discussion:
According to python 2.7 docs for regex:
https://docs.python.org/2/library/re.html

>(?<=...)
Matches if the current position in the string is preceded by a match for ... that ends at the current position. This is called a positive lookbehind assertion. (?<=abc)def will find a match in abcdef, since the lookbehind will back up 3 characters and check if the contained pattern matches. The contained pattern must only match strings of some fixed length, meaning that abc or a|b are allowed, but a* and a{3,4} are not. Group references are not supported even if they match strings of some fixed length. Note that patterns which start with positive lookbehind assertions will not match at the beginning of the string being searched; you will most likely want to use the search() function rather than the match() function:

>>> import re
>>> m = re.search('(?<=abc)def', 'abcdef')
>>> m.group(0)
'def'
This example looks for a word following a hyphen:

>>> m = re.search('(?<=-)\w+', 'spam-egg')
>>> m.group(0)
'egg'
(?<!...)
Matches if the current position in the string is not preceded by a match for .... This is called a negative lookbehind assertion. Similar to positive lookbehind assertions, the contained pattern must only match strings of some fixed length and shouldn’t contain group references. Patterns which start with negative lookbehind assertions may match at the beginning of the string being searched.


Perhaps `_RE_DBREF` the following?

```python
_RE_DBREF = re.compile(r"(?<=\$obj\()(#[0-9]+)")
```

```python
>>> v = 'obj(#123)'
>>> _RE_DBREF = re.compile(r"(?<=\$obj\()(#[0-9]+)")
>>> _RE_DBREF.sub('$obj(\\1)', v)
'obj(#123)'
>>> v = 'hp(#123)'
>>> _RE_DBREF.sub('$obj(\\1)', v)
'hp(#123)'
>>> v = '$obj(#123)'

### Although the following don't look right either
>>> _RE_DBREF.sub('$obj(\\1)', v)
'$obj($obj(#123))'  
### perhaps just `$obj(#123)`
```

I could tackle this issue if the expected behavior for each case is clarified. 
Hi @Griatch, what do you recommend?

I think it's just a matter of defining the requirements - no?

Are strings like `#[0-9]+` always interpreted as `dbref` even when not enclosed by `$obj()`?
@Griatch, I think I could wrap up this quickly today ... just need clarification though 
The first error is not really a bug  - what the regex does is to convert `#1234` into the protfunc `$obj(#1234)` just before passing it into the protfunc parser (this way the protfunc parser can just deal with `$protfuncs` without needing any special cases for `#dbrefs`) . The ValueError says that the object with dbref `1234` does not exist - so it's a real error raised by the `$obj` protfunc. 

What can rather be discussed is the suitability of letting that `ValueError` pass to the user uncaught. Better would likely be for the system to let through `stone(#1234)` as a normal string in this case. 

```
### Although the following don't look right either
>>> _RE_DBREF.sub('$obj(\\1)', v)
'$obj($obj(#123))'  
### perhaps just `$obj(#123)
```

Yes, this is an error - if you manually put `$obj(#123)`, the regex should not add another nesting. 
> what the regex does is to convert #1234 into the protfunc $obj(#1234) 

So any ‘r”#\d+”’ regex will always translate into ‘$obj()’ call?

Isn’t that too strict?

I would think that “stone(#123)” just means that. If it were a protfunc, it must be preceded by $. No?

Coincidentally, github treats # followed by number as a ref to bug. Unless correctly quoted. ```stone(#123)```

These are the tests of what's being passed down to the parser.

https://github.com/evennia/evennia/blob/6ca5b83802bf3b230f774ea5f8e1f3af1d4519ac/evennia/prototypes/tests.py#L315-L319

However, I am still unsure about ```stone(#1234)``` being treated as a protofunc
> I would think that “stone(`#123`)” just means that. If it were a protfunc, it must be preceded by $. No?

Maybe; `#1234` is always a `#dbref` in other parts of Evennia though, so it should be a pretty intuitive thing for builders to use `#dbref` in the prototype. But I can see the argument for always requiring `$` to perform a substutution. 

> Coincidentally, github treats # followed by number as a ref to bug. Unless correctly quoted. `#123`

Yes, note how I always use backticks for those? That's why. :smile: Same thing for talking about  `@command`, or you'll ping a user named `command` somewhere on Github ...

As for the unit tests, might well be that that particular example (`$obj($obj(...))`) was not covered? 

Resolving patch:
diff --git a/evennia/prototypes/protfuncs.py b/evennia/prototypes/protfuncs.py
--- a/evennia/prototypes/protfuncs.py
+++ b/evennia/prototypes/protfuncs.py
@@ -37,12 +37,15 @@ def funcname (*args, **kwargs)
 
 from ast import literal_eval
 from random import randint as base_randint, random as base_random, choice as base_choice
+import re
 
 from evennia.utils import search
 from evennia.utils.utils import justify as base_justify, is_iter, to_str
 
 _PROTLIB = None
 
+_RE_DBREF = re.compile(r"\#[0-9]+")
+
 
 # default protfuncs
 
@@ -325,3 +328,14 @@ def objlist(*args, **kwargs):
 
     """
     return ["#{}".format(obj.id) for obj in _obj_search(return_list=True, *args, **kwargs)]
+
+
+def dbref(*args, **kwargs):
+    """
+    Usage $dbref(<#dbref>)
+    Returns one Object searched globally by #dbref. Error if #dbref is invalid.
+    """
+    if not args or len(args) < 1 or _RE_DBREF.match(args[0]) is None:
+        raise ValueError('$dbref requires a valid #dbref argument.')
+
+    return obj(args[0])
diff --git a/evennia/prototypes/prototypes.py b/evennia/prototypes/prototypes.py
--- a/evennia/prototypes/prototypes.py
+++ b/evennia/prototypes/prototypes.py
@@ -5,7 +5,6 @@
 
 """
 
-import re
 import hashlib
 import time
 from ast import literal_eval
@@ -33,8 +32,6 @@
 _PROTOTYPE_TAG_META_CATEGORY = "db_prototype"
 PROT_FUNCS = {}
 
-_RE_DBREF = re.compile(r"(?<!\$obj\()(#[0-9]+)")
-
 
 class PermissionError(RuntimeError):
     pass
@@ -576,9 +573,6 @@ def protfunc_parser(value, available_functions=None, testing=False, stacktrace=F
 
     available_functions = PROT_FUNCS if available_functions is None else available_functions
 
-    # insert $obj(#dbref) for #dbref
-    value = _RE_DBREF.sub("$obj(\\1)", value)
-
     result = inlinefuncs.parse_inlinefunc(
         value, available_funcs=available_functions,
         stacktrace=stacktrace, testing=testing, **kwargs)


Test patch:
diff --git a/evennia/prototypes/tests.py b/evennia/prototypes/tests.py
--- a/evennia/prototypes/tests.py
+++ b/evennia/prototypes/tests.py
@@ -11,6 +11,7 @@
 from evennia.utils.tests.test_evmenu import TestEvMenu
 from evennia.prototypes import spawner, prototypes as protlib
 from evennia.prototypes import menus as olc_menus
+from evennia.prototypes import protfuncs as protofuncs
 
 from evennia.prototypes.prototypes import _PROTOTYPE_TAG_META_CATEGORY
 
@@ -312,11 +313,83 @@ def test_protfuncs(self):
         self.assertEqual(protlib.protfunc_parser(
             "$eval({'test': '1', 2:3, 3: $toint(3.5)})"), {'test': '1', 2: 3, 3: 3})
 
-        self.assertEqual(protlib.protfunc_parser("$obj(#1)", session=self.session), '#1')
-        self.assertEqual(protlib.protfunc_parser("#1", session=self.session), '#1')
-        self.assertEqual(protlib.protfunc_parser("$obj(Char)", session=self.session), '#6')
-        self.assertEqual(protlib.protfunc_parser("$obj(Char)", session=self.session), '#6')
-        self.assertEqual(protlib.protfunc_parser("$objlist(#1)", session=self.session), ['#1'])
+
+        # no object search
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("obj(#1)", session=self.session), 'obj(#1)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("dbref(#1)", session=self.session), 'dbref(#1)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("stone(#12345)", session=self.session), 'stone(#12345)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("#1", session=self.session), '#1')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("#12345", session=self.session), '#12345')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("nothing(#1)", session=self.session), 'nothing(#1)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("(#12345)", session=self.session), '(#12345)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("obj(Char)", session=self.session), 'obj(Char)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("objlist(#1)", session=self.session), 'objlist(#1)')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("dbref(Char)", session=self.session), 'dbref(Char)')
+            mocked__obj_search.assert_not_called()
+
+
+        # obj search happens
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("$objlist(#1)", session=self.session), ['#1'])
+            mocked__obj_search.assert_called_once()
+            assert ('#1',) == mocked__obj_search.call_args[0]
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("$obj(#1)", session=self.session), '#1')
+            mocked__obj_search.assert_called_once()
+            assert ('#1',) == mocked__obj_search.call_args[0]
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("$dbref(#1)", session=self.session), '#1')
+            mocked__obj_search.assert_called_once()
+            assert ('#1',) == mocked__obj_search.call_args[0]
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("$obj(Char)", session=self.session), '#6')
+            mocked__obj_search.assert_called_once()
+            assert ('Char',) == mocked__obj_search.call_args[0]
+
+
+        # bad invocation
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertEqual(protlib.protfunc_parser("$badfunc(#1)", session=self.session), '<UNKNOWN>')
+            mocked__obj_search.assert_not_called()
+
+        with mock.patch("evennia.prototypes.protfuncs._obj_search", wraps=protofuncs._obj_search) as mocked__obj_search:
+            self.assertRaises(ValueError, protlib.protfunc_parser, "$dbref(Char)")
+            mocked__obj_search.assert_not_called()
+
 
         self.assertEqual(protlib.value_to_obj(
             protlib.protfunc_parser("#6", session=self.session)), self.char1)