Repository: evennia/evennia

Issue:
Errors in Commands cause default cmdsets to not load
#### Brief summary of issue / Description of requested feature:
I have noticed that errors in some original commands I'm creating, will apparently prevent any commands from loading. 

#### Steps to reproduce the issue / Reasons for adding feature:

1.  Introduce a syntax error into a command that is part of the CharacterCmdSet
2.  `@reload`
3.  Enter any command. 
> `@reload`
> help
> quit

#### Error output / Expected result of feature
```
SyntaxError encountered when loading cmdset 'commands.default_cmdsets.CharacterCmdSet'.
(Traceback was logged 2017-02-06 16:27:32-0000)
Command '@reload' is not available. Type "help" for help.
Command 'help' is not available. Type "help" for help.
Command 'quit' is not available. Type "help" for help.
```

Since no commands work, the player cannot quit without closing their client.  In practice I've had to go back to my pyenv and run > evennia reload after fixing the error.

#### Extra information, such as Evennia revision/repo/branch, operating system and ideas for how to solve / implement:

IMHO an error in a single command ought not have such a devastating effect it would be better if only the command in question failed to run... or perhaps at least limit it to a single command set.

EDIT: My opinion has changed to agree with Griatch explanation below.  Now I believe that it would be better for evennia to refuse to load at all, rather than let user log into a crippled mud.


Discussion:
Limiting this to one Command is not possible, because what happens when you make a critical syntax error at the module level is that you have created an invalid Python module - this means the module cannot be imported at all and will take all it contains with it. Running `evennia reload` (or `evennia start`) is not an extraordinary thing, you should generally be ready to do so if you create a situation where the server cannot start. 
That said, it was many years since I looked at this since, so it may be worth to examine further. 
@j3b4 Actually, introducing a syntax error generally does not allow the server to come back at all after a `@reload` - you must fix it and run `evennia start`. Could you give an example of a syntax error that allows you to `@reload` and leads to the loss of cmdsets? 
I know it happened one or more times previously with different errors, but I didn't really question it.  This last error is reproducible by copy-pasting a non-ASCII character into a string literal in a world file that gets imported into one of my commands.

This is the server log:
``
17-02-07 00:30:57-0000)
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::] Traceback (most recent call last):
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/evennia/evennia/commands/cmdsethandler.py", line 160, in import_cmdset
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]     module = import_module(modpath, package="evennia")
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]     __import__(name)
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/Apendixie/commands/default_cmdsets.py", line 20, in <module>
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]     from tables import CmdTable
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/Apendixie/commands/tables.py", line 9, in <module>
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]     from world.tables import TABLES as tableset
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/Apendixie/world/tables.py", line 458
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [::] SyntaxError: Non-ASCII character '\xe2' in file /home/aeon/Evennia/Apendixie/world/tables.py on line 459, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
2017-02-07 00:30:57+0000 [AMPProtocol,0,127.0.0.1] [EE] Traceback (most recent call last):
``

And this is what I see in the client:
`Traceback (most recent call last):
  File "/home/aeon/Evennia/evennia/evennia/commands/cmdsethandler.py", line 160, in import_cmdset
    module = import_module(modpath, package="evennia")
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/aeon/Evennia/Apendixie/commands/default_cmdsets.py", line 20, in <module>
    from tables import CmdTable
  File "/home/aeon/Evennia/Apendixie/commands/tables.py", line 9, in <module>
    from world.tables import TABLES as tableset
  File "/home/aeon/Evennia/Apendixie/world/tables.py", line 458
SyntaxError: Non-ASCII character '\xe2' in file /home/aeon/Evennia/Apendixie/world/tables.py on line 459, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details`

The client allows me to connect at this point.  But I don't seem to have any commands.
Thanks. This is not a general syntax error then but a specific subset of error related to the lack of encoding. 
Here is another example I stumbled on today:

```python
`Server restart initiated ...
...
...
...
...
...
...
...
...
...
...
...
...
Traceback (most recent call last):
  File "/home/aeon/Evennia/evennia/evennia/commands/cmdsethandler.py", line 160, in import_cmdset
    module = import_module(modpath, package="evennia")
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/aeon/Evennia/Apendixie/commands/default_cmdsets.py", line 19, in <module>
    from typeclasses.doors import CmdOpen, CmdOpenCloseDoor
  File "/home/aeon/Evennia/Apendixie/typeclasses/doors.py", line 31, in <module>
    class SealedDoor(typeclasses.objects.DefaultObject):
NameError: name 'typeclasses' is not defined

Compile/Run error when loading cmdset 'commands.default_cmdsets.SessionCmdSet'.",
(Traceback was logged 2017-02-09 00:17:46-0000)
... Server restarted.`


`2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]     class SealedDoor(typeclasses.objects.DefaultObject):
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] NameError: name 'typeclasses' is not defined
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] Compile/Run error when loading cmdset 'commands.default_cmdsets.PlayerCmdSet'.",
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] (Traceback was logged 2017-02-09 00:18:49-0000)
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::] Traceback (most recent call last):
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/evennia/evennia/commands/cmdsethandler.py", line 160, in import_cmdset
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]     module = import_module(modpath, package="evennia")
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]     __import__(name)
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/Apendixie/commands/default_cmdsets.py", line 19, in <module>
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]     from typeclasses.doors import CmdOpen, CmdOpenCloseDoor
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]   File "/home/aeon/Evennia/Apendixie/typeclasses/doors.py", line 31, in <module>
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::]     class SealedDoor(typeclasses.objects.DefaultObject):
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [::] NameError: name 'typeclasses' is not defined
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] Traceback (most recent call last):
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]   File "/home/aeon/Evennia/evennia/evennia/commands/cmdsethandler.py", line 160, in import_cmdset
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]     module = import_module(modpath, package="evennia")
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]   File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]     __import__(name)
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]   File "/home/aeon/Evennia/Apendixie/commands/default_cmdsets.py", line 19, in <module>
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]     from typeclasses.doors import CmdOpen, CmdOpenCloseDoor
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]   File "/home/aeon/Evennia/Apendixie/typeclasses/doors.py", line 31, in <module>
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]     class SealedDoor(typeclasses.objects.DefaultObject):
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] NameError: name 'typeclasses' is not defined
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE]
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] Compile/Run error when loading cmdset 'commands.default_cmdsets.CharacterCmdSet'.",
2017-02-09 00:18:49+0000 [AMPProtocol,0,127.0.0.1] [EE] (Traceback was logged 2017-02-09 00:18:49-0000)
```


Alas, the report is pointless unless you point out what the error was in your code and exactly where it was defined. 
Okay sorry,  I'm just trying to figure that out myself as well.

 I mean, I can see it was a missing `typeclass` argument at line 31 in doors.py, but I need to know at which level (global, in a method) it was, and at which point it was called.
I'm dealing with a small flurry of errors on account of being interrupted coding this morning and stopping abruptly.  Each time I fix one I get another.  In each case the server restarts but fails to load any commands.

So far the errors have included:
* Trying to import a non-existent local module into a typeclass module 
* one more encoding error for (apostrophe again)
* simple syntax error in a variable assignment in a class defining a new command in a local module.

I have pasted my code from the last example below:

The error occurs at :         typeclass = typeclasses.

this is in MyGame/commands/building.py

`class CmdPaintDoor(COMMAND_DEFAULT_CLASS):
    """
    create a 'sealed door' in a particular direction. The sealed door doesn't
    function as a door but is a placeholder for later placing a real door
    leading to a new node.

    Usage:
      @door <direction> [= door_desc]

    """

    key = "@door"
    aliases = "@paint"
    locks = "cmd: perm(tunnel) or perm(Builders)"
    help_category = "Building"

    # store the direction, full name and its opposite
    directions = {"n": ("north", "s"),
                  "ne": ("northeast", "sw"),
                  "e": ("east", "w"),
                  "se": ("southeast", "nw"),
                  "s": ("south", "n"),
                  "sw": ("southwest", "ne"),
                  "w": ("west", "e"),
                  "nw": ("northwest", "se"),
                  "u": ("up", "d"),
                  "d": ("down", "u"),
                  "i": ("in", "o"),
                  "o": ("out", "i")}

    def func(self):
        "Implements the @door command"
        if not self.args or not self.lhs:
            string = ("Usage: @door <direction>")
            self.caller.msg(string)
            return
        # ... from tunnel
        if self.lhs not in self.directions:
            string = ("@door can only understand the following directions:"
                      "%s.") % ",".join(sorted(self.directions.keys()))
            self.caller.msg(string)
            return
        # retrieve all input and parse it
        dirshort = self.lhs  # n, s, e, w, etc
        dirlong, dirback = self.directions[dirshort]

        doorname = "%s door" % dirlong
        # not sure if I'll ever use the dirback
        dirback

        if self.rhs:
            dooralias = doorname
            doorname = self.rhs  # this may include aliases; that's fine.

        # Create an object. Locked to the wall with name and alias set and one
        # property - direction. Can I set a description too?

        typeclass = typeclasses.

        new_door = create.create_object(typeclass, name, location,
                                       aliases = dooralias,
                                       locks = lockstring,
                                       report_to = caller)
`
You have a period after typeclasses without anything following it, which is probably a syntax error. Also, unless you've defined typeclasses elsewhere, that'll be a name error since it's an unknown variable. Was that line supposed to be a string that matches the typeclass path you want to use, something like `typeclass = "typeclasses.exits.Exit"`?
Yes definitely. That was the error.  I'm just posting these to provide examples of the kinds of errors which prevent evennia from functioning but still allows it to load. At this point  I have yet to find an error that didn't allow me to reload --- and instead simply blocks the loading of all cmdsets.
Oh, I see what you're saying. You can get around blocking the loading of cmdsets by putting all the imports for your commands in try/except blocks that print out the errors with something like `traceback.print_exc()`. I do that just for convenience so if I make a silly mistake and have a syntax error in a module it only breaks those particular commands, not everything else.
Oh wow. That makes sense.  I'm not sure then whether this issue is of any use any more. I suppose Griatch will close it if not though.
Still, wouldn't it make more sense to have Ennia be prepared for this kind of situation?
 if you make a syntax error and reload, there's virtually no way around it, except for shutting down the server.  At least as far as I'm concerned, even fixing the problem doesn't seem to work (Evennia says it keeps on trying, but it feels like it doesn't :D )

Resolving patch:
diff --git a/evennia/commands/cmdsethandler.py b/evennia/commands/cmdsethandler.py
--- a/evennia/commands/cmdsethandler.py
+++ b/evennia/commands/cmdsethandler.py
@@ -351,6 +351,17 @@ def update(self, init_mode=False):
                     elif path:
                         cmdset = self._import_cmdset(path)
                         if cmdset:
+                            if cmdset.key == '_CMDSET_ERROR':
+                                # If a cmdset fails to load, check if we have a fallback path to use
+                                fallback_path = settings.CMDSET_FALLBACKS.get(path, None)
+                                if fallback_path:
+                                    logger.log_err("Error encountered for cmdset at path %s. Replacing with: %s" % (
+                                        path, fallback_path))
+                                    cmdset = self._import_cmdset(fallback_path)
+                                # If no cmdset is returned from the fallback, we can't go further
+                                if not cmdset:
+                                    logger.log_err("Fallback path '%s' failed to generate a cmdset." % fallback_path)
+                                    continue
                             cmdset.permanent = cmdset.key != '_CMDSET_ERROR'
                             self.cmdset_stack.append(cmdset)
 
diff --git a/evennia/settings_default.py b/evennia/settings_default.py
--- a/evennia/settings_default.py
+++ b/evennia/settings_default.py
@@ -348,6 +348,12 @@
 CMDSET_PLAYER = "commands.default_cmdsets.PlayerCmdSet"
 # Location to search for cmdsets if full path not given
 CMDSET_PATHS = ["commands", "evennia", "contribs"]
+# Fallbacks for cmdset paths that fail to load. Note that if you change the path for your default cmdsets,
+# you will also need to copy CMDSET_FALLBACKS after your change in your settings file for it to detect the change.
+CMDSET_FALLBACKS = {CMDSET_CHARACTER: 'evennia.commands.default.cmdset_character.CharacterCmdSet',
+                    CMDSET_PLAYER: 'evennia.commands.default.cmdset_player.PlayerCmdSet',
+                    CMDSET_SESSION: 'evennia.commands.default.cmdset_session.SessionCmdSet',
+                    CMDSET_UNLOGGEDIN: 'evennia.commands.default.cmdset_unloggedin.UnloggedinCmdSet'}
 # Parent class for all default commands. Changing this class will
 # modify all default commands, so do so carefully.
 COMMAND_DEFAULT_CLASS = "evennia.commands.default.muxcommand.MuxCommand"


Test patch: