Repository: evennia/evennia

Issue:
Bug: Issues with ANSIString
#### Brief summary of issue

This was originally brought up by @tekmunkey [here](https://github.com/evennia/evennia/pull/1386). The `ANSIString.center` is failing. 


    Using @py, 3 arguments gets:

    >>> from evennia.utils.ansi import ANSIString; teststr = ANSIString("|h|rMy testing string|n");self.msg(teststr.center(78,"-",2))
    TypeError: wrapped() takes at most 3 arguments (4 given)

    Using @py, 2 arguments gets:

    >>> from evennia.utils.ansi import ANSIString; teststr = ANSIString("|h|rMy testing string|n");self.msg(teststr.center(78,"-"))
    Done (use self.msg() if you want to catch output)


#### Error output / Expected result of feature

See above.

#### Extra information, such as Evennia revision/repo/branch, operating system and ideas for how to solve / implement:

It is possible that The latter issue may be an error with the outputhandler and a direct ANSIString - it may not identify as a "string" and thus mix up the outputfunc which in the case of a non-string should have the look `(name, (args), {kwargs})` (where the various parts are optional). 


Discussion:
@tekmunkey @Griatch 

The exception being thrown is a documentation issue. The code is working as expected-- the `difference` argument is dynamically generated by the function decorator `_spacing_preflight` to account for the offset of ANSI codes added to the string. ANSIString is supposed to be used identically to unicode save for one particularly notable exception (there may be others, perhaps something with regex, but I'm unaware of them):

The use of string.join on an ANSIString results in bogus output, due to string.join being implemented in Python C code and only checking if the target object is of unicode type before proceeding. Since ANSIString is a subclass of unicode, it gets handled at the raw level and its internal tables are dropped. You have to use ANSIString.join, or first convert the ANSIString to a string. I should have documented this and did not. So let's not consider this issue closed until I've made the necessary documentation adjustments.

I suspect the issue with .msg() not sending the right data may actually be due to this quirk of its behavior. I'm looking into it now.

Resolving patch:
diff --git a/evennia/server/sessionhandler.py b/evennia/server/sessionhandler.py
--- a/evennia/server/sessionhandler.py
+++ b/evennia/server/sessionhandler.py
@@ -201,7 +201,12 @@ def _validate(data):
                 if _INLINEFUNC_ENABLED and not raw and isinstance(self, ServerSessionHandler):
                     # only parse inlinefuncs on the outgoing path (sessionhandler->)
                     data = parse_inlinefunc(data, strip=strip_inlinefunc, session=session)
-                return data
+                # At this point the object is certainly the right encoding, but may still be a unicode object--
+                # to_str does not actually force objects to become bytestrings.
+                # If the unicode object is a subclass of unicode, such as ANSIString, this can cause a problem,
+                # as special behavior for that class will still be in play. Since we're now transferring raw data,
+                # we must now force this to be a proper bytestring.
+                return str(data)
             elif hasattr(data, "id") and hasattr(data, "db_date_created") \
                     and hasattr(data, '__dbclass__'):
                 # convert database-object to their string representation.
diff --git a/evennia/utils/ansi.py b/evennia/utils/ansi.py
--- a/evennia/utils/ansi.py
+++ b/evennia/utils/ansi.py
@@ -83,6 +83,7 @@ class ANSIParser(object):
     We also allow to escape colour codes
     by prepending with a \ for xterm256,
     an extra | for Merc-style codes
+
     """
 
     def sub_ansi(self, ansimatch):
@@ -461,7 +462,7 @@ def do_xterm256(part):
         (r'\{\[=[a-z]', ""),
         (r'\|=[a-z]', ""),
         (r'\|\[=[a-z]', ""),
-        ]
+    ]
 
     mxp_re = r'\|lc(.*?)\|lt(.*?)\|le'
 
@@ -535,6 +536,7 @@ def strip_raw_ansi(string, parser=ANSI_PARSER):
 
     Returns:
         string (str): the stripped string.
+
     """
     return parser.strip_raw_codes(string)
 
@@ -551,13 +553,6 @@ def raw(string):
     return string.replace('{', '{{').replace('|', '||')
 
 
-def group(lst, n):
-    for i in range(0, len(lst), n):
-        val = lst[i:i+n]
-        if len(val) == n:
-            yield tuple(val)
-
-
 def _spacing_preflight(func):
     """
     This wrapper function is used to do some preflight checks on
@@ -571,10 +566,10 @@ def wrapped(self, width, fillchar=None):
             raise TypeError("must be char, not %s" % type(fillchar))
         if not isinstance(width, int):
             raise TypeError("integer argument expected, got %s" % type(width))
-        difference = width - len(self)
-        if difference <= 0:
+        _difference = width - len(self)
+        if _difference <= 0:
             return self
-        return func(self, width, fillchar, difference)
+        return func(self, width, fillchar, _difference)
     return wrapped
 
 
@@ -661,20 +656,30 @@ def __init__(cls, *args, **kwargs):
 
 class ANSIString(with_metaclass(ANSIMeta, unicode)):
     """
-    String-like object that is aware of ANSI codes.
+    Unicode-like object that is aware of ANSI codes.
+
+    This class can be used nearly identically to unicode, in that it will
+    report string length, handle slices, etc, much like a unicode or
+    string object would. The methods should be used identically as unicode
+    methods are.
+
+    There is at least one exception to this (and there may be more, though
+    they have not come up yet). When using ''.join() or u''.join() on an
+    ANSIString, color information will get lost. You must use
+    ANSIString('').join() to preserve color information.
 
-    This isn't especially efficient, as it doesn't really have an
+    This implementation isn't perfectly clean, as it doesn't really have an
     understanding of what the codes mean in order to eliminate
-    redundant characters. This could be made as an enhancement to ANSI_PARSER.
+    redundant characters-- though cleaning up the strings might end up being
+    inefficient and slow without some C code when dealing with larger values.
+    Such enhancements could be made as an enhancement to ANSI_PARSER
+    if needed, however.
 
     If one is going to use ANSIString, one should generally avoid converting
     away from it until one is about to send information on the wire. This is
     because escape sequences in the string may otherwise already be decoded,
     and taken literally the second time around.
 
-    Please refer to the Metaclass, ANSIMeta, which is used to apply wrappers
-    for several of the methods that need not be defined directly here.
-
     """
 
     def __new__(cls, *args, **kwargs):
@@ -922,6 +927,9 @@ def clean(self):
         """
         Return a unicode object without the ANSI escapes.
 
+        Returns:
+            clean_string (unicode): A unicode object with no ANSI escapes.
+
         """
         return self._clean_string
 
@@ -929,20 +937,31 @@ def raw(self):
         """
         Return a unicode object with the ANSI escapes.
 
+        Returns:
+            raw (unicode): A unicode object with the raw ANSI escape sequences.
+
         """
         return self._raw_string
 
     def partition(self, sep, reverse=False):
         """
-        Similar to split, but always creates a tuple with three items:
-
-        1. The part before the separator
-        2. The separator itself.
-        3. The part after.
+        Splits once into three sections (with the separator being the middle section)
 
         We use the same techniques we used in split() to make sure each are
         colored.
 
+        Args:
+            sep (str): The separator to split the string on.
+            reverse (boolean): Whether to split the string on the last
+                occurrence of the separator rather than the first.
+        Returns:
+            result (tuple):
+               prefix (ANSIString): The part of the string before the
+                   separator
+               sep (ANSIString): The separator itself
+               postfix (ANSIString): The part of the string after the
+                   separator.
+
         """
         if hasattr(sep, '_clean_string'):
             sep = sep.clean()
@@ -1032,12 +1051,26 @@ def __rmul__(self, other):
 
     def split(self, by=None, maxsplit=-1):
         """
+        Splits a string based on a separator.
+
         Stolen from PyPy's pure Python string implementation, tweaked for
         ANSIString.
 
         PyPy is distributed under the MIT licence.
         http://opensource.org/licenses/MIT
 
+        Args:
+            by (str): A string to search for which will be used to split
+                the string. For instance, ',' for 'Hello,world' would
+                result in ['Hello', 'world']
+            maxsplit (int): The maximum number of times to split the string.
+                For example, a maxsplit of 2 with a by of ',' on the string
+                'Hello,world,test,string' would result in
+                ['Hello', 'world', 'test,string']
+        Returns:
+            result (list of ANSIStrings): A list of ANSIStrings derived from
+                this string.
+
         """
         drop_spaces = by is None
         if drop_spaces:
@@ -1065,12 +1098,27 @@ def split(self, by=None, maxsplit=-1):
 
     def rsplit(self, by=None, maxsplit=-1):
         """
+        Like split, but starts from the end of the string rather than the
+        beginning.
+
         Stolen from PyPy's pure Python string implementation, tweaked for
         ANSIString.
 
         PyPy is distributed under the MIT licence.
         http://opensource.org/licenses/MIT
 
+        Args:
+            by (str): A string to search for which will be used to split
+                the string. For instance, ',' for 'Hello,world' would
+                result in ['Hello', 'world']
+            maxsplit (int): The maximum number of times to split the string.
+                For example, a maxsplit of 2 with a by of ',' on the string
+                'Hello,world,test,string' would result in
+                ['Hello,world', 'test', 'string']
+        Returns:
+            result (list of ANSIStrings): A list of ANSIStrings derived from
+                this string.
+
         """
         res = []
         end = len(self)
@@ -1099,6 +1147,15 @@ def rsplit(self, by=None, maxsplit=-1):
     def strip(self, chars=None):
         """
         Strip from both ends, taking ANSI markers into account.
+
+        Args:
+            chars (str, optional): A string containing individual characters
+                to strip off of both ends of the string. By default, any blank
+                spaces are trimmed.
+        Returns:
+            result (ANSIString): A new ANSIString with the ends trimmed of the
+                relevant characters.
+
         """
         clean = self._clean_string
         raw = self._raw_string
@@ -1136,6 +1193,15 @@ def strip(self, chars=None):
     def lstrip(self, chars=None):
         """
         Strip from the left, taking ANSI markers into account.
+
+        Args:
+            chars (str, optional): A string containing individual characters
+                to strip off of the left end of the string. By default, any
+                blank spaces are trimmed.
+        Returns:
+            result (ANSIString): A new ANSIString with the left end trimmed of
+                the relevant characters.
+
         """
         clean = self._clean_string
         raw = self._raw_string
@@ -1160,6 +1226,15 @@ def lstrip(self, chars=None):
     def rstrip(self, chars=None):
         """
         Strip from the right, taking ANSI markers into account.
+
+        Args:
+            chars (str, optional): A string containing individual characters
+                to strip off of the right end of the string. By default, any
+                blank spaces are trimmed.
+        Returns:
+            result (ANSIString): A new ANSIString with the right end trimmed of
+                the relevant characters.
+
         """
         clean = self._clean_string
         raw = self._raw_string
@@ -1180,7 +1255,22 @@ def rstrip(self, chars=None):
 
     def join(self, iterable):
         """
-        Joins together strings in an iterable.
+        Joins together strings in an iterable, using this string between each
+        one.
+
+        NOTE: This should always be used for joining strings when ANSIStrings
+            are involved. Otherwise color information will be discarded by
+            python, due to details in the C implementation of unicode strings.
+
+        Args:
+            iterable (list of strings): A list of strings to join together
+        Returns:
+            result (ANSIString): A single string with all of the iterable's
+                contents concatenated, with this string between each. For
+                example:
+                    ANSIString(', ').join(['up', 'right', 'left', 'down'])
+                ...Would return:
+                    ANSIString('up, right, left, down')
 
         """
         result = ANSIString('')
@@ -1222,30 +1312,53 @@ def _filler(self, char, amount):
             raw_string, clean_string=line, char_indexes=char_indexes,
             code_indexes=code_indexes)
 
+    # The following methods should not be called with the '_difference' argument explicitly. This is
+    # data provided by the wrapper _spacing_preflight.
     @_spacing_preflight
-    def center(self, width, fillchar, difference):
+    def center(self, width, fillchar, _difference):
         """
         Center some text with some spaces padding both sides.
 
+        Args:
+            width (int): The target width of the output string.
+            fillchar (str): A single character string to pad the output string
+                with.
+        Returns:
+            result (ANSIString): A string padded on both ends with fillchar.
+
         """
-        remainder = difference % 2
-        difference /= 2
-        spacing = self._filler(fillchar, difference)
+        remainder = _difference % 2
+        _difference /= 2
+        spacing = self._filler(fillchar, _difference)
         result = spacing + self + spacing + self._filler(fillchar, remainder)
         return result
 
     @_spacing_preflight
-    def ljust(self, width, fillchar, difference):
+    def ljust(self, width, fillchar, _difference):
         """
         Left justify some text.
 
+        Args:
+            width (int): The target width of the output string.
+            fillchar (str): A single character string to pad the output string
+                with.
+        Returns:
+            result (ANSIString): A string padded on the right with fillchar.
+
         """
-        return self + self._filler(fillchar, difference)
+        return self + self._filler(fillchar, _difference)
 
     @_spacing_preflight
-    def rjust(self, width, fillchar, difference):
+    def rjust(self, width, fillchar, _difference):
         """
         Right justify some text.
 
+        Args:
+            width (int): The target width of the output string.
+            fillchar (str): A single character string to pad the output string
+                with.
+        Returns:
+            result (ANSIString): A string padded on the left with fillchar.
+
         """
-        return self._filler(fillchar, difference) + self
+        return self._filler(fillchar, _difference) + self


Test patch: