Repository: pandas-dev/pandas

Issue:
string truncation when using Series.astype(str)
When converting a pandas Series object to type string using `astype(str)`, long strings are truncated to 64 characters silently.

Pandas version:  0.12
Numpy version:  1.7.1

``` python

In [1]: s = '0123456789' * 10

In [2]: tmp = np.array([s]).astype(str)
         tmp
Out[2]: array([ '0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789'], 
      dtype='|S100')

In [3]: tmp = pd.Series([s]).astype(str)
       tmp[0]
Out[3]: '0123456789012345678901234567890123456789012345678901234567890123'

In [4]: len(tmp[0])
Out[4]: 64
```



Discussion:
actually this looks like a numpy bug

```
In [1]: s = '0123456789' * 10

In [2]: tmp = np.array([s]).astype(str)

In [3]: s = Series([s])

In [4]: s.values
Out[4]: array([ '0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789'], dtype=object)

In [5]: s.values.astype(str)
Out[5]:
array(['0123456789012345678901234567890123456789012345678901234567890123'],
      dtype='|S64')
```

and `astype(str)` should just be converted to `astype(object)` internally (as str type is not a valid type)

if you need to do this then you can do

```
In [29]: s.values.astype('|S%d' % len(s[0]))
```

`str` must implcity do `S64` somewhere (in numpy)

maybe fixed in np master

https://github.com/numpy/numpy/pull/3270

i don't have the patience right now to recompile everything...anyone want to try this out on numpy master? i guess we leave it open until that numpy release


Resolving patch:
diff --git a/pandas/core/common.py b/pandas/core/common.py
--- a/pandas/core/common.py
+++ b/pandas/core/common.py
@@ -3,7 +3,6 @@
 """
 
 import re
-from datetime import datetime
 import codecs
 import csv
 
@@ -1628,7 +1627,7 @@ def _is_sequence(x):
 _ensure_object = algos.ensure_object
 
 
-def _astype_nansafe(arr, dtype, copy = True):
+def _astype_nansafe(arr, dtype, copy=True):
     """ return a view if copy is False """
     if not isinstance(dtype, np.dtype):
         dtype = np.dtype(dtype)
@@ -1659,6 +1658,8 @@ def _astype_nansafe(arr, dtype, copy = True):
     elif arr.dtype == np.object_ and np.issubdtype(dtype.type, np.integer):
         # work around NumPy brokenness, #1987
         return lib.astype_intsafe(arr.ravel(), dtype).reshape(arr.shape)
+    elif issubclass(dtype.type, compat.string_types):
+        return lib.astype_str(arr.ravel()).reshape(arr.shape)
 
     if copy:
         return arr.astype(dtype)
diff --git a/pandas/core/series.py b/pandas/core/series.py
--- a/pandas/core/series.py
+++ b/pandas/core/series.py
@@ -5,7 +5,6 @@
 # pylint: disable=E1101,E1103
 # pylint: disable=W0703,W0622,W0613,W0201
 
-from pandas import compat
 import operator
 from distutils.version import LooseVersion
 import types


Test patch:
diff --git a/pandas/tests/test_series.py b/pandas/tests/test_series.py
--- a/pandas/tests/test_series.py
+++ b/pandas/tests/test_series.py
@@ -4,6 +4,7 @@
 import os
 import operator
 import unittest
+import string
 
 import nose
 
@@ -2029,6 +2030,7 @@ def test_timedelta64_functions(self):
         expected = Series([timedelta(1)],dtype='timedelta64[ns]')
         assert_series_equal(result,expected)
 
+
     def test_sub_of_datetime_from_TimeSeries(self):
         from pandas.core import common as com
         from datetime import datetime
@@ -3354,6 +3356,19 @@ def test_astype_datetimes(self):
         s = s.astype('O')
         self.assert_(s.dtype == np.object_)
 
+    def test_astype_str(self):
+        # GH4405
+        digits = string.digits
+        s1 = Series([digits * 10, tm.rands(63), tm.rands(64),
+                    tm.rands(1000)])
+        s2 = Series([digits * 10, tm.rands(63), tm.rands(64), nan, 1.0])
+        types = (compat.text_type,) + (np.str_, np.unicode_)
+        for typ in types:
+            for s in (s1, s2):
+                res = s.astype(typ)
+                expec = s.map(compat.text_type)
+                assert_series_equal(res, expec)
+
     def test_map(self):
         index, data = tm.getMixedTypeDict()