UnicodeEncodeError: 'ascii' codec can't encode characters in position 18-20: ordinal not in range(128)

wsy: <p>I am using KNN to realize the prediction of Titanic on kaggle. I want to know how to solve this coding problem</p>
<h1>-<em>- coding: utf-8 -</em>-</h1>
<p>import numpy as np<br>
import pandas as pd<br>
import seaborn as sns<br>
import sys<br>
import importlib<br>
importlib.reload(sys)</p>
<p>test=pd.read_csv(“test.csv”,encoding =‘utf_8’)<br>
train=pd.read_csv(‘train.csv’,encoding =‘utf_8’)<br>
train[‘Age’]=train[‘Age’].map(lambda x: ‘child’ if x&lt;12 else ‘youth’ if x&lt;30 else ‘adlut’ if x&lt;60 else ‘old’ if x&lt;75 else ‘tooold’ if x&gt;=75 else ‘null’)<br>
train[‘SibSp’]=train[‘SibSp’].map(lambda x: ‘small’ if x&lt;1 else ‘middle’ if x&lt;3 else ‘large’)<br>
train[‘Parch’]=train[‘Parch’].map(lambda x: ‘small’ if x&lt;1 else ‘middle’ if x&lt;4 else ‘large’)<span class="hashtag">#根据小提琴图</span>，分成三个部分<br>
train[‘Fare’]=train[‘Fare’].map(lambda x:np.log(x+1))<span class="hashtag">#lambda这种是匿名函数</span>，+1是由于定义域所导致的<br>
train[‘Fare’]=train[‘Fare’].map(lambda x: ‘poor’ if x&lt;2.5 else ‘rich’)<br>
train[‘Cabin’]=train[‘Cabin’].map(lambda x:‘yes’ if type(x)==str else ‘no’)<br>
train.dropna(axis=0,inplace=True)<br>
labels= train[‘Survived’]<br>
features= train.drop([‘Survived’,‘PassengerId’,‘Name’,‘Ticket’],axis=1)<br>
features = pd.get_dummies(features)<br>
encoded = list(features.columns)<br>
print ("{} total features after one-hot encoding.".format(len(encoded)))<br>
<span class="hashtag">#对</span>’Age’,‘SibSp’，'Parch’特征进行分段分类<br>
test[‘Age’]=test[‘Age’].map(lambda x: ‘child’ if x&lt;12 else ‘youth’ if x&lt;30 else ‘adlut’ if x&lt;60 else ‘old’ if x&lt;75 else ‘tooold’ if x&gt;=75 else ‘null’)<br>
test[‘SibSp’]=test[‘SibSp’].map(lambda x: ‘small’ if x&lt;1 else ‘middle’ if x&lt;3 else ‘large’)<br>
test[‘Parch’]=test[‘Parch’].map(lambda x: ‘small’ if x&lt;1 else ‘middle’ if x&lt;4 else ‘large’)<br>
<span class="hashtag">#均值补齐</span>’Fare’特征值并作对数转换和分类<br>
test.Fare.fillna(test[‘Fare’].mean(), inplace=True)<br>
test[‘Fare’]=test[‘Fare’].map(lambda x:np.log(x+1))<br>
test[‘Fare’]=test[‘Fare’].map(lambda x: ‘poor’ if x&lt;2.5 else ‘rich’)<br>
<span class="hashtag">#按</span>’Cabin’是否缺损分类<br>
test[‘Cabin’]=test[‘Cabin’].map(lambda x:‘yes’ if type(x)==str else ‘no’)<br>
<span class="hashtag">#删除不需要的特征并进行独热编码</span><br>
#Id=test[‘PassengerId’]，好像是多余的<br>
test=test.drop([‘PassengerId’,‘Name’,‘Ticket’],axis=1)<br>
test=pd.get_dummies(test)<br>
encoded = list(test.columns)<br>
print ("{} total features after one-hot encoding.".format(len(encoded)))<br>
from sklearn.model_selection import GridSearchCV<br>
from sklearn.metrics import make_scorer<br>
from sklearn.metrics import accuracy_score,roc_auc_score<br>
from time import time<br>
from sklearn.neighbors import KNeighborsClassifier</p>
<p>def fit_model(alg,parameters):<br>
X=features<br>
y=labels  <span class="hashtag">#由于数据较少</span>，使用全部数据进行网格搜索<br>
#？这里是可以优化的地方，网格搜索<br>
scorer=make_scorer(roc_auc_score)  <span class="hashtag">#使用roc_auc_score作为评分标准</span><br>
grid = GridSearchCV(alg,parameters,scoring=scorer,cv=5)  <span class="hashtag">#使用网格搜索</span>，输入参数，实现自动调参<br>
start=time()  <span class="hashtag">#计时</span><br>
grid=grid.fit(X,y)  <span class="hashtag">#模型训练</span><br>
end=time()<br>
t=round(end-start,3)<br>
print (grid.best_params_)  <span class="hashtag">#输出最佳参数</span><br>
print (‘searching time for {} is {} s’.format(alg.<strong>class</strong>.<strong>name</strong>,t)) <span class="hashtag">#输出搜索时间</span><br>
return grid <span class="hashtag">#返回训练好的模型</span></p>
<p>alg5=KNeighborsClassifier(n_jobs=-1)</p>
<p>parameters5 = {‘n_neighbors’:range(2,10),‘leaf_size’:range(10,80,20)  }<br>
clf5=fit_model(alg5,parameters5)</p>

ptrblck: <p>This error is raised from a lib which tries to load the data and fails in the encoder.<br>
<a href="https://docs.python.org/2.7/howto/unicode.html">The Unicode Howto doc</a> might be helpful in solving the issue.<br>
Also, this issue seems to be raised by <code>pandas</code>, <code>sklearn</code> or another lib you are using, so you might get a better and faster answer on e.g. StackOverflow. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<p>PS: You can post code snippets by wrapping them into three backticks ```, which would make debugging easier.</p>

carlhyde: <p>On Windows, many editors assume the default ANSI encoding (CP1252 on US Windows) instead of UTF-8 if there is no byte order mark (BOM) character at the start of the file. Files store bytes, which means all unicode have to be encoded into bytes before they can be stored in a file. read_csv takes an encoding option to deal with files in different formats. So, you have to specify an encoding, such as utf-8.</p>
<p><code>df.to_csv('D:\panda.csv',sep='\t',encoding='utf-8')</code></p>
<p>If you don’t specify an encoding, then the encoding used by df.<a href="http://net-informations.com/ds/pd/tocsv.htm" rel="noopener nofollow ugc">tocsv</a> defaults to ascii in Python2, or utf-8 in Python3.</p>
<p>Also, you can encode a problematic series first then decode it back to utf-8.</p>
<p><code>df['column-name'] = df['column-name'].map(lambda x: x.encode('unicode-escape').decode('utf-8'))</code></p>
<p>This will also rectify the problem.</p>

Kang_Zhu: <p>change<br>
alg5=KNeighborsClassifier(n_jobs=-1)<br>
to<br>
alg5=KNeighborsClassifier(n_jobs=1)</p>