Python で業務システムからエクスポートした Shift-JIS とされているファイルを処理する際には cp932 として扱う方がよさげという話

## 概要

Pythonで業務システムからエクスポートしたShift-JISとされているファイルを処理する際、エンコーディングとしてcp932を使用することが推奨されます。これは、Shift-JISとcp932の間には微妙な違いがあり、特に日本語の一部の文字（一部の記号や特殊文字など）がShift-JISではなくcp932でしか正しく表現できないためです。そのため、エラーを避けるためには、Pythonでこれらのファイルを扱う際にはcp932として処理することが望ましいです。

`㈱`という特殊な文字を`shift_jis`エンコーディングで処理しようとすると、エラーが発生します。これは、`shift_jis`がこの特殊な文字をサポートしていないためです。このような問題を避けるためには、`cp932`エンコーディングの使用を検討してみてください。

```python
import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('shift_jis')
df
```

```text
UnicodeEncodeError: 'shift_jis' codec can't encode character '\u3231' in position 0: illegal multibyte sequence
```

本記事にて、エラーとなるコードのサンプルと同様のエラーが発生する文字を記載します。

## エラーとなるコードサンプル

### ファイルとして書き込む場合

`shift_jis`ではエラーとなってしまう。

```python
# Importing Required Library
import os

# Creating test.csv with data to be written
data = """
col_1
㈱
""".strip()

# Creating and Writing Data to test.csv file in dbfs/test folder
with open("/dbfs/test.csv", "w", encoding='shift_jis') as file:
    file.write(data)
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/15d82c47-1b16-2518-9ad7-7eba87c9f18a.png)

`cp932`に変更すると正常終了する。

```python
# Importing Required Library
import os

# Creating test.csv with data to be written
data = """
col_1
㈱
""".strip()

# Creating and Writing Data to test.csv file in dbfs/test folder
with open("/dbfs/test.csv", "w", encoding='cp932') as file:
    file.write(data)
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/bf222bff-c1c7-e74b-a82e-387eaf13f1aa.png)

### Pandas データフレームを作成する場合

`shift_jis`ではエラーとなってしまう。

```python
import pandas as pd

df = pd.read_csv("/dbfs/test.csv", encoding='shift_jis')
df
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/c573e53f-29f3-90f1-f7c1-0bdb2a87a58f.png)

`cp932`に変更すると正常終了する。

```python
import pandas as pd

df = pd.read_csv("/dbfs/test.csv", encoding='cp932')
df
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/690caeb4-eefa-2c5e-99f3-98c4cc7dc7b5.png)

### Pandas データフレームにてカラムを encode する場合

`shift_jis`ではエラーとなってしまう。

```python
import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('shift_jis')
df
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/1c5b8309-ffe0-8ac3-12b4-9273ef7ba8fd.png)

`cp932`に変更すると正常終了する。

```python
import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('cp932')
df
```

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/24031/127c732c-0a73-4981-16cd-4607b853e427.png)

## 同様のエラーが発生する文字列

下記の文字が含まれている場合には、`shift_jis`として扱う場合にはエラーとなります。

| col_1 | col_1_encoded |
| ----- | ------------- |
| ①     | h0A=          |
| ②     | h0E=          |
| ③     | h0I=          |
| ④     | h0M=          |
| ⑤     | h0Q=          |
| ⑥     | h0U=          |
| ⑦     | h0Y=          |
| ⑧     | h0c=          |
| ⑨     | h0g=          |
| ⑩     | h0k=          |
| ⑪     | h0o=          |
| ⑫     | h0s=          |
| ⑬     | h0w=          |
| ⑭     | h00=          |
| ⑮     | h04=          |
| ⑯     | h08=          |
| ⑰     | h1A=          |
| ⑱     | h1E=          |
| ⑲     | h1I=          |
| ⑳     | h1M=          |
| Ⅰ     | h1Q=          |
| Ⅱ     | h1U=          |
| Ⅲ     | h1Y=          |
| Ⅳ     | h1c=          |
| Ⅴ     | h1g=          |
| Ⅵ     | h1k=          |
| Ⅶ     | h1o=          |
| Ⅷ     | h1s=          |
| Ⅸ     | h1w=          |
| Ⅹ     | h10=          |
| ⅰ     | 7u8=          |
| ⅱ     | 7vA=          |
| ⅲ     | 7vE=          |
| ⅳ     | 7vI=          |
| ⅴ     | 7vM=          |
| ⅵ     | 7vQ=          |
| ⅶ     | 7vU=          |
| ⅷ     | 7vY=          |
| ⅸ     | 7vc=          |
| ⅹ     | 7vg=          |
| ・    | gUU=          |
| ㍉    | h18=          |
| ㌔    | h2A=          |
| ㌢    | h2E=          |
| ㍍    | h2I=          |
| ㌘    | h2M=          |
| ㌧    | h2Q=          |
| ㌃    | h2U=          |
| ㌶    | h2Y=          |
| ㍑    | h2c=          |
| ㍗    | h2g=          |
| ㌍    | h2k=          |
| ㌦    | h2o=          |
| ㌣    | h2s=          |
| ㌫    | h2w=          |
| ㍊    | h20=          |
| ㌻    | h24=          |
| ㎜    | h28=          |
| ㎝    | h3A=          |
| ㎞    | h3E=          |
| ㎎    | h3I=          |
| ㎏    | h3M=          |
| ㏄    | h3Q=          |
| ㎡    | h3U=          |
| ㍻    | h34=          |
| 〝    | h4A=          |
| 〟    | h4E=          |
| №     | h4I=          |
| ㏍    | h4M=          |
| ℡     | h4Q=          |
| ㊤    | h4U=          |
| ㊥    | h4Y=          |
| ㊦    | h4c=          |
| ㊧    | h4g=          |
| ㊨    | h4k=          |
| ㈱    | h4o=          |
| ㈲    | h4s=          |
| ㈹    | h4w=          |
| ㍾    | h40=          |
| ㍽    | h44=          |
| ㍼    | h48=          |
| ≒     | geA=          |
| ≡     | gd8=          |
| ∫     | gec=          |
| ∮     | h5M=          |
| ∑     | h5Q=          |
| √     | geM=          |
| ⊥     | gds=          |
| ∠     | gdo=          |
| ⊿     | h5k=          |
| ∵     | geY=          |
| ∩     | gb8=          |
| ∪     | gb4=          |
| 亜    | iJ8=          |
| 唖    | iKA=          |
| 娃    | iKE=          |
| 阿    | iKI=          |
| 哀    | iKM=          |
| 愛    | iKQ=          |
| 挨    | iKU=          |
| 姶    | iKY=          |
| 逢    | iKc=          |
| 葵    | iKg=          |
| 茜    | iKk=          |
| 穐    | iKo=          |
| 悪    | iKs=          |
| 握    | iKw=          |
| 渥    | iK0=          |
| 旭    | iK4=          |
| 葦    | iK8=          |
| ￢    | gco=          |
| ￤    | 7vo=          |
| ＇    | 7vs=          |
| ＂    | 7vw=          |
| ㈱    | h4o=          |
| №     | h4I=          |
| ℡     | h4Q=          |
| ∵     | geY=          |
| 纊    | 7UA=          |
| 褜    | 7UE=          |
| 鍈    | 7UI=          |
| 銈    | 7UM=          |
| 蓜    | 7UQ=          |
| 俉    | 7UU=          |
| 炻    | 7UY=          |
| 昱    | 7Uc=          |
| 棈    | 7Ug=          |
| 鋹    | 7Uk=          |
| 曻    | 7Uo=          |
| 彅    | 7Us=          |
| 丨    | 7Uw=          |
| 仡    | 7U0=          |
| 仼    | 7U4=          |
| 伀    | 7U8=          |
| 伃    | 7VA=          |
| 伹    | 7VE=          |
| 佖    | 7VI=          |
| 侒    | 7VM=          |
| 侊    | 7VQ=          |
| 侚    | 7VU=          |
| 侔    | 7VY=          |
| 俍    | 7Vc=          |
| 偀    | 7Vg=          |
| 倢    | 7Vk=          |
| 俿    | 7Vo=          |
| 倞    | 7Vs=          |
| 偆    | 7Vw=          |
| 偰    | 7V0=          |
| 偂    | 7V4=          |
| 傔    | 7V8=          |
| 僴    | 7WA=          |
| 僘    | 7WE=          |
| 兊    | 7WI=          |
| 兤    | 7WM=          |
| 冝    | 7WQ=          |
| 冾    | 7WU=          |
| 凬    | 7WY=          |
| 刕    | 7Wc=          |
| 劜    | 7Wg=          |
| 劦    | 7Wk=          |
| 勀    | 7Wo=          |
| 勛    | 7Ws=          |
| 匀    | 7Ww=          |
| 匇    | 7W0=          |
| 匤    | 7W4=          |
| 卲    | 7W8=          |
| 厓    | 7XA=          |
| 厲    | 7XE=          |
| 叝    | 7XI=          |
| 﨎    | 7XM=          |
| 咜    | 7XQ=          |
| 咊    | 7XU=          |
| 咩    | 7XY=          |
| 哿    | 7Xc=          |
| 喆    | 7Xg=          |
| 坙    | 7Xk=          |
| 坥    | 7Xo=          |
| 垬    | 7Xs=          |
| 埈    | 7Xw=          |
| 埇    | 7X0=          |
| 﨏    | 7X4=          |
| 塚    | 7YA=          |
| 增    | 7YE=          |
| 墲    | 7YI=          |
| 夋    | 7YM=          |
| 奓    | 7YQ=          |
| 奛    | 7YU=          |
| 奝    | 7YY=          |
| 奣    | 7Yc=          |
| 妤    | 7Yg=          |
| 妺    | 7Yk=          |
| 孖    | 7Yo=          |
| 寀    | 7Ys=          |
| 甯    | 7Yw=          |
| 寘    | 7Y0=          |
| 寬    | 7Y4=          |
| 尞    | 7Y8=          |
| 岦    | 7ZA=          |
| 岺    | 7ZE=          |
| 峵    | 7ZI=          |
| 崧    | 7ZM=          |
| 嵓    | 7ZQ=          |
| 﨑    | 7ZU=          |
| 嵂    | 7ZY=          |
| 嵭    | 7Zc=          |
| 嶸    | 7Zg=          |
| 嶹    | 7Zk=          |
| 巐    | 7Zo=          |
| 弡    | 7Zs=          |
| 弴    | 7Zw=          |
| 彧    | 7Z0=          |
| 德    | 7Z4=          |
| 忞    | 7Z8=          |
| 恝    | 7aA=          |
| 悅    | 7aE=          |
| 悊    | 7aI=          |
| 惞    | 7aM=          |
| 惕    | 7aQ=          |
| 愠    | 7aU=          |
| 惲    | 7aY=          |
| 愑    | 7ac=          |
| 愷    | 7ag=          |
| 愰    | 7ak=          |
| 憘    | 7ao=          |
| 戓    | 7as=          |
| 抦    | 7aw=          |
| 揵    | 7a0=          |
| 摠    | 7a4=          |
| 撝    | 7a8=          |
| 擎    | 7bA=          |
| 敎    | 7bE=          |
| 昀    | 7bI=          |
| 昕    | 7bM=          |
| 昻    | 7bQ=          |
| 昉    | 7bU=          |
| 昮    | 7bY=          |
| 昞    | 7bc=          |
| 昤    | 7bg=          |
| 晥    | 7bk=          |
| 晗    | 7bo=          |
| 晙    | 7bs=          |
| 晴    | 7bw=          |
| 晳    | 7b0=          |
| 暙    | 7b4=          |
| 暠    | 7b8=          |
| 暲    | 7cA=          |
| 暿    | 7cE=          |
| 曺    | 7cI=          |
| 朎    | 7cM=          |
| 朗    | 7cQ=          |
| 杦    | 7cU=          |
| 枻    | 7cY=          |
| 桒    | 7cc=          |
| 柀    | 7cg=          |
| 栁    | 7ck=          |
| 桄    | 7co=          |
| 棏    | 7cs=          |
| 﨓    | 7cw=          |
| 楨    | 7c0=          |
| 﨔    | 7c4=          |
| 榘    | 7c8=          |
| 槢    | 7dA=          |
| 樰    | 7dE=          |
| 橫    | 7dI=          |
| 橆    | 7dM=          |
| 橳    | 7dQ=          |
| 橾    | 7dU=          |
| 櫢    | 7dY=          |
| 櫤    | 7dc=          |
| 毖    | 7dg=          |
| 氿    | 7dk=          |
| 汜    | 7do=          |
| 沆    | 7ds=          |
| 汯    | 7dw=          |
| 泚    | 7d0=          |
| 洄    | 7d4=          |
| 涇    | 7d8=          |
| 浯    | 7eA=          |
| 涖    | 7eE=          |
| 涬    | 7eI=          |
| 淏    | 7eM=          |
| 淸    | 7eQ=          |
| 淲    | 7eU=          |
| 淼    | 7eY=          |
| 渹    | 7ec=          |
| 湜    | 7eg=          |
| 渧    | 7ek=          |
| 渼    | 7eo=          |
| 溿    | 7es=          |
| 澈    | 7ew=          |
| 澵    | 7e0=          |
| 濵    | 7e4=          |
| 瀅    | 7e8=          |
| 瀇    | 7fA=          |
| 瀨    | 7fE=          |
| 炅    | 7fI=          |
| 炫    | 7fM=          |
| 焏    | 7fQ=          |
| 焄    | 7fU=          |
| 煜    | 7fY=          |
| 煆    | 7fc=          |
| 煇    | 7fg=          |
| 凞    | 7fk=          |
| 燁    | 7fo=          |
| 燾    | 7fs=          |
| 犱    | 7fw=          |
| 犾    | 7kA=          |
| 猤    | 7kE=          |
| 猪    | 7kI=          |
| 獷    | 7kM=          |
| 玽    | 7kQ=          |
| 珉    | 7kU=          |
| 珖    | 7kY=          |
| 珣    | 7kc=          |
| 珒    | 7kg=          |
| 琇    | 7kk=          |
| 珵    | 7ko=          |
| 琦    | 7ks=          |
| 琪    | 7kw=          |
| 琩    | 7k0=          |
| 琮    | 7k4=          |
| 瑢    | 7k8=          |
| 璉    | 7lA=          |
| 璟    | 7lE=          |
| 甁    | 7lI=          |
| 畯    | 7lM=          |
| 皂    | 7lQ=          |
| 皜    | 7lU=          |
| 皞    | 7lY=          |
| 皛    | 7lc=          |
| 皦    | 7lg=          |
| 益    | 7lk=          |
| 睆    | 7lo=          |
| 劯    | 7ls=          |
| 砡    | 7lw=          |
| 硎    | 7l0=          |
| 硤    | 7l4=          |
| 硺    | 7l8=          |
| 礰    | 7mA=          |
| 礼    | 7mE=          |
| 神    | 7mI=          |
| 祥    | 7mM=          |
| 禔    | 7mQ=          |
| 福    | 7mU=          |
| 禛    | 7mY=          |
| 竑    | 7mc=          |
| 竧    | 7mg=          |
| 靖    | 7mk=          |
| 竫    | 7mo=          |
| 箞    | 7ms=          |
| 精    | 7mw=          |
| 絈    | 7m0=          |
| 絜    | 7m4=          |
| 綷    | 7m8=          |
| 綠    | 7nA=          |
| 緖    | 7nE=          |
| 繒    | 7nI=          |
| 罇    | 7nM=          |
| 羡    | 7nQ=          |
| 羽    | 7nU=          |
| 茁    | 7nY=          |
| 荢    | 7nc=          |
| 荿    | 7ng=          |
| 菇    | 7nk=          |
| 菶    | 7no=          |
| 葈    | 7ns=          |
| 蒴    | 7nw=          |
| 蕓    | 7n0=          |
| 蕙    | 7n4=          |
| 蕫    | 7oA=          |
| 﨟    | 7oE=          |
| 薰    | 7oI=          |
| 蘒    | 7oM=          |
| 﨡    | 7oQ=          |
| 蠇    | 7oU=          |
| 裵    | 7oY=          |
| 訒    | 7oc=          |
| 訷    | 7og=          |
| 詹    | 7ok=          |
| 誧    | 7oo=          |
| 誾    | 7os=          |
| 諟    | 7ow=          |
| 諸    | 7o0=          |
| 諶    | 7o4=          |
| 譓    | 7o8=          |
| 譿    | 7pA=          |
| 賰    | 7pE=          |
| 賴    | 7pI=          |
| 贒    | 7pM=          |
| 赶    | 7pQ=          |
| 﨣    | 7pU=          |
| 軏    | 7pY=          |
| 﨤    | 7pc=          |
| 逸    | 7pg=          |
| 遧    | 7pk=          |
| 郞    | 7po=          |
| 都    | 7ps=          |
| 鄕    | 7pw=          |
| 鄧    | 7p0=          |
| 釚    | 7p4=          |
| 釗    | 7p8=          |
| 釞    | 7qA=          |
| 釭    | 7qE=          |
| 釮    | 7qI=          |
| 釤    | 7qM=          |
| 釥    | 7qQ=          |
| 鈆    | 7qU=          |
| 鈐    | 7qY=          |
| 鈊    | 7qc=          |
| 鈺    | 7qg=          |
| 鉀    | 7qk=          |
| 鈼    | 7qo=          |
| 鉎    | 7qs=          |
| 鉙    | 7qw=          |
| 鉑    | 7q0=          |
| 鈹    | 7q4=          |
| 鉧    | 7q8=          |
| 銧    | 7rA=          |
| 鉷    | 7rE=          |
| 鉸    | 7rI=          |
| 鋧    | 7rM=          |
| 鋗    | 7rQ=          |
| 鋙    | 7rU=          |
| 鋐    | 7rY=          |
| 﨧    | 7rc=          |
| 鋕    | 7rg=          |
| 鋠    | 7rk=          |
| 鋓    | 7ro=          |
| 錥    | 7rs=          |
| 錡    | 7rw=          |
| 鋻    | 7r0=          |
| 﨨    | 7r4=          |
| 錞    | 7r8=          |
| 鋿    | 7sA=          |
| 錝    | 7sE=          |
| 錂    | 7sI=          |
| 鍰    | 7sM=          |
| 鍗    | 7sQ=          |
| 鎤    | 7sU=          |
| 鏆    | 7sY=          |
| 鏞    | 7sc=          |
| 鏸    | 7sg=          |
| 鐱    | 7sk=          |
| 鑅    | 7so=          |
| 鑈    | 7ss=          |
| 閒    | 7sw=          |
| 隆    | 7s0=          |
| 﨩    | 7s4=          |
| 隝    | 7s8=          |
| 隯    | 7tA=          |
| 霳    | 7tE=          |
| 霻    | 7tI=          |
| 靃    | 7tM=          |
| 靍    | 7tQ=          |
| 靏    | 7tU=          |
| 靑    | 7tY=          |
| 靕    | 7tc=          |
| 顗    | 7tg=          |
| 顥    | 7tk=          |
| 飯    | 7to=          |
| 飼    | 7ts=          |
| 餧    | 7tw=          |
| 館    | 7t0=          |
| 馞    | 7t4=          |
| 驎    | 7t8=          |
| 髙    | 7uA=          |
| 髜    | 7uE=          |
| 魵    | 7uI=          |
| 魲    | 7uM=          |
| 鮏    | 7uQ=          |
| 鮱    | 7uU=          |
| 鮻    | 7uY=          |
| 鰀    | 7uc=          |
| 鵰    | 7ug=          |
| 鵫    | 7uk=          |
| 鶴    | 7uo=          |
| 鸙    | 7us=          |
| 黑    | 7uw=          |

```pyton
data = [
    {"col_1": "①"},
    {"col_1": "②"},
    {"col_1": "③"},
    {"col_1": "④"},
    {"col_1": "⑤"},
    {"col_1": "⑥"},
    {"col_1": "⑦"},
    {"col_1": "⑧"},
    {"col_1": "⑨"},
    {"col_1": "⑩"},
    {"col_1": "⑪"},
    {"col_1": "⑫"},
    {"col_1": "⑬"},
    {"col_1": "⑭"},
    {"col_1": "⑮"},
    {"col_1": "⑯"},
    {"col_1": "⑰"},
    {"col_1": "⑱"},
    {"col_1": "⑲"},
    {"col_1": "⑳"},
    {"col_1": "Ⅰ"},
    {"col_1": "Ⅱ"},
    {"col_1": "Ⅲ"},
    {"col_1": "Ⅳ"},
    {"col_1": "Ⅴ"},
    {"col_1": "Ⅵ"},
    {"col_1": "Ⅶ"},
    {"col_1": "Ⅷ"},
    {"col_1": "Ⅸ"},
    {"col_1": "Ⅹ"},
    {"col_1": "ⅰ"},
    {"col_1": "ⅱ"},
    {"col_1": "ⅲ"},
    {"col_1": "ⅳ"},
    {"col_1": "ⅴ"},
    {"col_1": "ⅵ"},
    {"col_1": "ⅶ"},
    {"col_1": "ⅷ"},
    {"col_1": "ⅸ"},
    {"col_1": "ⅹ"},
    {"col_1": "・"},
    {"col_1": "㍉"},
    {"col_1": "㌔"},
    {"col_1": "㌢"},
    {"col_1": "㍍"},
    {"col_1": "㌘"},
    {"col_1": "㌧"},
    {"col_1": "㌃"},
    {"col_1": "㌶"},
    {"col_1": "㍑"},
    {"col_1": "㍗"},
    {"col_1": "㌍"},
    {"col_1": "㌦"},
    {"col_1": "㌣"},
    {"col_1": "㌫"},
    {"col_1": "㍊"},
    {"col_1": "㌻"},
    {"col_1": "㎜"},
    {"col_1": "㎝"},
    {"col_1": "㎞"},
    {"col_1": "㎎"},
    {"col_1": "㎏"},
    {"col_1": "㏄"},
    {"col_1": "㎡"},
    {"col_1": "㍻"},
    {"col_1": "〝"},
    {"col_1": "〟"},
    {"col_1": "№"},
    {"col_1": "㏍"},
    {"col_1": "℡"},
    {"col_1": "㊤"},
    {"col_1": "㊥"},
    {"col_1": "㊦"},
    {"col_1": "㊧"},
    {"col_1": "㊨"},
    {"col_1": "㈱"},
    {"col_1": "㈲"},
    {"col_1": "㈹"},
    {"col_1": "㍾"},
    {"col_1": "㍽"},
    {"col_1": "㍼"},
    {"col_1": "≒"},
    {"col_1": "≡"},
    {"col_1": "∫"},
    {"col_1": "∮"},
    {"col_1": "∑"},
    {"col_1": "√"},
    {"col_1": "⊥"},
    {"col_1": "∠"},
    {"col_1": "⊿"},
    {"col_1": "∵"},
    {"col_1": "∩"},
    {"col_1": "∪"},
    {"col_1": "亜"},
    {"col_1": "唖"},
    {"col_1": "娃"},
    {"col_1": "阿"},
    {"col_1": "哀"},
    {"col_1": "愛"},
    {"col_1": "挨"},
    {"col_1": "姶"},
    {"col_1": "逢"},
    {"col_1": "葵"},
    {"col_1": "茜"},
    {"col_1": "穐"},
    {"col_1": "悪"},
    {"col_1": "握"},
    {"col_1": "渥"},
    {"col_1": "旭"},
    {"col_1": "葦"},
    {"col_1": "￢"},
    {"col_1": "￤"},
    {"col_1": "＇"},
    {"col_1": "＂"},
    {"col_1": "㈱"},
    {"col_1": "№"},
    {"col_1": "℡"},
    {"col_1": "∵"},
    {"col_1": "纊"},
    {"col_1": "褜"},
    {"col_1": "鍈"},
    {"col_1": "銈"},
    {"col_1": "蓜"},
    {"col_1": "俉"},
    {"col_1": "炻"},
    {"col_1": "昱"},
    {"col_1": "棈"},
    {"col_1": "鋹"},
    {"col_1": "曻"},
    {"col_1": "彅"},
    {"col_1": "丨"},
    {"col_1": "仡"},
    {"col_1": "仼"},
    {"col_1": "伀"},
    {"col_1": "伃"},
    {"col_1": "伹"},
    {"col_1": "佖"},
    {"col_1": "侒"},
    {"col_1": "侊"},
    {"col_1": "侚"},
    {"col_1": "侔"},
    {"col_1": "俍"},
    {"col_1": "偀"},
    {"col_1": "倢"},
    {"col_1": "俿"},
    {"col_1": "倞"},
    {"col_1": "偆"},
    {"col_1": "偰"},
    {"col_1": "偂"},
    {"col_1": "傔"},
    {"col_1": "僴"},
    {"col_1": "僘"},
    {"col_1": "兊"},
    {"col_1": "兤"},
    {"col_1": "冝"},
    {"col_1": "冾"},
    {"col_1": "凬"},
    {"col_1": "刕"},
    {"col_1": "劜"},
    {"col_1": "劦"},
    {"col_1": "勀"},
    {"col_1": "勛"},
    {"col_1": "匀"},
    {"col_1": "匇"},
    {"col_1": "匤"},
    {"col_1": "卲"},
    {"col_1": "厓"},
    {"col_1": "厲"},
    {"col_1": "叝"},
    {"col_1": "﨎"},
    {"col_1": "咜"},
    {"col_1": "咊"},
    {"col_1": "咩"},
    {"col_1": "哿"},
    {"col_1": "喆"},
    {"col_1": "坙"},
    {"col_1": "坥"},
    {"col_1": "垬"},
    {"col_1": "埈"},
    {"col_1": "埇"},
    {"col_1": "﨏"},
    {"col_1": "塚"},
    {"col_1": "增"},
    {"col_1": "墲"},
    {"col_1": "夋"},
    {"col_1": "奓"},
    {"col_1": "奛"},
    {"col_1": "奝"},
    {"col_1": "奣"},
    {"col_1": "妤"},
    {"col_1": "妺"},
    {"col_1": "孖"},
    {"col_1": "寀"},
    {"col_1": "甯"},
    {"col_1": "寘"},
    {"col_1": "寬"},
    {"col_1": "尞"},
    {"col_1": "岦"},
    {"col_1": "岺"},
    {"col_1": "峵"},
    {"col_1": "崧"},
    {"col_1": "嵓"},
    {"col_1": "﨑"},
    {"col_1": "嵂"},
    {"col_1": "嵭"},
    {"col_1": "嶸"},
    {"col_1": "嶹"},
    {"col_1": "巐"},
    {"col_1": "弡"},
    {"col_1": "弴"},
    {"col_1": "彧"},
    {"col_1": "德"},
    {"col_1": "忞"},
    {"col_1": "恝"},
    {"col_1": "悅"},
    {"col_1": "悊"},
    {"col_1": "惞"},
    {"col_1": "惕"},
    {"col_1": "愠"},
    {"col_1": "惲"},
    {"col_1": "愑"},
    {"col_1": "愷"},
    {"col_1": "愰"},
    {"col_1": "憘"},
    {"col_1": "戓"},
    {"col_1": "抦"},
    {"col_1": "揵"},
    {"col_1": "摠"},
    {"col_1": "撝"},
    {"col_1": "擎"},
    {"col_1": "敎"},
    {"col_1": "昀"},
    {"col_1": "昕"},
    {"col_1": "昻"},
    {"col_1": "昉"},
    {"col_1": "昮"},
    {"col_1": "昞"},
    {"col_1": "昤"},
    {"col_1": "晥"},
    {"col_1": "晗"},
    {"col_1": "晙"},
    {"col_1": "晴"},
    {"col_1": "晳"},
    {"col_1": "暙"},
    {"col_1": "暠"},
    {"col_1": "暲"},
    {"col_1": "暿"},
    {"col_1": "曺"},
    {"col_1": "朎"},
    {"col_1": "朗"},
    {"col_1": "杦"},
    {"col_1": "枻"},
    {"col_1": "桒"},
    {"col_1": "柀"},
    {"col_1": "栁"},
    {"col_1": "桄"},
    {"col_1": "棏"},
    {"col_1": "﨓"},
    {"col_1": "楨"},
    {"col_1": "﨔"},
    {"col_1": "榘"},
    {"col_1": "槢"},
    {"col_1": "樰"},
    {"col_1": "橫"},
    {"col_1": "橆"},
    {"col_1": "橳"},
    {"col_1": "橾"},
    {"col_1": "櫢"},
    {"col_1": "櫤"},
    {"col_1": "毖"},
    {"col_1": "氿"},
    {"col_1": "汜"},
    {"col_1": "沆"},
    {"col_1": "汯"},
    {"col_1": "泚"},
    {"col_1": "洄"},
    {"col_1": "涇"},
    {"col_1": "浯"},
    {"col_1": "涖"},
    {"col_1": "涬"},
    {"col_1": "淏"},
    {"col_1": "淸"},
    {"col_1": "淲"},
    {"col_1": "淼"},
    {"col_1": "渹"},
    {"col_1": "湜"},
    {"col_1": "渧"},
    {"col_1": "渼"},
    {"col_1": "溿"},
    {"col_1": "澈"},
    {"col_1": "澵"},
    {"col_1": "濵"},
    {"col_1": "瀅"},
    {"col_1": "瀇"},
    {"col_1": "瀨"},
    {"col_1": "炅"},
    {"col_1": "炫"},
    {"col_1": "焏"},
    {"col_1": "焄"},
    {"col_1": "煜"},
    {"col_1": "煆"},
    {"col_1": "煇"},
    {"col_1": "凞"},
    {"col_1": "燁"},
    {"col_1": "燾"},
    {"col_1": "犱"},
    {"col_1": "犾"},
    {"col_1": "猤"},
    {"col_1": "猪"},
    {"col_1": "獷"},
    {"col_1": "玽"},
    {"col_1": "珉"},
    {"col_1": "珖"},
    {"col_1": "珣"},
    {"col_1": "珒"},
    {"col_1": "琇"},
    {"col_1": "珵"},
    {"col_1": "琦"},
    {"col_1": "琪"},
    {"col_1": "琩"},
    {"col_1": "琮"},
    {"col_1": "瑢"},
    {"col_1": "璉"},
    {"col_1": "璟"},
    {"col_1": "甁"},
    {"col_1": "畯"},
    {"col_1": "皂"},
    {"col_1": "皜"},
    {"col_1": "皞"},
    {"col_1": "皛"},
    {"col_1": "皦"},
    {"col_1": "益"},
    {"col_1": "睆"},
    {"col_1": "劯"},
    {"col_1": "砡"},
    {"col_1": "硎"},
    {"col_1": "硤"},
    {"col_1": "硺"},
    {"col_1": "礰"},
    {"col_1": "礼"},
    {"col_1": "神"},
    {"col_1": "祥"},
    {"col_1": "禔"},
    {"col_1": "福"},
    {"col_1": "禛"},
    {"col_1": "竑"},
    {"col_1": "竧"},
    {"col_1": "靖"},
    {"col_1": "竫"},
    {"col_1": "箞"},
    {"col_1": "精"},
    {"col_1": "絈"},
    {"col_1": "絜"},
    {"col_1": "綷"},
    {"col_1": "綠"},
    {"col_1": "緖"},
    {"col_1": "繒"},
    {"col_1": "罇"},
    {"col_1": "羡"},
    {"col_1": "羽"},
    {"col_1": "茁"},
    {"col_1": "荢"},
    {"col_1": "荿"},
    {"col_1": "菇"},
    {"col_1": "菶"},
    {"col_1": "葈"},
    {"col_1": "蒴"},
    {"col_1": "蕓"},
    {"col_1": "蕙"},
    {"col_1": "蕫"},
    {"col_1": "﨟"},
    {"col_1": "薰"},
    {"col_1": "蘒"},
    {"col_1": "﨡"},
    {"col_1": "蠇"},
    {"col_1": "裵"},
    {"col_1": "訒"},
    {"col_1": "訷"},
    {"col_1": "詹"},
    {"col_1": "誧"},
    {"col_1": "誾"},
    {"col_1": "諟"},
    {"col_1": "諸"},
    {"col_1": "諶"},
    {"col_1": "譓"},
    {"col_1": "譿"},
    {"col_1": "賰"},
    {"col_1": "賴"},
    {"col_1": "贒"},
    {"col_1": "赶"},
    {"col_1": "﨣"},
    {"col_1": "軏"},
    {"col_1": "﨤"},
    {"col_1": "逸"},
    {"col_1": "遧"},
    {"col_1": "郞"},
    {"col_1": "都"},
    {"col_1": "鄕"},
    {"col_1": "鄧"},
    {"col_1": "釚"},
    {"col_1": "釗"},
    {"col_1": "釞"},
    {"col_1": "釭"},
    {"col_1": "釮"},
    {"col_1": "釤"},
    {"col_1": "釥"},
    {"col_1": "鈆"},
    {"col_1": "鈐"},
    {"col_1": "鈊"},
    {"col_1": "鈺"},
    {"col_1": "鉀"},
    {"col_1": "鈼"},
    {"col_1": "鉎"},
    {"col_1": "鉙"},
    {"col_1": "鉑"},
    {"col_1": "鈹"},
    {"col_1": "鉧"},
    {"col_1": "銧"},
    {"col_1": "鉷"},
    {"col_1": "鉸"},
    {"col_1": "鋧"},
    {"col_1": "鋗"},
    {"col_1": "鋙"},
    {"col_1": "鋐"},
    {"col_1": "﨧"},
    {"col_1": "鋕"},
    {"col_1": "鋠"},
    {"col_1": "鋓"},
    {"col_1": "錥"},
    {"col_1": "錡"},
    {"col_1": "鋻"},
    {"col_1": "﨨"},
    {"col_1": "錞"},
    {"col_1": "鋿"},
    {"col_1": "錝"},
    {"col_1": "錂"},
    {"col_1": "鍰"},
    {"col_1": "鍗"},
    {"col_1": "鎤"},
    {"col_1": "鏆"},
    {"col_1": "鏞"},
    {"col_1": "鏸"},
    {"col_1": "鐱"},
    {"col_1": "鑅"},
    {"col_1": "鑈"},
    {"col_1": "閒"},
    {"col_1": "隆"},
    {"col_1": "﨩"},
    {"col_1": "隝"},
    {"col_1": "隯"},
    {"col_1": "霳"},
    {"col_1": "霻"},
    {"col_1": "靃"},
    {"col_1": "靍"},
    {"col_1": "靏"},
    {"col_1": "靑"},
    {"col_1": "靕"},
    {"col_1": "顗"},
    {"col_1": "顥"},
    {"col_1": "飯"},
    {"col_1": "飼"},
    {"col_1": "餧"},
    {"col_1": "館"},
    {"col_1": "馞"},
    {"col_1": "驎"},
    {"col_1": "髙"},
    {"col_1": "髜"},
    {"col_1": "魵"},
    {"col_1": "魲"},
    {"col_1": "鮏"},
    {"col_1": "鮱"},
    {"col_1": "鮻"},
    {"col_1": "鰀"},
    {"col_1": "鵰"},
    {"col_1": "鵫"},
    {"col_1": "鶴"},
    {"col_1": "鸙"},
    {"col_1": "黑"},
]

df = pd.DataFrame(data)
df['col_1_encoded'] = df['col_1'].str.encode('cp932')
df
```