CurlでWatson Personality Insightsを使ってみる

Bluemix上のWatsonには、自然言語処理を活用したPersonality Insightsというサービスがあります。これは、適当な文章を与えると、その文章を分析して、著者に関するプロファイリングを行う、つまりパーソナリティ(性格)についての洞察(insights)を与えてくれるというものです。

# 準備

以下は準備してあるものとします。

* IBM ID (Bluemixアカウント)
* [cfコマンド](https://github.com/cloudfoundry/cli/releases)のインストール

# Personality-Insightsサービスの作成

```console
# cfの接続先としてBluemixを指定
$ cf api https://api.ng.bluemix.net

# cfにログイン
$ cf login -u <ユーザ名>

# Personality-Insightsサービスを確認
# サービスの名前とプランを確認します
$ cf marketplace | grep personality
personality_insights           standard*                                                                                                                                                                      The Watson Personality Insights derives insights from transactional and social media data to identify psychological traits

# サービスを作成(サービス名: pi)
# 「Standard」サービスはフリーではない、という注意書きがされています。
$ cf create-service personality_insights standard pi
Creating service instance pi in org <mail-address> / space <space> as <mail-address>...
OK

Attention: The plan `standard` of service `personality_insights` is not free.  The instance `pi` will incur a cost.  Contact your administrator if you think this is in error.

# 資格情報(サービスキー)を作成(資格情報名: pi-credentials)
$ cf create-service-key pi pi-credentials
Creating service key pi-credentials for service instance pi as <mail-address>...
OK

# 資格情報の内容を確認(username, password, urlを含むjsonが返ってきます)
$ cf service-key pi pi-credentials
Getting key pi-credentials for service instance pi as <mail-address>...

{
 "password": "<password>",
 "url": "<url>",
 "username": "<username>"
}
```

2015年8月19日時点、Personality-Insightsサービスは以下のような仕様的制約があります。

* 分析できる言語(Content-Language)はスペイン語(es)および英語(en)
* レスポンスの言語(Accept-Language)は英語(en)のみ
* 唯一のプラン(standard)では、毎月100回のAPIコールまでは無償、それ以降は1コールあたり63円。
* 以前はプロファイルAPIのほかに可視化API (プロファイル結果をグラフ上に描画するAPI)がったが、現在DEPRECATEDとなっている。ドキュメントによると将来的には削除する、とのこと。

# Personality-Insightsサービスの利用

それでは、curlを使ってサービスを利用してみましょう。日本語が通りませんので、[戦後70年の安倍総理大臣談話](http://japan.kantei.go.jp/97_abe/statement/201508/0814statement.html) ([コピー：statement-by-prime-minister-Abe-20150815.txt](https://gist.github.com/blackaplysia/8a6432424d81436187ea))を使ってみます。何度もAPIコールするのはもったいないですからファイル (personality-insights-by-watson-of-statement-by-prime-minister-Abe-20150815.json)に出力し、その後`json_pp`で整形してみましょう。

```console
$ curl -u "<username>:<password>" -X POST -H "Accept: application/json" -H "Content-Type: text/plain" --data-binary @statement-by-prime-minister-Abe-20150815.txt <url>/v2/profile > personality-insights-by-watson-of-statement-by-prime-minister-Abe-20150815.json

$ cat personality-insights-by-watson-of-statement-by-prime-minister-Abe-20150815.json | json_pp
{
   "word_count" : 1663,
   "processed_lang" : "en",
   "tree" : {
      "name" : "root",
      "children" : [
         {
            "name" : "Big 5 ",
            "children" : [
               {
                  "id" : "Openness_parent",
                  "children" : [
                     {
                        "sampling_error" : 0.0571496181,
                        "category" : "personality",
                        "name" : "Openness",
                        "percentage" : 0.985531881645297,
                        "id" : "Openness",
                        "children" : [
                           {
                              "id" : "Adventurousness",
                              "percentage" : 0.955298673768664,
                              "name" : "Adventurousness",
                              "sampling_error" : 0.0488113794,
                              "category" : "personality"
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.0998366943,
                              "percentage" : 0.592628952703858,
                              "name" : "Artistic interests",
                              "id" : "Artistic interests"
                           },
                           {
                              "sampling_error" : 0.0461219909,
                              "category" : "personality",
                              "name" : "Emotionality",
                              "percentage" : 0.251675129136317,
                              "id" : "Emotionality"
                           },
                           {
                              "sampling_error" : 0.0608295286,
                              "category" : "personality",
                              "id" : "Imagination",
                              "name" : "Imagination",
                              "percentage" : 0.930478668392966
                           },
                           {
                              "sampling_error" : 0.0532948824,
                              "category" : "personality",
                              "id" : "Intellect",
                              "percentage" : 0.980715015277147,
                              "name" : "Intellect"
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.0798135205,
                              "id" : "Liberalism",
                              "name" : "Authority-challenging",
                              "percentage" : 0.944000076764697
                           }
                        ]
                     },
                     {
                        "children" : [
                           {
                              "name" : "Achievement striving",
                              "percentage" : 0.474981825529316,
                              "id" : "Achievement striving",
                              "category" : "personality",
                              "sampling_error" : 0.0945453435
                           },
                           {
                              "id" : "Cautiousness",
                              "percentage" : 0.864293617215869,
                              "name" : "Cautiousness",
                              "sampling_error" : 0.0876421328,
                              "category" : "personality"
                           },
                           {
                              "sampling_error" : 0.0579122121,
                              "category" : "personality",
                              "name" : "Dutifulness",
                              "percentage" : 0.357662136741296,
                              "id" : "Dutifulness"
                           },
                           {
                              "sampling_error" : 0.0672540892,
                              "category" : "personality",
                              "name" : "Orderliness",
                              "percentage" : 0.221788688280043,
                              "id" : "Orderliness"
                           },
                           {
                              "name" : "Self-discipline",
                              "percentage" : 0.533917427198465,
                              "id" : "Self-discipline",
                              "category" : "personality",
                              "sampling_error" : 0.0448076016
                           },
                           {
                              "sampling_error" : 0.0880296304,
                              "category" : "personality",
                              "percentage" : 0.95135933929905,
                              "name" : "Self-efficacy",
                              "id" : "Self-efficacy"
                           }
                        ],
                        "id" : "Conscientiousness",
                        "percentage" : 0.757917285597684,
                        "name" : "Conscientiousness",
                        "sampling_error" : 0.0727038033,
                        "category" : "personality"
                     },
                     {
                        "category" : "personality",
                        "sampling_error" : 0.0535787226,
                        "percentage" : 0.268966555713819,
                        "name" : "Extraversion",
                        "id" : "Extraversion",
                        "children" : [
                           {
                              "sampling_error" : 0.0743274905,
                              "category" : "personality",
                              "name" : "Activity level",
                              "percentage" : 0.236498849668593,
                              "id" : "Activity level"
                           },
                           {
                              "percentage" : 0.147596241973124,
                              "name" : "Assertiveness",
                              "id" : "Assertiveness",
                              "category" : "personality",
                              "sampling_error" : 0.0792883861
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.0995465664,
                              "percentage" : 0.0680187007764542,
                              "name" : "Cheerfulness",
                              "id" : "Cheerfulness"
                           },
                           {
                              "percentage" : 0.0185447613645495,
                              "name" : "Excitement-seeking",
                              "id" : "Excitement-seeking",
                              "category" : "personality",
                              "sampling_error" : 0.0785121397
                           },
                           {
                              "id" : "Friendliness",
                              "percentage" : 0.320766187763908,
                              "name" : "Outgoing",
                              "sampling_error" : 0.0716222006,
                              "category" : "personality"
                           },
                           {
                              "sampling_error" : 0.0555084109,
                              "category" : "personality",
                              "name" : "Gregariousness",
                              "percentage" : 0.0475087296760301,
                              "id" : "Gregariousness"
                           }
                        ]
                     },
                     {
                        "percentage" : 0.202590080377138,
                        "name" : "Agreeableness",
                        "children" : [
                           {
                              "percentage" : 0.702744630547856,
                              "name" : "Altruism",
                              "id" : "Altruism",
                              "sampling_error" : 0.0675776853,
                              "category" : "personality"
                           },
                           {
                              "sampling_error" : 0.0764497065,
                              "category" : "personality",
                              "id" : "Cooperation",
                              "name" : "Cooperation",
                              "percentage" : 0.815783616461597
                           },
                           {
                              "id" : "Modesty",
                              "name" : "Modesty",
                              "percentage" : 0.378328509396603,
                              "sampling_error" : 0.0530798363,
                              "category" : "personality"
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.0600532331,
                              "id" : "Morality",
                              "name" : "Uncompromising",
                              "percentage" : 0.359536783227909
                           },
                           {
                              "id" : "Sympathy",
                              "name" : "Sympathy",
                              "percentage" : 1,
                              "category" : "personality",
                              "sampling_error" : 0.0926901829
                           },
                           {
                              "sampling_error" : 0.052704547,
                              "category" : "personality",
                              "id" : "Trust",
                              "percentage" : 0.589804489457293,
                              "name" : "Trust"
                           }
                        ],
                        "id" : "Agreeableness",
                        "category" : "personality",
                        "sampling_error" : 0.0927347177
                     },
                     {
                        "category" : "personality",
                        "sampling_error" : 0.0852718795,
                        "id" : "Neuroticism",
                        "children" : [
                           {
                              "sampling_error" : 0.089492093,
                              "category" : "personality",
                              "name" : "Fiery",
                              "percentage" : 0.301966706689488,
                              "id" : "Anger"
                           },
                           {
                              "name" : "Prone to worry",
                              "percentage" : 0.113762643009641,
                              "id" : "Anxiety",
                              "category" : "personality",
                              "sampling_error" : 0.0525488814
                           },
                           {
                              "sampling_error" : 0.0557466861,
                              "category" : "personality",
                              "id" : "Depression",
                              "name" : "Melancholy",
                              "percentage" : 0.154636060773852
                           },
                           {
                              "id" : "Immoderation",
                              "name" : "Immoderation",
                              "percentage" : 0.034017973230493,
                              "category" : "personality",
                              "sampling_error" : 0.0509011784
                           },
                           {
                              "sampling_error" : 0.0536273265,
                              "category" : "personality",
                              "id" : "Self-consciousness",
                              "name" : "Self-consciousness",
                              "percentage" : 0.024674440177167
                           },
                           {
                              "percentage" : 0.0191832698953367,
                              "name" : "Susceptible to stress",
                              "id" : "Vulnerability",
                              "category" : "personality",
                              "sampling_error" : 0.081406113
                           }
                        ],
                        "name" : "Emotional range",
                        "percentage" : 0.287004538323023
                     }
                  ],
                  "name" : "Openness",
                  "percentage" : 0.985531881645297,
                  "category" : "personality"
               }
            ],
            "id" : "personality"
         },
         {
            "name" : "Needs",
            "id" : "needs",
            "children" : [
               {
                  "category" : "needs",
                  "id" : "Challenge_parent",
                  "children" : [
                     {
                        "category" : "needs",
                        "sampling_error" : 0.0801473494,
                        "name" : "Challenge",
                        "percentage" : 0.987323190315372,
                        "id" : "Challenge"
                     },
                     {
                        "id" : "Closeness",
                        "percentage" : 0.82003316906521,
                        "name" : "Closeness",
                        "sampling_error" : 0.0791345764,
                        "category" : "needs"
                     },
                     {
                        "name" : "Curiosity",
                        "percentage" : 0.751109065890169,
                        "id" : "Curiosity",
                        "category" : "needs",
                        "sampling_error" : 0.1138235661
                     },
                     {
                        "sampling_error" : 0.1033336639,
                        "category" : "needs",
                        "id" : "Excitement",
                        "percentage" : 0.550839571680571,
                        "name" : "Excitement"
                     },
                     {
                        "id" : "Harmony",
                        "name" : "Harmony",
                        "percentage" : 0.809729719712853,
                        "category" : "needs",
                        "sampling_error" : 0.1027621665
                     },
                     {
                        "sampling_error" : 0.0932734664,
                        "category" : "needs",
                        "id" : "Ideal",
                        "percentage" : 0.710924997718907,
                        "name" : "Ideal"
                     },
                     {
                        "sampling_error" : 0.1382534011,
                        "category" : "needs",
                        "percentage" : 0.548307107545345,
                        "name" : "Liberty",
                        "id" : "Liberty"
                     },
                     {
                        "name" : "Love",
                        "percentage" : 0.827694039664942,
                        "id" : "Love",
                        "sampling_error" : 0.0945970322,
                        "category" : "needs"
                     },
                     {
                        "percentage" : 0.698890806447052,
                        "name" : "Practicality",
                        "id" : "Practicality",
                        "sampling_error" : 0.0824332482,
                        "category" : "needs"
                     },
                     {
                        "id" : "Self-expression",
                        "percentage" : 0.504330074355742,
                        "name" : "Self-expression",
                        "sampling_error" : 0.0773842253,
                        "category" : "needs"
                     },
                     {
                        "percentage" : 0.811629730319094,
                        "name" : "Stability",
                        "id" : "Stability",
                        "sampling_error" : 0.100767893,
                        "category" : "needs"
                     },
                     {
                        "percentage" : 0.898100266099327,
                        "name" : "Structure",
                        "id" : "Structure",
                        "sampling_error" : 0.0756059399,
                        "category" : "needs"
                     }
                  ],
                  "name" : "Challenge",
                  "percentage" : 0.987323190315372
               }
            ]
         },
         {
            "name" : "Values",
            "id" : "values",
            "children" : [
               {
                  "category" : "values",
                  "id" : "Self-transcendence_parent",
                  "children" : [
                     {
                        "sampling_error" : 0.0656047615,
                        "category" : "values",
                        "percentage" : 0.00994559307044479,
                        "name" : "Conservation",
                        "id" : "Conservation"
                     },
                     {
                        "sampling_error" : 0.0615400136,
                        "category" : "values",
                        "name" : "Openness to change",
                        "percentage" : 0.973430859299974,
                        "id" : "Openness to change"
                     },
                     {
                        "percentage" : 0.0768905657583112,
                        "name" : "Hedonism",
                        "id" : "Hedonism",
                        "sampling_error" : 0.1302910617,
                        "category" : "values"
                     },
                     {
                        "sampling_error" : 0.0968774432,
                        "category" : "values",
                        "percentage" : 0.0417488057977589,
                        "name" : "Self-enhancement",
                        "id" : "Self-enhancement"
                     },
                     {
                        "name" : "Self-transcendence",
                        "percentage" : 1,
                        "id" : "Self-transcendence",
                        "sampling_error" : 0.0752591481,
                        "category" : "values"
                     }
                  ],
                  "percentage" : 1,
                  "name" : "Self-transcendence"
               }
            ]
         }
      ],
      "id" : "r"
   },
   "word_count_message" : "There were 1,663 words in the input. We need a minimum of 3,500, preferably 6,000 or more, to compute a reliable estimate",
   "id" : "*UNKNOWN*",
   "source" : "*UNKNOWN*"
}
```

# Personality-Insightsの結果の解釈

Personality-Insightsが返してくるjsonは、著者のプロファイルに関するさまざまな観点の標準スコア(0〜1.0の間の値にしたもの)を含んでいます。このままではわかりにくいので、以下のツールを使って必要な項目だけ取り出してみます。

* [`jq`](https://stedolan.github.io/jq/): jsonから必要な項目を取り出す
* `sed`: 1行1項目に変換する
* [`json2csv`](https://github.com/jehiah/json2csv): jsonをcsvに変換する
* `csvsort` ([`csvkit`](https://csvkit.readthedocs.org/en/0.9.1/)): スコアの高い項目からの降順でソートする
* `csvlook` ([`csvkit`](https://csvkit.readthedocs.org/en/0.9.1/)): csvを見やすい表で出力する

```console
$ cat personality-insights-by-watson-of-statement-by-prime-minister-Abe-20150815.json | jq '.tree.children[].children[].children[] | {name:.name, category:.category, percentage:.percentage}' | sed ':loop; N; /}/!b loop; s/\n//g' | json2csv -k name,category,percentage -p=true | csvsort -rc percentage | csvlook
|---------------------+-------------+-------------|
|  name               | category    | percentage  |
|---------------------+-------------+-------------|
|  Self-transcendence | values      | 1.0         |
|  Challenge          | needs       | 0.987323    |
|  Openness           | personality | 0.985532    |
|  Openness to change | values      | 0.973431    |
|  Structure          | needs       | 0.8981      |
|  Love               | needs       | 0.827694    |
|  Closeness          | needs       | 0.820033    |
|  Stability          | needs       | 0.81163     |
|  Harmony            | needs       | 0.80973     |
|  Conscientiousness  | personality | 0.757917    |
|  Curiosity          | needs       | 0.751109    |
|  Ideal              | needs       | 0.710925    |
|  Practicality       | needs       | 0.698891    |
|  Excitement         | needs       | 0.55084     |
|  Liberty            | needs       | 0.548307    |
|  Self-expression    | needs       | 0.50433     |
|  Emotional range    | personality | 0.287005    |
|  Extraversion       | personality | 0.268967    |
|  Agreeableness      | personality | 0.20259     |
|  Hedonism           | values      | 0.076891    |
|  Self-enhancement   | values      | 0.041749    |
|  Conservation       | values      | 0.009946    |
|---------------------+-------------+-------------|
```

社会心理学の理論にもとづいてカテゴリ(category)ごとにいくつかの因子が設定されており、それぞれについて標準スコアが出力されています。

|カテゴリ(category)|因子数|意味|
|:------|:-----:|:------------------------|
|パーソナリティ(personality)|5|Goldbergが提唱したパーソナリティを表す指標で、性格診断などによく使われる。5因子からなるためBig Fiveと呼ばれている。たとえば、[Goldberg(1990)](http://projects.ori.org/lrg/PDFs_papers/Goldberg.Big-Five-FactorsStructure.JPSP.1990.pdf)、[Digman(1990)](http://orientation.uchicagolawandecon.org/readings/Digman%20on%20Five%20Factor%20Model.pdf)。|
|関心(needs)|12|[Ford(2005)](http://www.amazon.co.jp/dp/B000PY4M6O/)のUniversal Needs Modelに基づいた、ソーシャルメディアからユーザのニーズをあらわす因子。[IBMの関連研究(2013)](https://domino.research.ibm.com/library/cyberdig.nsf/papers/28778C32C50DDB9185257BFA005AF49B/$File/rj10513.pdf)に詳しい。|
|価値観(values)|5|[Schwarz(1992)](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.220.3674&rep=rep1&type=pdf)の基本価値理論(Theory of Basic Values)の10因子を代表する5因子。[加納&谷田(2015)](http://openarchive.cbs.dk/bitstream/handle/10398/9151/Fumiko%20Kano%20Gl%C3%BCckstad_1D3-NFC-00-1.pdf?sequence=1)を参照。|

なお、パーソナリティについては、さらに詳細な因子ごとのスコアも含まれています。

```console
$ cat personality-insights-by-watson-of-statement-by-prime-minister-Abe-20150815.json | jq '.tree.children[].children[].children[].children[] | {name:.name, category:.category, percentage:.percentage}' | sed ':loop; N; /}/!b loop; s/\n//g' | json2csv -k name,category,percentage -p=true | csvsort -rc percentage | csvlook
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
|------------------------+-------------+-------------|
|  name                  | category    | percentage  |
|------------------------+-------------+-------------|
|  Sympathy              | personality | 1.0         |
|  Intellect             | personality | 0.980715    |
|  Adventurousness       | personality | 0.955299    |
|  Self-efficacy         | personality | 0.951359    |
|  Authority-challenging | personality | 0.944       |
|  Imagination           | personality | 0.930479    |
|  Cautiousness          | personality | 0.864294    |
|  Cooperation           | personality | 0.815784    |
|  Altruism              | personality | 0.702745    |
|  Artistic interests    | personality | 0.592629    |
|  Trust                 | personality | 0.589804    |
|  Self-discipline       | personality | 0.533917    |
|  Achievement striving  | personality | 0.474982    |
|  Modesty               | personality | 0.378329    |
|  Uncompromising        | personality | 0.359537    |
|  Dutifulness           | personality | 0.357662    |
|  Outgoing              | personality | 0.320766    |
|  Fiery                 | personality | 0.301967    |
|  Emotionality          | personality | 0.251675    |
|  Activity level        | personality | 0.236499    |
|  Orderliness           | personality | 0.221789    |
|  Melancholy            | personality | 0.154636    |
|  Assertiveness         | personality | 0.147596    |
|  Prone to worry        | personality | 0.113763    |
|  Cheerfulness          | personality | 0.068019    |
|  Gregariousness        | personality | 0.047509    |
|  Immoderation          | personality | 0.034018    |
|  Self-consciousness    | personality | 0.024674    |
|  Susceptible to stress | personality | 0.019183    |
|  Excitement-seeking    | personality | 0.018545    |
|------------------------+-------------+-------------|
```

さて、もうひとつのサンプルとして、[オバマ大統領の一般教書演説](http://edition.cnn.com/2015/01/20/politics/state-of-the-union-2015-transcript-full-text/)  ([コピー：state-of-the-union-2015-by-president-Obama-20150120.txt](https://gist.github.com/blackaplysia/b172c9c85777a54096d5))も分析してみましょう。

```console
$ curl -u "<username>:<password>" -X POST -H "Accept: application/json" -H "Content-Type: text/plain" --data-binary @state-of-the-union-2015-by-president-Obama-20150120.txt <url>/v2/profile > personality-insights-by-watson-of-state-of-the-union-2015-by-president-Obama-20150120.json

$ cat personality-insights-by-watson-of-state-of-the-union-2015-by-president-Obama-20150120.json | json_pp
{
   "word_count" : 6536,
   "id" : "*UNKNOWN*",
   "processed_lang" : "en",
   "source" : "*UNKNOWN*",
   "tree" : {
      "children" : [
         {
            "name" : "Big 5 ",
            "id" : "personality",
            "children" : [
               {
                  "category" : "personality",
                  "name" : "Agreeableness",
                  "percentage" : 0.932001746134206,
                  "id" : "Agreeableness_parent",
                  "children" : [
                     {
                        "id" : "Openness",
                        "children" : [
                           {
                              "percentage" : 0.620044504134801,
                              "category" : "personality",
                              "sampling_error" : 0.0446153468,
                              "name" : "Adventurousness",
                              "id" : "Adventurousness"
                           },
                           {
                              "sampling_error" : 0.09137023608,
                              "name" : "Artistic interests",
                              "category" : "personality",
                              "percentage" : 0.574326197406973,
                              "id" : "Artistic interests"
                           },
                           {
                              "sampling_error" : 0.04150301592,
                              "name" : "Emotionality",
                              "category" : "personality",
                              "percentage" : 0.379811057957015,
                              "id" : "Emotionality"
                           },
                           {
                              "percentage" : 0.407712532611342,
                              "category" : "personality",
                              "sampling_error" : 0.05538940792,
                              "name" : "Imagination",
                              "id" : "Imagination"
                           },
                           {
                              "percentage" : 0.768419402346327,
                              "name" : "Intellect",
                              "sampling_error" : 0.04808994128,
                              "category" : "personality",
                              "id" : "Intellect"
                           },
                           {
                              "category" : "personality",
                              "name" : "Authority-challenging",
                              "sampling_error" : 0.07412246952,
                              "percentage" : 0.424714914939045,
                              "id" : "Liberalism"
                           }
                        ],
                        "percentage" : 0.639011097234495,
                        "category" : "personality",
                        "sampling_error" : 0.05049954656,
                        "name" : "Openness"
                     },
                     {
                        "children" : [
                           {
                              "id" : "Achievement striving",
                              "category" : "personality",
                              "name" : "Achievement striving",
                              "sampling_error" : 0.08704894088,
                              "percentage" : 0.68504897918378
                           },
                           {
                              "id" : "Cautiousness",
                              "percentage" : 0.805173916995247,
                              "category" : "personality",
                              "sampling_error" : 0.08098496776,
                              "name" : "Cautiousness"
                           },
                           {
                              "percentage" : 0.809184801247061,
                              "sampling_error" : 0.05338133104,
                              "name" : "Dutifulness",
                              "category" : "personality",
                              "id" : "Dutifulness"
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.06145212224,
                              "name" : "Orderliness",
                              "percentage" : 0.524309719114008,
                              "id" : "Orderliness"
                           },
                           {
                              "percentage" : 0.444030296335035,
                              "name" : "Self-discipline",
                              "sampling_error" : 0.04189993792,
                              "category" : "personality",
                              "id" : "Self-discipline"
                           },
                           {
                              "percentage" : 0.897428730503625,
                              "sampling_error" : 0.080181318,
                              "name" : "Self-efficacy",
                              "category" : "personality",
                              "id" : "Self-efficacy"
                           }
                        ],
                        "id" : "Conscientiousness",
                        "sampling_error" : 0.06418342424,
                        "name" : "Conscientiousness",
                        "category" : "personality",
                        "percentage" : 0.619950376031026
                     },
                     {
                        "children" : [
                           {
                              "id" : "Activity level",
                              "name" : "Activity level",
                              "sampling_error" : 0.06727625984,
                              "category" : "personality",
                              "percentage" : 0.880133742481659
                           },
                           {
                              "percentage" : 0.533835129814763,
                              "sampling_error" : 0.07157665928,
                              "name" : "Assertiveness",
                              "category" : "personality",
                              "id" : "Assertiveness"
                           },
                           {
                              "id" : "Cheerfulness",
                              "percentage" : 0.469303553776799,
                              "category" : "personality",
                              "name" : "Cheerfulness",
                              "sampling_error" : 0.08994587168
                           },
                           {
                              "category" : "personality",
                              "sampling_error" : 0.07363806904,
                              "name" : "Excitement-seeking",
                              "percentage" : 0.0929852192811082,
                              "id" : "Excitement-seeking"
                           },
                           {
                              "id" : "Friendliness",
                              "name" : "Outgoing",
                              "sampling_error" : 0.06624518304,
                              "category" : "personality",
                              "percentage" : 0.945336200923735
                           },
                           {
                              "category" : "personality",
                              "name" : "Gregariousness",
                              "sampling_error" : 0.05136579536,
                              "percentage" : 0.588086118326349,
                              "id" : "Gregariousness"
                           }
                        ],
                        "id" : "Extraversion",
                        "percentage" : 0.55034519329129,
                        "name" : "Extraversion",
                        "sampling_error" : 0.0484656948,
                        "category" : "personality"
                     },
                     {
                        "children" : [
                           {
                              "id" : "Altruism",
                              "percentage" : 0.947307343968675,
                              "category" : "personality",
                              "name" : "Altruism",
                              "sampling_error" : 0.0616063704
                           },
                           {
                              "id" : "Cooperation",
                              "percentage" : 0.804359501675388,
                              "sampling_error" : 0.07163908384,
                              "name" : "Cooperation",
                              "category" : "personality"
                           },
                           {
                              "percentage" : 0.433395460499093,
                              "name" : "Modesty",
                              "sampling_error" : 0.0473167684,
                              "category" : "personality",
                              "id" : "Modesty"
                           },
                           {
                              "percentage" : 0.893304534648274,
                              "category" : "personality",
                              "sampling_error" : 0.05500443816,
                              "name" : "Uncompromising",
                              "id" : "Morality"
                           },
                           {
                              "id" : "Sympathy",
                              "percentage" : 0.984218709364435,
                              "category" : "personality",
                              "name" : "Sympathy",
                              "sampling_error" : 0.0855932792
                           },
                           {
                              "id" : "Trust",
                              "percentage" : 0.872562226636081,
                              "name" : "Trust",
                              "sampling_error" : 0.0480674052,
                              "category" : "personality"
                           }
                        ],
                        "id" : "Agreeableness",
                        "percentage" : 0.932001746134206,
                        "category" : "personality",
                        "sampling_error" : 0.08598048048,
                        "name" : "Agreeableness"
                     },
                     {
                        "sampling_error" : 0.07824572992,
                        "name" : "Emotional range",
                        "category" : "personality",
                        "percentage" : 0.430609968085766,
                        "children" : [
                           {
                              "id" : "Anger",
                              "name" : "Fiery",
                              "sampling_error" : 0.082150072,
                              "category" : "personality",
                              "percentage" : 0.478899220087682
                           },
                           {
                              "id" : "Anxiety",
                              "name" : "Prone to worry",
                              "sampling_error" : 0.04709943064,
                              "category" : "personality",
                              "percentage" : 0.356977664894361
                           },
                           {
                              "percentage" : 0.172346957563551,
                              "name" : "Melancholy",
                              "sampling_error" : 0.04909909808,
                              "category" : "personality",
                              "id" : "Depression"
                           },
                           {
                              "id" : "Immoderation",
                              "percentage" : 0.0434493370459736,
                              "name" : "Immoderation",
                              "sampling_error" : 0.04646411168,
                              "category" : "personality"
                           },
                           {
                              "id" : "Self-consciousness",
                              "category" : "personality",
                              "sampling_error" : 0.04827076928,
                              "name" : "Self-consciousness",
                              "percentage" : 0.125956479235689
                           },
                           {
                              "sampling_error" : 0.07376902928,
                              "name" : "Susceptible to stress",
                              "category" : "personality",
                              "percentage" : 0.465978328693608,
                              "id" : "Vulnerability"
                           }
                        ],
                        "id" : "Neuroticism"
                     }
                  ]
               }
            ]
         },
         {
            "name" : "Needs",
            "children" : [
               {
                  "id" : "Ideal_parent",
                  "children" : [
                     {
                        "sampling_error" : 0.07242805976,
                        "name" : "Challenge",
                        "category" : "needs",
                        "percentage" : 0.183384903314868,
                        "id" : "Challenge"
                     },
                     {
                        "sampling_error" : 0.07178429128,
                        "name" : "Closeness",
                        "category" : "needs",
                        "percentage" : 0.241868382118144,
                        "id" : "Closeness"
                     },
                     {
                        "name" : "Curiosity",
                        "sampling_error" : 0.101468752,
                        "category" : "needs",
                        "percentage" : 0.0913698629845685,
                        "id" : "Curiosity"
                     },
                     {
                        "percentage" : 0.22944890105905,
                        "name" : "Excitement",
                        "sampling_error" : 0.0914217388,
                        "category" : "needs",
                        "id" : "Excitement"
                     },
                     {
                        "id" : "Harmony",
                        "percentage" : 0.592258857573556,
                        "category" : "needs",
                        "name" : "Harmony",
                        "sampling_error" : 0.09109814352
                     },
                     {
                        "category" : "needs",
                        "sampling_error" : 0.08264678688,
                        "name" : "Ideal",
                        "percentage" : 0.0433731098207683,
                        "id" : "Ideal"
                     },
                     {
                        "name" : "Liberty",
                        "sampling_error" : 0.12355816792,
                        "category" : "needs",
                        "percentage" : 0.320151564867171,
                        "id" : "Liberty"
                     },
                     {
                        "id" : "Love",
                        "percentage" : 0.762688393238037,
                        "category" : "needs",
                        "sampling_error" : 0.08343663592,
                        "name" : "Love"
                     },
                     {
                        "id" : "Practicality",
                        "sampling_error" : 0.07383334224,
                        "name" : "Practicality",
                        "category" : "needs",
                        "percentage" : 0.745279651432487
                     },
                     {
                        "percentage" : 0.133674041035053,
                        "category" : "needs",
                        "sampling_error" : 0.07084312128,
                        "name" : "Self-expression",
                        "id" : "Self-expression"
                     },
                     {
                        "sampling_error" : 0.09042063256,
                        "name" : "Stability",
                        "category" : "needs",
                        "percentage" : 0.209847185427173,
                        "id" : "Stability"
                     },
                     {
                        "percentage" : 0.16124089766668,
                        "category" : "needs",
                        "sampling_error" : 0.06852254096,
                        "name" : "Structure",
                        "id" : "Structure"
                     }
                  ],
                  "percentage" : 0.0433731098207683,
                  "category" : "needs",
                  "name" : "Ideal"
               }
            ],
            "id" : "needs"
         },
         {
            "id" : "values",
            "children" : [
               {
                  "percentage" : 1,
                  "name" : "Self-transcendence",
                  "category" : "values",
                  "id" : "Self-transcendence_parent",
                  "children" : [
                     {
                        "id" : "Conservation",
                        "percentage" : 0.0243881102112696,
                        "name" : "Conservation",
                        "sampling_error" : 0.0628115876,
                        "category" : "values"
                     },
                     {
                        "id" : "Openness to change",
                        "category" : "values",
                        "name" : "Openness to change",
                        "sampling_error" : 0.05677913816,
                        "percentage" : 0.963258874955989
                     },
                     {
                        "name" : "Hedonism",
                        "sampling_error" : 0.11356124552,
                        "category" : "values",
                        "percentage" : 0.0340708508165267,
                        "id" : "Hedonism"
                     },
                     {
                        "percentage" : 0.0854884747032393,
                        "name" : "Self-enhancement",
                        "sampling_error" : 0.08686127656,
                        "category" : "values",
                        "id" : "Self-enhancement"
                     },
                     {
                        "id" : "Self-transcendence",
                        "percentage" : 1,
                        "sampling_error" : 0.0654105024,
                        "name" : "Self-transcendence",
                        "category" : "values"
                     }
                  ]
               }
            ],
            "name" : "Values"
         }
      ],
      "id" : "r",
      "name" : "root"
   }
}

$ cat personality-insights-by-watson-of-state-of-the-union-2015-by-president-Obama-20150120.json  | jq '.tree.children[].children[].children[] | {name:.name, category:.category, percentage:.percentage}' | sed ':loop; N; /}/!b loop; s/\n//g' | json2csv -k name,category,percentage -p=true | csvsort -rc percentage | csvlook
|---------------------+-------------+-------------|
|  name               | category    | percentage  |
|---------------------+-------------+-------------|
|  Self-transcendence | values      | 1.0         |
|  Openness to change | values      | 0.963259    |
|  Agreeableness      | personality | 0.932002    |
|  Love               | needs       | 0.762688    |
|  Practicality       | needs       | 0.74528     |
|  Openness           | personality | 0.639011    |
|  Conscientiousness  | personality | 0.61995     |
|  Harmony            | needs       | 0.592259    |
|  Extraversion       | personality | 0.550345    |
|  Emotional range    | personality | 0.43061     |
|  Liberty            | needs       | 0.320152    |
|  Closeness          | needs       | 0.241868    |
|  Excitement         | needs       | 0.229449    |
|  Stability          | needs       | 0.209847    |
|  Challenge          | needs       | 0.183385    |
|  Structure          | needs       | 0.161241    |
|  Self-expression    | needs       | 0.133674    |
|  Curiosity          | needs       | 0.09137     |
|  Self-enhancement   | values      | 0.085488    |
|  Ideal              | needs       | 0.043373    |
|  Hedonism           | values      | 0.034071    |
|  Conservation       | values      | 0.024388    |
|---------------------+-------------+-------------|

$ cat personality-insights-by-watson-of-state-of-the-union-2015-by-president-Obama-20150120.json  | jq '.tree.children[].children[].children[].children[] | {name:.name, category:.category, percentage:.percentage}' | sed ':loop; N; /}/!b loop; s/\n//g' | json2csv -k name,category,percentage -p=true | csvsort -rc percentage | csvlook
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
jq: error: Cannot iterate over null
|------------------------+-------------+-------------|
|  name                  | category    | percentage  |
|------------------------+-------------+-------------|
|  Sympathy              | personality | 0.984219    |
|  Altruism              | personality | 0.947307    |
|  Outgoing              | personality | 0.945336    |
|  Self-efficacy         | personality | 0.897429    |
|  Uncompromising        | personality | 0.893305    |
|  Activity level        | personality | 0.880134    |
|  Trust                 | personality | 0.872562    |
|  Dutifulness           | personality | 0.809185    |
|  Cautiousness          | personality | 0.805174    |
|  Cooperation           | personality | 0.80436     |
|  Intellect             | personality | 0.768419    |
|  Achievement striving  | personality | 0.685049    |
|  Adventurousness       | personality | 0.620045    |
|  Gregariousness        | personality | 0.588086    |
|  Artistic interests    | personality | 0.574326    |
|  Assertiveness         | personality | 0.533835    |
|  Orderliness           | personality | 0.52431     |
|  Fiery                 | personality | 0.478899    |
|  Cheerfulness          | personality | 0.469304    |
|  Susceptible to stress | personality | 0.465978    |
|  Self-discipline       | personality | 0.44403     |
|  Modesty               | personality | 0.433395    |
|  Authority-challenging | personality | 0.424715    |
|  Imagination           | personality | 0.407713    |
|  Emotionality          | personality | 0.379811    |
|  Prone to worry        | personality | 0.356978    |
|  Melancholy            | personality | 0.172347    |
|  Self-consciousness    | personality | 0.125956    |
|  Excitement-seeking    | personality | 0.092985    |
|  Immoderation          | personality | 0.043449    |
|------------------------+-------------+-------------|
```

いずれも実際には官僚の作文でしかないでしょうからそれぞれの方の特徴とはいえないかもしれませんが、価値観やニーズの観点の標準スコア分布はよく似ています。しかし、パーソナリティの5因子はかなり傾向が違うようですね。たとえばagreeablenessはオバマ大統領の一般教書演説では0.93ですが、安倍総理大臣の談話では0.2しかありません。この項目はBig Fiveの第2因子に相当するもので、この因子のスコアが高いと、信頼できる、民主的、気前がいい、協力的、がまんづよい、利己的でない、あたたかい、モラルが感じられる、などの傾向があるそうです([Goldberg(1990)](http://projects.ori.org/lrg/PDFs_papers/Goldberg.Big-Five-FactorsStructure.JPSP.1990.pdf))。繰り返しますが、今回のサンプルはあくまでも官僚の作文の評価ですので、これは日米での文章の書き方があらわれているのかもしれませんね。

# まとめ

Personality-Insightsは顧客ひとりひとりのソーシャル情報やソーシャルネットワーク上のテキストに基づいたマーケティングのために使われるサービスで、この十数年の社会心理学(パーソナリティ、価値観、顧客ニーズ分析)の理論に基づいています。現在はまだ英語とスペイン語でしか分析ができませんが、日本語に対応すれば非常に有用性の高いツールとして利用することができそうです。