自然言語処理歌詞分析(日記)

自然言語処理の歌詞分析をまとめました。
```python
# プログラム名 / Program Name: nlp_semantic_analysis_extended.py
# 内容 / Overview: spaCyとTextBlobを用いた自然言語処理（形態素解析・構文解析・感情分析・SVO抽出・類似度・PCA可視化）

# --- 必要ライブラリのインストール（Google Colab初回時） ---
# !pip install spacy textblob nltk
# !python -m spacy download en_core_web_md
# !python -m textblob.download_corpora

# --- モジュールのインポート / Import modules ---
import spacy
from textblob import TextBlob
import nltk
from collections import Counter
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
import numpy as np

nltk.download('punkt')  # TextBlob用

# --- モデルの読み込み / Load spaCy medium model ---
nlp = spacy.load("en_core_web_md")

# --- 入力文 / Input sentence ---
text = "When you were here before Couldn't look you in the eye You're just like an angel."

# --- spaCy文解析 / Morphological & Syntactic Analysis ---
doc = nlp(text)

# --- 1. 形態素解析と品詞 / Token & POS ---
print("\n🔹 Token & POS (形態素と品詞)")
for token in doc:
    print(f"{token.text:<15} → {token.pos_}")

# --- 2. 構文解析 / Dependency Parsing ---
print("\n🔹 Dependency Parse (構文構造)")
for token in doc:
    print(f"{token.text:<15} → {token.dep_} (head: {token.head.text})")

# --- 3. 固有表現抽出 / Named Entity Recognition ---
print("\n🔹 Named Entities (固有表現抽出)")
for ent in doc.ents:
    print(f"{ent.text:<20} → {ent.label_}")

# --- 4. 感情分析 / Sentiment Analysis ---
blob = TextBlob(text)
polarity = blob.sentiment.polarity
subjectivity = blob.sentiment.subjectivity
sentiment_label = "Positive" if polarity > 0.1 else "Negative" if polarity < -0.1 else "Neutral"

print("\n💬 Sentiment Analysis")
print(f"Polarity: {polarity:.2f}  (−1: negative, +1: positive)")
print(f"Subjectivity: {subjectivity:.2f}  (0: fact, 1: opinion)")
print(f"Classification: {sentiment_label}")

# --- 5. 主語・述語・目的語 / Subject-Verb-Object Extraction ---
print("\n🧱 SVO Triplets (主語-述語-目的語 抽出)")
for token in doc:
    if token.dep_ == "ROOT":
        subject = [w for w in token.lefts if w.dep_ in ("nsubj", "nsubjpass")]
        obj = [w for w in token.rights if w.dep_ in ("dobj", "attr", "prep", "pobj")]
        print(f"Verb: {token.text}, Subject: {[w.text for w in subject]}, Object: {[w.text for w in obj]}")

# --- 6. 単語間の意味的類似度 / Word Similarity ---
print("\n🔗 Word Similarity (単語間の意味的類似度)")
tokens = [token for token in doc if token.has_vector and not token.is_stop and token.is_alpha]
if len(tokens) >= 2:
    for i in range(min(3, len(tokens) - 1)):
        token1 = tokens[i]
        token2 = tokens[i + 1]
        sim = token1.similarity(token2)
        print(f"{token1.text} ↔ {token2.text}: Similarity = {sim:.2f}")
else:
    print("（⚠ 類似度を計算できる十分なベクトル付き単語が見つかりませんでした）")

# --- 7. 各単語の感情スコア / Word-wise Sentiment Polarity ---
print("\n📘 Word-wise Sentiment (単語ごとの感情語強度)")
for word in blob.words:
    word_blob = TextBlob(word)
    score = word_blob.sentiment.polarity
    print(f"{word:<15} → Polarity: {score:.2f}")

# --- 8. 単語頻度 / Word Frequency Count ---
print("\n🧮 Word Frequency (単語頻度)")
words = [token.text.lower() for token in doc if token.is_alpha and not token.is_stop]
freq = Counter(words)
for word, count in freq.most_common():
    print(f"{word:<15} → {count}")

# --- 9. 品詞頻度 / POS Frequency ---
print("\n📊 POS Frequency (品詞の出現頻度)")
pos_counts = Counter([token.pos_ for token in doc])
for pos, count in pos_counts.items():
    print(f"{pos:<10} → {count}")

# --- 10. 名詞句抽出 / Noun Phrase Extraction ---
print("\n🧱 Noun Phrases (名詞句)")
for chunk in doc.noun_chunks:
    print(chunk.text)

# --- 11. 意味ベクトルのPCAプロット / Word Vector PCA Visualization ---
print("\n🖼 2D PCA Word Vector Visualization")
tokens_with_vec = [token for token in doc if token.has_vector and token.is_alpha and not token.is_stop]
if len(tokens_with_vec) >= 2:
    words = [token.text for token in tokens_with_vec]
    vectors = np.array([token.vector for token in tokens_with_vec])

    pca = PCA(n_components=2)
    reduced = pca.fit_transform(vectors)

    plt.figure(figsize=(8, 6))
    plt.scatter(reduced[:, 0], reduced[:, 1])
    for i, word in enumerate(words):
        plt.annotate(word, (reduced[i, 0], reduced[i, 1]))
    plt.title("2D PCA of Word Vectors")
    plt.xlabel("PC1")
    plt.ylabel("PC2")
    plt.grid(True)
    plt.tight_layout()
    plt.show()
else:
    print("⚠ 可視化できるベクトル付き単語が不足しています。")
```
結果
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Package punkt is already up-to-date!

🔹 Token & POS (形態素と品詞)
When            → SCONJ
you             → PRON
were            → AUX
here            → ADV
before          → ADV
Could           → AUX
n't             → PART
look            → VERB
you             → PRON
in              → ADP
the             → DET
eye             → NOUN
You             → PRON
're             → AUX
just            → ADV
like            → ADP
an              → DET
angel           → NOUN
.               → PUNCT

🔹 Dependency Parse (構文構造)
When            → advmod (head: were)
you             → nsubj (head: were)
were            → advcl (head: look)
here            → advmod (head: were)
before          → advmod (head: were)
Could           → aux (head: look)
n't             → neg (head: look)
look            → ROOT (head: look)
you             → dobj (head: look)
in              → prep (head: look)
the             → det (head: eye)
eye             → pobj (head: in)
You             → nsubj (head: 're)
're             → conj (head: look)
just            → advmod (head: like)
like            → prep (head: 're)
an              → det (head: angel)
angel           → pobj (head: like)
.               → punct (head: 're)

🔹 Named Entities (固有表現抽出)

💬 Sentiment Analysis
Polarity: 0.00  (−1: negative, +1: positive)
Subjectivity: 0.00  (0: fact, 1: opinion)
Classification: Neutral

🧱 SVO Triplets (主語-述語-目的語 抽出)
Verb: look, Subject: [], Object: ['you', 'in']

🔗 Word Similarity (単語間の意味的類似度)
look ↔ eye: Similarity = 0.27
eye ↔ like: Similarity = 0.18
like ↔ angel: Similarity = 0.36

📘 Word-wise Sentiment (単語ごとの感情語強度)
When            → Polarity: 0.00
you             → Polarity: 0.00
were            → Polarity: 0.00
here            → Polarity: 0.00
before          → Polarity: 0.00
Could           → Polarity: 0.00
n't             → Polarity: 0.00
look            → Polarity: 0.00
you             → Polarity: 0.00
in              → Polarity: 0.00
the             → Polarity: 0.00
eye             → Polarity: 0.00
You             → Polarity: 0.00
're             → Polarity: 0.00
just            → Polarity: 0.00
like            → Polarity: 0.00
an              → Polarity: 0.00
angel           → Polarity: 0.00

🧮 Word Frequency (単語頻度)
look            → 1
eye             → 1
like            → 1
angel           → 1

📊 POS Frequency (品詞の出現頻度)
SCONJ      → 1
PRON       → 3
AUX        → 3
ADV        → 3
PART       → 1
VERB       → 1
ADP        → 2
DET        → 2
NOUN       → 2
PUNCT      → 1

 Noun Phrases (名詞句)
you
you
the eye
You
an angel



![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/4046871/81477442-d639-4aef-8486-b6c9aff8b2f4.png)