Vu Anh
Claude
commited on
Commit
·
0b1c1cf
1
Parent(s):
e423877
Update technical report and README with latest SVC model results
Browse files- Updated technical report abstract and results with SVC accuracy: 71.72%
- Added SVC algorithm details in classification model section
- Updated README performance metrics with latest SVC results
- Updated model filename references to latest exported model
- Training accuracy: 94.31%, Test accuracy: 71.72%, Training time: 7.71s
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- README.md +10 -9
- analyze_results.py +136 -0
- inference.py +12 -4
- paper/pulse_core_1_technical_report.tex +16 -8
- runs/20250928_131527/metadata.json +1531 -0
- runs/20250928_131527/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib +3 -0
- runs/20250928_131527/models/labels.txt +35 -0
- runs/20250928_131527/models/model.joblib +3 -0
- runs/20250928_131527/training.log +62 -0
- runs/20250928_131716/metadata.json +1531 -0
- runs/20250928_131716/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib +3 -0
- runs/20250928_131716/models/labels.txt +35 -0
- runs/20250928_131716/models/model.joblib +3 -0
- runs/20250928_131716/training.log +63 -0
- uts2017_sentiment_20250928_131716.joblib +3 -0
README.md
CHANGED
|
@@ -33,7 +33,7 @@ model-index:
|
|
| 33 |
type: undertheseanlp/UTS2017_Bank
|
| 34 |
metrics:
|
| 35 |
- type: accuracy
|
| 36 |
-
value: 0.
|
| 37 |
name: Test Accuracy (SVC)
|
| 38 |
- type: accuracy
|
| 39 |
value: 0.6818
|
|
@@ -57,7 +57,7 @@ pipeline_tag: text-classification
|
|
| 57 |
|
| 58 |
# Pulse Core 1 - Vietnamese Banking Aspect Sentiment Analysis
|
| 59 |
|
| 60 |
-
A machine learning-based aspect sentiment analysis model designed for Vietnamese banking text processing. Built on TF-IDF feature extraction pipeline combined with various machine learning algorithms, achieving **
|
| 61 |
|
| 62 |
📋 **[View Detailed System Card](https://huggingface.co/undertheseanlp/pulse_core_1/blob/main/paper/pulse_core_1_technical_report.tex)** for comprehensive model documentation, performance analysis, and limitations.
|
| 63 |
|
|
@@ -156,18 +156,19 @@ comparison_results = train_notebook(
|
|
| 156 |
## Performance Metrics
|
| 157 |
|
| 158 |
### UTS2017_Bank Aspect Sentiment Analysis Performance
|
| 159 |
-
- **Training Accuracy**:
|
| 160 |
-
- **Test Accuracy**:
|
| 161 |
- **Training Samples**: 1,581
|
| 162 |
- **Test Samples**: 396
|
| 163 |
- **Number of Classes**: 35 aspect-sentiment combinations
|
| 164 |
-
- **Training Time**: ~
|
| 165 |
- **Best Performing Classes**:
|
|
|
|
| 166 |
- `CUSTOMER_SUPPORT#positive`: 88% F1-score
|
| 167 |
-
- `
|
| 168 |
-
- `CUSTOMER_SUPPORT#negative`:
|
| 169 |
- **Challenges**: Class imbalance affects minority aspect-sentiment combinations
|
| 170 |
-
- **Model Type**:
|
| 171 |
|
| 172 |
## Using the Pre-trained Models
|
| 173 |
|
|
@@ -177,7 +178,7 @@ comparison_results = train_notebook(
|
|
| 177 |
import joblib
|
| 178 |
|
| 179 |
# Load local exported model
|
| 180 |
-
sentiment_model = joblib.load("
|
| 181 |
|
| 182 |
# Or use inference script directly
|
| 183 |
from inference import predict_text
|
|
|
|
| 33 |
type: undertheseanlp/UTS2017_Bank
|
| 34 |
metrics:
|
| 35 |
- type: accuracy
|
| 36 |
+
value: 0.7172
|
| 37 |
name: Test Accuracy (SVC)
|
| 38 |
- type: accuracy
|
| 39 |
value: 0.6818
|
|
|
|
| 57 |
|
| 58 |
# Pulse Core 1 - Vietnamese Banking Aspect Sentiment Analysis
|
| 59 |
|
| 60 |
+
A machine learning-based aspect sentiment analysis model designed for Vietnamese banking text processing. Built on TF-IDF feature extraction pipeline combined with various machine learning algorithms, achieving **71.72% accuracy** on UTS2017_Bank aspect sentiment dataset with Support Vector Classification (SVC).
|
| 61 |
|
| 62 |
📋 **[View Detailed System Card](https://huggingface.co/undertheseanlp/pulse_core_1/blob/main/paper/pulse_core_1_technical_report.tex)** for comprehensive model documentation, performance analysis, and limitations.
|
| 63 |
|
|
|
|
| 156 |
## Performance Metrics
|
| 157 |
|
| 158 |
### UTS2017_Bank Aspect Sentiment Analysis Performance
|
| 159 |
+
- **Training Accuracy**: 94.31%
|
| 160 |
+
- **Test Accuracy**: 71.72%
|
| 161 |
- **Training Samples**: 1,581
|
| 162 |
- **Test Samples**: 396
|
| 163 |
- **Number of Classes**: 35 aspect-sentiment combinations
|
| 164 |
+
- **Training Time**: ~7.71 seconds
|
| 165 |
- **Best Performing Classes**:
|
| 166 |
+
- `TRADEMARK#positive`: 90% F1-score
|
| 167 |
- `CUSTOMER_SUPPORT#positive`: 88% F1-score
|
| 168 |
+
- `LOAN#negative`: 67% F1-score
|
| 169 |
+
- `CUSTOMER_SUPPORT#negative`: 65% F1-score
|
| 170 |
- **Challenges**: Class imbalance affects minority aspect-sentiment combinations
|
| 171 |
+
- **Model Type**: Support Vector Classification (SVC) with TF-IDF (20k features, 1-2 ngrams)
|
| 172 |
|
| 173 |
## Using the Pre-trained Models
|
| 174 |
|
|
|
|
| 178 |
import joblib
|
| 179 |
|
| 180 |
# Load local exported model
|
| 181 |
+
sentiment_model = joblib.load("uts2017_sentiment_20250928_131716.joblib")
|
| 182 |
|
| 183 |
# Or use inference script directly
|
| 184 |
from inference import predict_text
|
analyze_results.py
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
Script to analyze and compare training results from multiple model runs.
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import json
|
| 7 |
+
import os
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
|
| 10 |
+
def load_metadata(run_dir):
|
| 11 |
+
"""Load metadata from a training run directory"""
|
| 12 |
+
metadata_path = os.path.join(run_dir, "metadata.json")
|
| 13 |
+
if os.path.exists(metadata_path):
|
| 14 |
+
with open(metadata_path, 'r', encoding='utf-8') as f:
|
| 15 |
+
return json.load(f)
|
| 16 |
+
return None
|
| 17 |
+
|
| 18 |
+
def analyze_all_runs():
|
| 19 |
+
"""Analyze all training runs and create comparison"""
|
| 20 |
+
runs_dir = Path("runs")
|
| 21 |
+
results = []
|
| 22 |
+
|
| 23 |
+
# Find all metadata files
|
| 24 |
+
for run_dir in runs_dir.glob("*/"):
|
| 25 |
+
if run_dir.is_dir():
|
| 26 |
+
metadata = load_metadata(run_dir)
|
| 27 |
+
if metadata:
|
| 28 |
+
results.append({
|
| 29 |
+
'run_id': run_dir.name,
|
| 30 |
+
'model': metadata.get('classifier', 'Unknown'),
|
| 31 |
+
'dataset': 'VNTC' if 'VNTC' in metadata.get('config_name', '') else 'UTS2017_Bank',
|
| 32 |
+
'max_features': metadata.get('max_features', 0),
|
| 33 |
+
'ngram_range': metadata.get('ngram_range', [1,1]),
|
| 34 |
+
'train_accuracy': metadata.get('train_accuracy', 0),
|
| 35 |
+
'test_accuracy': metadata.get('test_accuracy', 0),
|
| 36 |
+
'train_time': metadata.get('train_time', 0),
|
| 37 |
+
'prediction_time': metadata.get('prediction_time', 0),
|
| 38 |
+
'train_samples': metadata.get('train_samples', 0),
|
| 39 |
+
'test_samples': metadata.get('test_samples', 0)
|
| 40 |
+
})
|
| 41 |
+
|
| 42 |
+
return results
|
| 43 |
+
|
| 44 |
+
def print_comparison_table(results):
|
| 45 |
+
"""Print formatted comparison table"""
|
| 46 |
+
print("\n" + "="*120)
|
| 47 |
+
print("VIETNAMESE TEXT CLASSIFICATION - MODEL COMPARISON RESULTS")
|
| 48 |
+
print("="*120)
|
| 49 |
+
|
| 50 |
+
# Filter for VNTC results (news classification)
|
| 51 |
+
vntc_results = [r for r in results if r['dataset'] == 'VNTC']
|
| 52 |
+
|
| 53 |
+
if vntc_results:
|
| 54 |
+
print("\nVNTC Dataset (Vietnamese News Classification):")
|
| 55 |
+
print("-"*120)
|
| 56 |
+
print(f"{'Model':<20} {'Features':<10} {'N-gram':<10} {'Train Acc':<12} {'Test Acc':<12} {'Train Time':<12} {'Pred Time':<12}")
|
| 57 |
+
print("-"*120)
|
| 58 |
+
|
| 59 |
+
# Sort by test accuracy
|
| 60 |
+
vntc_results.sort(key=lambda x: x['test_accuracy'], reverse=True)
|
| 61 |
+
|
| 62 |
+
for result in vntc_results:
|
| 63 |
+
model = result['model'][:18]
|
| 64 |
+
features = f"{result['max_features']//1000}k" if result['max_features'] > 0 else "N/A"
|
| 65 |
+
ngram = f"{result['ngram_range'][0]}-{result['ngram_range'][1]}"
|
| 66 |
+
train_acc = f"{result['train_accuracy']:.4f}"
|
| 67 |
+
test_acc = f"{result['test_accuracy']:.4f}"
|
| 68 |
+
train_time = f"{result['train_time']:.1f}s"
|
| 69 |
+
pred_time = f"{result['prediction_time']:.1f}s"
|
| 70 |
+
|
| 71 |
+
print(f"{model:<20} {features:<10} {ngram:<10} {train_acc:<12} {test_acc:<12} {train_time:<12} {pred_time:<12}")
|
| 72 |
+
|
| 73 |
+
# Filter for UTS2017_Bank results
|
| 74 |
+
bank_results = [r for r in results if r['dataset'] == 'UTS2017_Bank']
|
| 75 |
+
|
| 76 |
+
if bank_results:
|
| 77 |
+
print("\nUTS2017_Bank Dataset (Vietnamese Banking Text Classification):")
|
| 78 |
+
print("-"*120)
|
| 79 |
+
print(f"{'Model':<20} {'Features':<10} {'N-gram':<10} {'Train Acc':<12} {'Test Acc':<12} {'Train Time':<12} {'Pred Time':<12}")
|
| 80 |
+
print("-"*120)
|
| 81 |
+
|
| 82 |
+
# Sort by test accuracy
|
| 83 |
+
bank_results.sort(key=lambda x: x['test_accuracy'], reverse=True)
|
| 84 |
+
|
| 85 |
+
for result in bank_results:
|
| 86 |
+
model = result['model'][:18]
|
| 87 |
+
features = f"{result['max_features']//1000}k" if result['max_features'] > 0 else "N/A"
|
| 88 |
+
ngram = f"{result['ngram_range'][0]}-{result['ngram_range'][1]}"
|
| 89 |
+
train_acc = f"{result['train_accuracy']:.4f}"
|
| 90 |
+
test_acc = f"{result['test_accuracy']:.4f}"
|
| 91 |
+
train_time = f"{result['train_time']:.1f}s"
|
| 92 |
+
pred_time = f"{result['prediction_time']:.1f}s"
|
| 93 |
+
|
| 94 |
+
print(f"{model:<20} {features:<10} {ngram:<10} {train_acc:<12} {test_acc:<12} {train_time:<12} {pred_time:<12}")
|
| 95 |
+
|
| 96 |
+
print("="*120)
|
| 97 |
+
|
| 98 |
+
if vntc_results:
|
| 99 |
+
best_vntc = max(vntc_results, key=lambda x: x['test_accuracy'])
|
| 100 |
+
print(f"\nBest VNTC model: {best_vntc['model']} with {best_vntc['test_accuracy']:.4f} test accuracy")
|
| 101 |
+
|
| 102 |
+
if bank_results:
|
| 103 |
+
best_bank = max(bank_results, key=lambda x: x['test_accuracy'])
|
| 104 |
+
print(f"Best UTS2017_Bank model: {best_bank['model']} with {best_bank['test_accuracy']:.4f} test accuracy")
|
| 105 |
+
|
| 106 |
+
def main():
|
| 107 |
+
"""Main analysis function"""
|
| 108 |
+
print("Analyzing Vietnamese Text Classification Training Results...")
|
| 109 |
+
|
| 110 |
+
results = analyze_all_runs()
|
| 111 |
+
|
| 112 |
+
if not results:
|
| 113 |
+
print("No training results found in runs/ directory.")
|
| 114 |
+
return
|
| 115 |
+
|
| 116 |
+
print(f"Found {len(results)} training runs.")
|
| 117 |
+
print_comparison_table(results)
|
| 118 |
+
|
| 119 |
+
# Create summary statistics
|
| 120 |
+
vntc_results = [r for r in results if r['dataset'] == 'VNTC']
|
| 121 |
+
bank_results = [r for r in results if r['dataset'] == 'UTS2017_Bank']
|
| 122 |
+
|
| 123 |
+
print("\nSummary:")
|
| 124 |
+
print(f"- VNTC runs: {len(vntc_results)}")
|
| 125 |
+
print(f"- UTS2017_Bank runs: {len(bank_results)}")
|
| 126 |
+
|
| 127 |
+
if vntc_results:
|
| 128 |
+
avg_vntc_acc = sum(r['test_accuracy'] for r in vntc_results) / len(vntc_results)
|
| 129 |
+
print(f"- Average VNTC test accuracy: {avg_vntc_acc:.4f}")
|
| 130 |
+
|
| 131 |
+
if bank_results:
|
| 132 |
+
avg_bank_acc = sum(r['test_accuracy'] for r in bank_results) / len(bank_results)
|
| 133 |
+
print(f"- Average UTS2017_Bank test accuracy: {avg_bank_acc:.4f}")
|
| 134 |
+
|
| 135 |
+
if __name__ == "__main__":
|
| 136 |
+
main()
|
inference.py
CHANGED
|
@@ -23,11 +23,19 @@ def find_local_models():
|
|
| 23 |
if filename.startswith('uts2017_sentiment_'):
|
| 24 |
models['exported']['uts2017_sentiment'] = filename
|
| 25 |
|
| 26 |
-
# Find models in runs directory
|
| 27 |
sentiment_runs = glob.glob('runs/*/models/UTS2017_Bank_AspectSentiment_*.joblib')
|
| 28 |
|
| 29 |
if sentiment_runs:
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
return models
|
| 33 |
|
|
@@ -188,8 +196,8 @@ def main():
|
|
| 188 |
"--source",
|
| 189 |
type=str,
|
| 190 |
choices=["exported", "runs"],
|
| 191 |
-
default="
|
| 192 |
-
help="Model source: exported files or runs directory (default:
|
| 193 |
)
|
| 194 |
|
| 195 |
args = parser.parse_args()
|
|
|
|
| 23 |
if filename.startswith('uts2017_sentiment_'):
|
| 24 |
models['exported']['uts2017_sentiment'] = filename
|
| 25 |
|
| 26 |
+
# Find models in runs directory - prioritize SVC models
|
| 27 |
sentiment_runs = glob.glob('runs/*/models/UTS2017_Bank_AspectSentiment_*.joblib')
|
| 28 |
|
| 29 |
if sentiment_runs:
|
| 30 |
+
# Sort by modification time (most recent first)
|
| 31 |
+
sentiment_runs.sort(key=lambda x: os.path.getmtime(x), reverse=True)
|
| 32 |
+
|
| 33 |
+
# Prefer SVC models over other types
|
| 34 |
+
svc_models = [m for m in sentiment_runs if 'SVC' in m]
|
| 35 |
+
if svc_models:
|
| 36 |
+
models['runs']['uts2017_sentiment'] = svc_models[0] # Most recent SVC
|
| 37 |
+
else:
|
| 38 |
+
models['runs']['uts2017_sentiment'] = sentiment_runs[0] # Most recent any model
|
| 39 |
|
| 40 |
return models
|
| 41 |
|
|
|
|
| 196 |
"--source",
|
| 197 |
type=str,
|
| 198 |
choices=["exported", "runs"],
|
| 199 |
+
default="runs",
|
| 200 |
+
help="Model source: exported files or runs directory (default: runs)"
|
| 201 |
)
|
| 202 |
|
| 203 |
args = parser.parse_args()
|
paper/pulse_core_1_technical_report.tex
CHANGED
|
@@ -23,7 +23,7 @@
|
|
| 23 |
\maketitle
|
| 24 |
|
| 25 |
\begin{abstract}
|
| 26 |
-
This paper presents Pulse Core 1, a Vietnamese banking aspect sentiment analysis system employing Term Frequency-Inverse Document Frequency (TF-IDF) feature extraction combined with machine learning classification algorithms. The system is evaluated on the UTS2017\_Bank aspect sentiment dataset containing 35 combined aspect-sentiment categories, achieving 68.18\% accuracy with Logistic Regression and 72
|
| 27 |
\end{abstract}
|
| 28 |
|
| 29 |
\section{Introduction}
|
|
@@ -92,12 +92,20 @@ The system employs Term Frequency-Inverse Document Frequency (TF-IDF) vectorizat
|
|
| 92 |
|
| 93 |
\subsubsection{Classification Model}
|
| 94 |
|
| 95 |
-
The system implements
|
| 96 |
|
|
|
|
| 97 |
\begin{itemize}
|
| 98 |
\item \textbf{Optimization Algorithm}: Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) quasi-Newton method for efficient convergence
|
| 99 |
\item \textbf{Regularization}: L2 (Ridge) penalty with automatic parameter tuning to prevent overfitting
|
| 100 |
\item \textbf{Convergence Criteria}: Maximum 1,000 iterations with tolerance-based early stopping
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
\item \textbf{Multi-class Extension}: One-vs-Rest strategy for handling multi-label classification tasks
|
| 102 |
\end{itemize}
|
| 103 |
|
|
@@ -168,7 +176,7 @@ This work establishes the first comprehensive baseline for Vietnamese banking as
|
|
| 168 |
\hline
|
| 169 |
\textbf{Dataset} & \textbf{Method} & \textbf{Accuracy} \\
|
| 170 |
\hline
|
| 171 |
-
UTS2017\_Bank (35 aspect-sentiment) & \textbf{Pulse Core 1 - SVC with TF-IDF} & \textbf{72
|
| 172 |
UTS2017\_Bank (35 aspect-sentiment) & \textbf{Pulse Core 1 - Logistic Regression with TF-IDF} & \textbf{68.18\%} \\
|
| 173 |
\hline
|
| 174 |
\end{tabular}
|
|
@@ -187,7 +195,7 @@ This section presents comprehensive experimental results across both Vietnamese
|
|
| 187 |
\textbf{UTS2017\_Bank Dataset (Banking Aspect Sentiment Analysis):}
|
| 188 |
The system exhibits competitive performance on the banking aspect sentiment analysis task:
|
| 189 |
\begin{itemize}
|
| 190 |
-
\item \textbf{Test Classification Accuracy (SVC)}: 72
|
| 191 |
\item \textbf{Test Classification Accuracy (Logistic Regression)}: 68.18\%
|
| 192 |
\item \textbf{Training Latency (SVC)}: 5.3 seconds
|
| 193 |
\item \textbf{Training Latency (Logistic Regression)}: 2.13 seconds
|
|
@@ -224,14 +232,14 @@ This section presents a detailed comparison of per-class performance between Sup
|
|
| 224 |
|
| 225 |
\textbf{Model Performance Summary:}
|
| 226 |
\begin{itemize}
|
| 227 |
-
\item \textbf{SVC Model}: 72
|
| 228 |
\item \textbf{Logistic Regression Model}: 70.96\% overall accuracy with 20k features and 1-2 n-gram range
|
| 229 |
\item \textbf{Performance Gain}: SVC achieves a 1.51 percentage point improvement over Logistic Regression
|
| 230 |
\end{itemize}
|
| 231 |
|
| 232 |
\textbf{UTS2017\_Bank Dataset Comparative Per-Class Results:}
|
| 233 |
|
| 234 |
-
The following table presents a side-by-side comparison of per-class performance metrics for both SVC (72
|
| 235 |
|
| 236 |
\begin{longtable}{lcccccccc}
|
| 237 |
\toprule
|
|
@@ -345,7 +353,7 @@ Analysis of the banking aspect sentiment task reveals important insights about V
|
|
| 345 |
The experimental results establish that systematically optimized traditional machine learning methodologies maintain competitive performance for Vietnamese banking aspect sentiment analysis tasks, providing an efficient baseline for financial domain applications. These findings yield several significant research implications:
|
| 346 |
|
| 347 |
\begin{itemize}
|
| 348 |
-
\item \textbf{Computational Resource Efficiency}: The proposed approach exhibits substantially reduced computational complexity compared to transformer-based alternatives while achieving 72
|
| 349 |
\item \textbf{Model Interpretability}: TF-IDF feature representations provide transparent attribution mechanisms for banking aspect sentiment decisions, essential for financial applications requiring algorithmic accountability.
|
| 350 |
\item \textbf{Production Deployment Viability}: The system's constrained computational requirements (2.13s training time) facilitate deployment in banking environments with limited computational resources.
|
| 351 |
\end{itemize}
|
|
@@ -416,7 +424,7 @@ The current investigation establishes several promising research trajectories fo
|
|
| 416 |
This paper presents Pulse Core 1, a Vietnamese banking aspect sentiment analysis system that establishes the viability of systematically optimized traditional machine learning methodologies for financial domain applications. The investigation yields several significant findings:
|
| 417 |
|
| 418 |
\begin{enumerate}
|
| 419 |
-
\item Traditional machine learning approaches achieve competitive performance on Vietnamese banking aspect sentiment analysis tasks (72
|
| 420 |
\item Feature engineering methodologies retain critical importance for Vietnamese banking applications, with the implemented 20,000-dimensional TF-IDF representation effectively capturing aspect-sentiment relationships across 35 combined categories.
|
| 421 |
\item Class distribution imbalance constitutes the primary performance limitation for aspect sentiment analysis, with minority aspect-sentiment combinations achieving zero performance due to insufficient training data.
|
| 422 |
\item The fundamental trade-off between algorithmic complexity and model interpretability substantially favors TF-IDF approaches for banking applications requiring transparency and regulatory compliance.
|
|
|
|
| 23 |
\maketitle
|
| 24 |
|
| 25 |
\begin{abstract}
|
| 26 |
+
This paper presents Pulse Core 1, a Vietnamese banking aspect sentiment analysis system employing Term Frequency-Inverse Document Frequency (TF-IDF) feature extraction combined with machine learning classification algorithms. The system is evaluated on the UTS2017\_Bank aspect sentiment dataset containing 35 combined aspect-sentiment categories, achieving 68.18\% accuracy with Logistic Regression and 71.72\% accuracy with Support Vector Classification (SVC). The implementation utilizes a 20,000-dimensional TF-IDF feature space with n-gram analysis and incorporates hash-based caching for computational optimization. The model predicts combined aspect-sentiment labels in the format \texttt{<aspect>\#<sentiment>}, enabling fine-grained analysis of Vietnamese banking customer feedback across 14 banking aspects (ACCOUNT, CARD, CUSTOMER\_SUPPORT, etc.) and 3 sentiment polarities (positive, negative, neutral). These results establish baseline performance metrics for Vietnamese banking aspect sentiment analysis and demonstrate the efficacy of traditional machine learning approaches for Vietnamese financial domain natural language processing tasks.
|
| 27 |
\end{abstract}
|
| 28 |
|
| 29 |
\section{Introduction}
|
|
|
|
| 92 |
|
| 93 |
\subsubsection{Classification Model}
|
| 94 |
|
| 95 |
+
The system implements both Logistic Regression and Support Vector Classification (SVC) algorithms with the following hyperparameter configurations:
|
| 96 |
|
| 97 |
+
\textbf{Logistic Regression}:
|
| 98 |
\begin{itemize}
|
| 99 |
\item \textbf{Optimization Algorithm}: Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) quasi-Newton method for efficient convergence
|
| 100 |
\item \textbf{Regularization}: L2 (Ridge) penalty with automatic parameter tuning to prevent overfitting
|
| 101 |
\item \textbf{Convergence Criteria}: Maximum 1,000 iterations with tolerance-based early stopping
|
| 102 |
+
\end{itemize}
|
| 103 |
+
|
| 104 |
+
\textbf{Support Vector Classification (SVC)}:
|
| 105 |
+
\begin{itemize}
|
| 106 |
+
\item \textbf{Kernel Function}: Linear kernel for computational efficiency and interpretability
|
| 107 |
+
\item \textbf{Regularization Parameter}: C=1.0 with automatic tuning capability
|
| 108 |
+
\item \textbf{Probability Estimation}: Enabled for confidence scoring and multi-class probability distribution
|
| 109 |
\item \textbf{Multi-class Extension}: One-vs-Rest strategy for handling multi-label classification tasks
|
| 110 |
\end{itemize}
|
| 111 |
|
|
|
|
| 176 |
\hline
|
| 177 |
\textbf{Dataset} & \textbf{Method} & \textbf{Accuracy} \\
|
| 178 |
\hline
|
| 179 |
+
UTS2017\_Bank (35 aspect-sentiment) & \textbf{Pulse Core 1 - SVC with TF-IDF} & \textbf{71.72\%} \\
|
| 180 |
UTS2017\_Bank (35 aspect-sentiment) & \textbf{Pulse Core 1 - Logistic Regression with TF-IDF} & \textbf{68.18\%} \\
|
| 181 |
\hline
|
| 182 |
\end{tabular}
|
|
|
|
| 195 |
\textbf{UTS2017\_Bank Dataset (Banking Aspect Sentiment Analysis):}
|
| 196 |
The system exhibits competitive performance on the banking aspect sentiment analysis task:
|
| 197 |
\begin{itemize}
|
| 198 |
+
\item \textbf{Test Classification Accuracy (SVC)}: 71.72\%
|
| 199 |
\item \textbf{Test Classification Accuracy (Logistic Regression)}: 68.18\%
|
| 200 |
\item \textbf{Training Latency (SVC)}: 5.3 seconds
|
| 201 |
\item \textbf{Training Latency (Logistic Regression)}: 2.13 seconds
|
|
|
|
| 232 |
|
| 233 |
\textbf{Model Performance Summary:}
|
| 234 |
\begin{itemize}
|
| 235 |
+
\item \textbf{SVC Model}: 71.72\% overall accuracy with 20k features and 1-2 n-gram range
|
| 236 |
\item \textbf{Logistic Regression Model}: 70.96\% overall accuracy with 20k features and 1-2 n-gram range
|
| 237 |
\item \textbf{Performance Gain}: SVC achieves a 1.51 percentage point improvement over Logistic Regression
|
| 238 |
\end{itemize}
|
| 239 |
|
| 240 |
\textbf{UTS2017\_Bank Dataset Comparative Per-Class Results:}
|
| 241 |
|
| 242 |
+
The following table presents a side-by-side comparison of per-class performance metrics for both SVC (71.72\% accuracy) and Logistic Regression (70.96\% accuracy) models:
|
| 243 |
|
| 244 |
\begin{longtable}{lcccccccc}
|
| 245 |
\toprule
|
|
|
|
| 353 |
The experimental results establish that systematically optimized traditional machine learning methodologies maintain competitive performance for Vietnamese banking aspect sentiment analysis tasks, providing an efficient baseline for financial domain applications. These findings yield several significant research implications:
|
| 354 |
|
| 355 |
\begin{itemize}
|
| 356 |
+
\item \textbf{Computational Resource Efficiency}: The proposed approach exhibits substantially reduced computational complexity compared to transformer-based alternatives while achieving 71.72\% accuracy on complex aspect sentiment analysis tasks.
|
| 357 |
\item \textbf{Model Interpretability}: TF-IDF feature representations provide transparent attribution mechanisms for banking aspect sentiment decisions, essential for financial applications requiring algorithmic accountability.
|
| 358 |
\item \textbf{Production Deployment Viability}: The system's constrained computational requirements (2.13s training time) facilitate deployment in banking environments with limited computational resources.
|
| 359 |
\end{itemize}
|
|
|
|
| 424 |
This paper presents Pulse Core 1, a Vietnamese banking aspect sentiment analysis system that establishes the viability of systematically optimized traditional machine learning methodologies for financial domain applications. The investigation yields several significant findings:
|
| 425 |
|
| 426 |
\begin{enumerate}
|
| 427 |
+
\item Traditional machine learning approaches achieve competitive performance on Vietnamese banking aspect sentiment analysis tasks (71.72\% accuracy with SVC) while maintaining substantial computational efficiency advantages (5.3s training time).
|
| 428 |
\item Feature engineering methodologies retain critical importance for Vietnamese banking applications, with the implemented 20,000-dimensional TF-IDF representation effectively capturing aspect-sentiment relationships across 35 combined categories.
|
| 429 |
\item Class distribution imbalance constitutes the primary performance limitation for aspect sentiment analysis, with minority aspect-sentiment combinations achieving zero performance due to insufficient training data.
|
| 430 |
\item The fundamental trade-off between algorithmic complexity and model interpretability substantially favors TF-IDF approaches for banking applications requiring transparency and regulatory compliance.
|
runs/20250928_131527/metadata.json
ADDED
|
@@ -0,0 +1,1531 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"timestamp": "20250928_131527",
|
| 3 |
+
"config_name": "UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2",
|
| 4 |
+
"model_name": "svc_linear",
|
| 5 |
+
"classifier": "SVC",
|
| 6 |
+
"max_features": 20000,
|
| 7 |
+
"ngram_range": [
|
| 8 |
+
1,
|
| 9 |
+
2
|
| 10 |
+
],
|
| 11 |
+
"split_ratio": 0.2,
|
| 12 |
+
"n_samples": null,
|
| 13 |
+
"train_samples": 1581,
|
| 14 |
+
"test_samples": 396,
|
| 15 |
+
"unique_labels": 35,
|
| 16 |
+
"labels": [
|
| 17 |
+
"ACCOUNT#negative",
|
| 18 |
+
"CARD#negative",
|
| 19 |
+
"CARD#neutral",
|
| 20 |
+
"CARD#positive",
|
| 21 |
+
"CUSTOMER_SUPPORT#negative",
|
| 22 |
+
"CUSTOMER_SUPPORT#neutral",
|
| 23 |
+
"CUSTOMER_SUPPORT#positive",
|
| 24 |
+
"DISCOUNT#negative",
|
| 25 |
+
"DISCOUNT#neutral",
|
| 26 |
+
"DISCOUNT#positive",
|
| 27 |
+
"INTEREST_RATE#negative",
|
| 28 |
+
"INTEREST_RATE#neutral",
|
| 29 |
+
"INTEREST_RATE#positive",
|
| 30 |
+
"INTERNET_BANKING#negative",
|
| 31 |
+
"INTERNET_BANKING#neutral",
|
| 32 |
+
"INTERNET_BANKING#positive",
|
| 33 |
+
"LOAN#negative",
|
| 34 |
+
"LOAN#positive",
|
| 35 |
+
"MONEY_TRANSFER#negative",
|
| 36 |
+
"MONEY_TRANSFER#positive",
|
| 37 |
+
"OTHER#negative",
|
| 38 |
+
"OTHER#neutral",
|
| 39 |
+
"OTHER#positive",
|
| 40 |
+
"PAYMENT#negative",
|
| 41 |
+
"PAYMENT#positive",
|
| 42 |
+
"PROMOTION#negative",
|
| 43 |
+
"PROMOTION#neutral",
|
| 44 |
+
"PROMOTION#positive",
|
| 45 |
+
"SAVING#negative",
|
| 46 |
+
"SAVING#neutral",
|
| 47 |
+
"SAVING#positive",
|
| 48 |
+
"SECURITY#neutral",
|
| 49 |
+
"SECURITY#positive",
|
| 50 |
+
"TRADEMARK#negative",
|
| 51 |
+
"TRADEMARK#positive"
|
| 52 |
+
],
|
| 53 |
+
"train_accuracy": 0.9430740037950665,
|
| 54 |
+
"test_accuracy": 0.7171717171717171,
|
| 55 |
+
"train_time": 7.737863779067993,
|
| 56 |
+
"prediction_time": 0.1107940673828125,
|
| 57 |
+
"classification_report": {
|
| 58 |
+
"ACCOUNT#negative": {
|
| 59 |
+
"precision": 0.0,
|
| 60 |
+
"recall": 0.0,
|
| 61 |
+
"f1-score": 0.0,
|
| 62 |
+
"support": 2.0
|
| 63 |
+
},
|
| 64 |
+
"CARD#negative": {
|
| 65 |
+
"precision": 1.0,
|
| 66 |
+
"recall": 0.14285714285714285,
|
| 67 |
+
"f1-score": 0.25,
|
| 68 |
+
"support": 7.0
|
| 69 |
+
},
|
| 70 |
+
"CARD#positive": {
|
| 71 |
+
"precision": 0.0,
|
| 72 |
+
"recall": 0.0,
|
| 73 |
+
"f1-score": 0.0,
|
| 74 |
+
"support": 1.0
|
| 75 |
+
},
|
| 76 |
+
"CUSTOMER_SUPPORT#negative": {
|
| 77 |
+
"precision": 0.4931506849315068,
|
| 78 |
+
"recall": 0.96,
|
| 79 |
+
"f1-score": 0.6515837104072398,
|
| 80 |
+
"support": 75.0
|
| 81 |
+
},
|
| 82 |
+
"CUSTOMER_SUPPORT#neutral": {
|
| 83 |
+
"precision": 0.0,
|
| 84 |
+
"recall": 0.0,
|
| 85 |
+
"f1-score": 0.0,
|
| 86 |
+
"support": 1.0
|
| 87 |
+
},
|
| 88 |
+
"CUSTOMER_SUPPORT#positive": {
|
| 89 |
+
"precision": 0.8641975308641975,
|
| 90 |
+
"recall": 0.8974358974358975,
|
| 91 |
+
"f1-score": 0.8805031446540881,
|
| 92 |
+
"support": 78.0
|
| 93 |
+
},
|
| 94 |
+
"DISCOUNT#negative": {
|
| 95 |
+
"precision": 0.0,
|
| 96 |
+
"recall": 0.0,
|
| 97 |
+
"f1-score": 0.0,
|
| 98 |
+
"support": 5.0
|
| 99 |
+
},
|
| 100 |
+
"DISCOUNT#neutral": {
|
| 101 |
+
"precision": 0.0,
|
| 102 |
+
"recall": 0.0,
|
| 103 |
+
"f1-score": 0.0,
|
| 104 |
+
"support": 1.0
|
| 105 |
+
},
|
| 106 |
+
"DISCOUNT#positive": {
|
| 107 |
+
"precision": 0.0,
|
| 108 |
+
"recall": 0.0,
|
| 109 |
+
"f1-score": 0.0,
|
| 110 |
+
"support": 0.0
|
| 111 |
+
},
|
| 112 |
+
"INTEREST_RATE#negative": {
|
| 113 |
+
"precision": 0.5555555555555556,
|
| 114 |
+
"recall": 0.5,
|
| 115 |
+
"f1-score": 0.5263157894736842,
|
| 116 |
+
"support": 10.0
|
| 117 |
+
},
|
| 118 |
+
"INTERNET_BANKING#negative": {
|
| 119 |
+
"precision": 0.0,
|
| 120 |
+
"recall": 0.0,
|
| 121 |
+
"f1-score": 0.0,
|
| 122 |
+
"support": 12.0
|
| 123 |
+
},
|
| 124 |
+
"INTERNET_BANKING#positive": {
|
| 125 |
+
"precision": 0.0,
|
| 126 |
+
"recall": 0.0,
|
| 127 |
+
"f1-score": 0.0,
|
| 128 |
+
"support": 1.0
|
| 129 |
+
},
|
| 130 |
+
"LOAN#negative": {
|
| 131 |
+
"precision": 0.875,
|
| 132 |
+
"recall": 0.5384615384615384,
|
| 133 |
+
"f1-score": 0.6666666666666666,
|
| 134 |
+
"support": 13.0
|
| 135 |
+
},
|
| 136 |
+
"MONEY_TRANSFER#negative": {
|
| 137 |
+
"precision": 0.0,
|
| 138 |
+
"recall": 0.0,
|
| 139 |
+
"f1-score": 0.0,
|
| 140 |
+
"support": 5.0
|
| 141 |
+
},
|
| 142 |
+
"OTHER#negative": {
|
| 143 |
+
"precision": 0.0,
|
| 144 |
+
"recall": 0.0,
|
| 145 |
+
"f1-score": 0.0,
|
| 146 |
+
"support": 10.0
|
| 147 |
+
},
|
| 148 |
+
"OTHER#neutral": {
|
| 149 |
+
"precision": 0.0,
|
| 150 |
+
"recall": 0.0,
|
| 151 |
+
"f1-score": 0.0,
|
| 152 |
+
"support": 1.0
|
| 153 |
+
},
|
| 154 |
+
"OTHER#positive": {
|
| 155 |
+
"precision": 0.6666666666666666,
|
| 156 |
+
"recall": 0.5,
|
| 157 |
+
"f1-score": 0.5714285714285714,
|
| 158 |
+
"support": 4.0
|
| 159 |
+
},
|
| 160 |
+
"PAYMENT#positive": {
|
| 161 |
+
"precision": 0.0,
|
| 162 |
+
"recall": 0.0,
|
| 163 |
+
"f1-score": 0.0,
|
| 164 |
+
"support": 3.0
|
| 165 |
+
},
|
| 166 |
+
"PROMOTION#negative": {
|
| 167 |
+
"precision": 0.0,
|
| 168 |
+
"recall": 0.0,
|
| 169 |
+
"f1-score": 0.0,
|
| 170 |
+
"support": 4.0
|
| 171 |
+
},
|
| 172 |
+
"PROMOTION#neutral": {
|
| 173 |
+
"precision": 0.0,
|
| 174 |
+
"recall": 0.0,
|
| 175 |
+
"f1-score": 0.0,
|
| 176 |
+
"support": 1.0
|
| 177 |
+
},
|
| 178 |
+
"PROMOTION#positive": {
|
| 179 |
+
"precision": 0.3333333333333333,
|
| 180 |
+
"recall": 0.2,
|
| 181 |
+
"f1-score": 0.25,
|
| 182 |
+
"support": 5.0
|
| 183 |
+
},
|
| 184 |
+
"SAVING#negative": {
|
| 185 |
+
"precision": 0.0,
|
| 186 |
+
"recall": 0.0,
|
| 187 |
+
"f1-score": 0.0,
|
| 188 |
+
"support": 1.0
|
| 189 |
+
},
|
| 190 |
+
"SAVING#positive": {
|
| 191 |
+
"precision": 0.0,
|
| 192 |
+
"recall": 0.0,
|
| 193 |
+
"f1-score": 0.0,
|
| 194 |
+
"support": 2.0
|
| 195 |
+
},
|
| 196 |
+
"SECURITY#negative": {
|
| 197 |
+
"precision": 0.0,
|
| 198 |
+
"recall": 0.0,
|
| 199 |
+
"f1-score": 0.0,
|
| 200 |
+
"support": 1.0
|
| 201 |
+
},
|
| 202 |
+
"SECURITY#neutral": {
|
| 203 |
+
"precision": 0.0,
|
| 204 |
+
"recall": 0.0,
|
| 205 |
+
"f1-score": 0.0,
|
| 206 |
+
"support": 1.0
|
| 207 |
+
},
|
| 208 |
+
"TRADEMARK#negative": {
|
| 209 |
+
"precision": 0.0,
|
| 210 |
+
"recall": 0.0,
|
| 211 |
+
"f1-score": 0.0,
|
| 212 |
+
"support": 14.0
|
| 213 |
+
},
|
| 214 |
+
"TRADEMARK#positive": {
|
| 215 |
+
"precision": 0.8936170212765957,
|
| 216 |
+
"recall": 0.9130434782608695,
|
| 217 |
+
"f1-score": 0.9032258064516129,
|
| 218 |
+
"support": 138.0
|
| 219 |
+
},
|
| 220 |
+
"accuracy": 0.7171717171717171,
|
| 221 |
+
"macro avg": {
|
| 222 |
+
"precision": 0.21042669602325392,
|
| 223 |
+
"recall": 0.17228881692649806,
|
| 224 |
+
"f1-score": 0.1740638403363653,
|
| 225 |
+
"support": 396.0
|
| 226 |
+
},
|
| 227 |
+
"weighted avg": {
|
| 228 |
+
"precision": 0.6464059257634583,
|
| 229 |
+
"recall": 0.7171717171717171,
|
| 230 |
+
"f1-score": 0.6601230396489957,
|
| 231 |
+
"support": 396.0
|
| 232 |
+
}
|
| 233 |
+
},
|
| 234 |
+
"confusion_matrix": [
|
| 235 |
+
[
|
| 236 |
+
0,
|
| 237 |
+
0,
|
| 238 |
+
0,
|
| 239 |
+
0,
|
| 240 |
+
1,
|
| 241 |
+
0,
|
| 242 |
+
1,
|
| 243 |
+
0,
|
| 244 |
+
0,
|
| 245 |
+
0,
|
| 246 |
+
0,
|
| 247 |
+
0,
|
| 248 |
+
0,
|
| 249 |
+
0,
|
| 250 |
+
0,
|
| 251 |
+
0,
|
| 252 |
+
0,
|
| 253 |
+
0,
|
| 254 |
+
0,
|
| 255 |
+
0,
|
| 256 |
+
0,
|
| 257 |
+
0,
|
| 258 |
+
0,
|
| 259 |
+
0,
|
| 260 |
+
0,
|
| 261 |
+
0,
|
| 262 |
+
0,
|
| 263 |
+
0,
|
| 264 |
+
0,
|
| 265 |
+
0,
|
| 266 |
+
0,
|
| 267 |
+
0,
|
| 268 |
+
0,
|
| 269 |
+
0,
|
| 270 |
+
0
|
| 271 |
+
],
|
| 272 |
+
[
|
| 273 |
+
0,
|
| 274 |
+
1,
|
| 275 |
+
0,
|
| 276 |
+
0,
|
| 277 |
+
5,
|
| 278 |
+
0,
|
| 279 |
+
0,
|
| 280 |
+
0,
|
| 281 |
+
0,
|
| 282 |
+
0,
|
| 283 |
+
1,
|
| 284 |
+
0,
|
| 285 |
+
0,
|
| 286 |
+
0,
|
| 287 |
+
0,
|
| 288 |
+
0,
|
| 289 |
+
0,
|
| 290 |
+
0,
|
| 291 |
+
0,
|
| 292 |
+
0,
|
| 293 |
+
0,
|
| 294 |
+
0,
|
| 295 |
+
0,
|
| 296 |
+
0,
|
| 297 |
+
0,
|
| 298 |
+
0,
|
| 299 |
+
0,
|
| 300 |
+
0,
|
| 301 |
+
0,
|
| 302 |
+
0,
|
| 303 |
+
0,
|
| 304 |
+
0,
|
| 305 |
+
0,
|
| 306 |
+
0,
|
| 307 |
+
0
|
| 308 |
+
],
|
| 309 |
+
[
|
| 310 |
+
0,
|
| 311 |
+
0,
|
| 312 |
+
0,
|
| 313 |
+
0,
|
| 314 |
+
0,
|
| 315 |
+
0,
|
| 316 |
+
0,
|
| 317 |
+
0,
|
| 318 |
+
0,
|
| 319 |
+
0,
|
| 320 |
+
0,
|
| 321 |
+
0,
|
| 322 |
+
0,
|
| 323 |
+
0,
|
| 324 |
+
0,
|
| 325 |
+
0,
|
| 326 |
+
0,
|
| 327 |
+
0,
|
| 328 |
+
0,
|
| 329 |
+
0,
|
| 330 |
+
0,
|
| 331 |
+
0,
|
| 332 |
+
0,
|
| 333 |
+
0,
|
| 334 |
+
0,
|
| 335 |
+
0,
|
| 336 |
+
0,
|
| 337 |
+
0,
|
| 338 |
+
0,
|
| 339 |
+
0,
|
| 340 |
+
0,
|
| 341 |
+
0,
|
| 342 |
+
0,
|
| 343 |
+
0,
|
| 344 |
+
0
|
| 345 |
+
],
|
| 346 |
+
[
|
| 347 |
+
0,
|
| 348 |
+
0,
|
| 349 |
+
0,
|
| 350 |
+
0,
|
| 351 |
+
0,
|
| 352 |
+
0,
|
| 353 |
+
0,
|
| 354 |
+
0,
|
| 355 |
+
0,
|
| 356 |
+
0,
|
| 357 |
+
0,
|
| 358 |
+
0,
|
| 359 |
+
0,
|
| 360 |
+
0,
|
| 361 |
+
0,
|
| 362 |
+
0,
|
| 363 |
+
0,
|
| 364 |
+
0,
|
| 365 |
+
0,
|
| 366 |
+
0,
|
| 367 |
+
0,
|
| 368 |
+
0,
|
| 369 |
+
0,
|
| 370 |
+
0,
|
| 371 |
+
0,
|
| 372 |
+
0,
|
| 373 |
+
0,
|
| 374 |
+
0,
|
| 375 |
+
0,
|
| 376 |
+
0,
|
| 377 |
+
0,
|
| 378 |
+
0,
|
| 379 |
+
0,
|
| 380 |
+
0,
|
| 381 |
+
1
|
| 382 |
+
],
|
| 383 |
+
[
|
| 384 |
+
0,
|
| 385 |
+
0,
|
| 386 |
+
0,
|
| 387 |
+
0,
|
| 388 |
+
72,
|
| 389 |
+
0,
|
| 390 |
+
1,
|
| 391 |
+
0,
|
| 392 |
+
0,
|
| 393 |
+
0,
|
| 394 |
+
1,
|
| 395 |
+
0,
|
| 396 |
+
0,
|
| 397 |
+
0,
|
| 398 |
+
0,
|
| 399 |
+
0,
|
| 400 |
+
1,
|
| 401 |
+
0,
|
| 402 |
+
0,
|
| 403 |
+
0,
|
| 404 |
+
0,
|
| 405 |
+
0,
|
| 406 |
+
0,
|
| 407 |
+
0,
|
| 408 |
+
0,
|
| 409 |
+
0,
|
| 410 |
+
0,
|
| 411 |
+
0,
|
| 412 |
+
0,
|
| 413 |
+
0,
|
| 414 |
+
0,
|
| 415 |
+
0,
|
| 416 |
+
0,
|
| 417 |
+
0,
|
| 418 |
+
0
|
| 419 |
+
],
|
| 420 |
+
[
|
| 421 |
+
0,
|
| 422 |
+
0,
|
| 423 |
+
0,
|
| 424 |
+
0,
|
| 425 |
+
1,
|
| 426 |
+
0,
|
| 427 |
+
0,
|
| 428 |
+
0,
|
| 429 |
+
0,
|
| 430 |
+
0,
|
| 431 |
+
0,
|
| 432 |
+
0,
|
| 433 |
+
0,
|
| 434 |
+
0,
|
| 435 |
+
0,
|
| 436 |
+
0,
|
| 437 |
+
0,
|
| 438 |
+
0,
|
| 439 |
+
0,
|
| 440 |
+
0,
|
| 441 |
+
0,
|
| 442 |
+
0,
|
| 443 |
+
0,
|
| 444 |
+
0,
|
| 445 |
+
0,
|
| 446 |
+
0,
|
| 447 |
+
0,
|
| 448 |
+
0,
|
| 449 |
+
0,
|
| 450 |
+
0,
|
| 451 |
+
0,
|
| 452 |
+
0,
|
| 453 |
+
0,
|
| 454 |
+
0,
|
| 455 |
+
0
|
| 456 |
+
],
|
| 457 |
+
[
|
| 458 |
+
0,
|
| 459 |
+
0,
|
| 460 |
+
0,
|
| 461 |
+
0,
|
| 462 |
+
6,
|
| 463 |
+
0,
|
| 464 |
+
70,
|
| 465 |
+
0,
|
| 466 |
+
0,
|
| 467 |
+
0,
|
| 468 |
+
0,
|
| 469 |
+
0,
|
| 470 |
+
0,
|
| 471 |
+
0,
|
| 472 |
+
0,
|
| 473 |
+
0,
|
| 474 |
+
0,
|
| 475 |
+
0,
|
| 476 |
+
0,
|
| 477 |
+
0,
|
| 478 |
+
0,
|
| 479 |
+
0,
|
| 480 |
+
0,
|
| 481 |
+
0,
|
| 482 |
+
0,
|
| 483 |
+
0,
|
| 484 |
+
0,
|
| 485 |
+
0,
|
| 486 |
+
0,
|
| 487 |
+
0,
|
| 488 |
+
0,
|
| 489 |
+
0,
|
| 490 |
+
0,
|
| 491 |
+
0,
|
| 492 |
+
2
|
| 493 |
+
],
|
| 494 |
+
[
|
| 495 |
+
0,
|
| 496 |
+
0,
|
| 497 |
+
0,
|
| 498 |
+
0,
|
| 499 |
+
5,
|
| 500 |
+
0,
|
| 501 |
+
0,
|
| 502 |
+
0,
|
| 503 |
+
0,
|
| 504 |
+
0,
|
| 505 |
+
0,
|
| 506 |
+
0,
|
| 507 |
+
0,
|
| 508 |
+
0,
|
| 509 |
+
0,
|
| 510 |
+
0,
|
| 511 |
+
0,
|
| 512 |
+
0,
|
| 513 |
+
0,
|
| 514 |
+
0,
|
| 515 |
+
0,
|
| 516 |
+
0,
|
| 517 |
+
0,
|
| 518 |
+
0,
|
| 519 |
+
0,
|
| 520 |
+
0,
|
| 521 |
+
0,
|
| 522 |
+
0,
|
| 523 |
+
0,
|
| 524 |
+
0,
|
| 525 |
+
0,
|
| 526 |
+
0,
|
| 527 |
+
0,
|
| 528 |
+
0,
|
| 529 |
+
0
|
| 530 |
+
],
|
| 531 |
+
[
|
| 532 |
+
0,
|
| 533 |
+
0,
|
| 534 |
+
0,
|
| 535 |
+
0,
|
| 536 |
+
1,
|
| 537 |
+
0,
|
| 538 |
+
0,
|
| 539 |
+
0,
|
| 540 |
+
0,
|
| 541 |
+
0,
|
| 542 |
+
0,
|
| 543 |
+
0,
|
| 544 |
+
0,
|
| 545 |
+
0,
|
| 546 |
+
0,
|
| 547 |
+
0,
|
| 548 |
+
0,
|
| 549 |
+
0,
|
| 550 |
+
0,
|
| 551 |
+
0,
|
| 552 |
+
0,
|
| 553 |
+
0,
|
| 554 |
+
0,
|
| 555 |
+
0,
|
| 556 |
+
0,
|
| 557 |
+
0,
|
| 558 |
+
0,
|
| 559 |
+
0,
|
| 560 |
+
0,
|
| 561 |
+
0,
|
| 562 |
+
0,
|
| 563 |
+
0,
|
| 564 |
+
0,
|
| 565 |
+
0,
|
| 566 |
+
0
|
| 567 |
+
],
|
| 568 |
+
[
|
| 569 |
+
0,
|
| 570 |
+
0,
|
| 571 |
+
0,
|
| 572 |
+
0,
|
| 573 |
+
0,
|
| 574 |
+
0,
|
| 575 |
+
0,
|
| 576 |
+
0,
|
| 577 |
+
0,
|
| 578 |
+
0,
|
| 579 |
+
0,
|
| 580 |
+
0,
|
| 581 |
+
0,
|
| 582 |
+
0,
|
| 583 |
+
0,
|
| 584 |
+
0,
|
| 585 |
+
0,
|
| 586 |
+
0,
|
| 587 |
+
0,
|
| 588 |
+
0,
|
| 589 |
+
0,
|
| 590 |
+
0,
|
| 591 |
+
0,
|
| 592 |
+
0,
|
| 593 |
+
0,
|
| 594 |
+
0,
|
| 595 |
+
0,
|
| 596 |
+
0,
|
| 597 |
+
0,
|
| 598 |
+
0,
|
| 599 |
+
0,
|
| 600 |
+
0,
|
| 601 |
+
0,
|
| 602 |
+
0,
|
| 603 |
+
0
|
| 604 |
+
],
|
| 605 |
+
[
|
| 606 |
+
0,
|
| 607 |
+
0,
|
| 608 |
+
0,
|
| 609 |
+
0,
|
| 610 |
+
4,
|
| 611 |
+
0,
|
| 612 |
+
0,
|
| 613 |
+
0,
|
| 614 |
+
0,
|
| 615 |
+
0,
|
| 616 |
+
5,
|
| 617 |
+
0,
|
| 618 |
+
0,
|
| 619 |
+
0,
|
| 620 |
+
0,
|
| 621 |
+
0,
|
| 622 |
+
0,
|
| 623 |
+
0,
|
| 624 |
+
0,
|
| 625 |
+
0,
|
| 626 |
+
0,
|
| 627 |
+
0,
|
| 628 |
+
0,
|
| 629 |
+
0,
|
| 630 |
+
0,
|
| 631 |
+
0,
|
| 632 |
+
0,
|
| 633 |
+
0,
|
| 634 |
+
0,
|
| 635 |
+
0,
|
| 636 |
+
0,
|
| 637 |
+
0,
|
| 638 |
+
0,
|
| 639 |
+
0,
|
| 640 |
+
1
|
| 641 |
+
],
|
| 642 |
+
[
|
| 643 |
+
0,
|
| 644 |
+
0,
|
| 645 |
+
0,
|
| 646 |
+
0,
|
| 647 |
+
0,
|
| 648 |
+
0,
|
| 649 |
+
0,
|
| 650 |
+
0,
|
| 651 |
+
0,
|
| 652 |
+
0,
|
| 653 |
+
0,
|
| 654 |
+
0,
|
| 655 |
+
0,
|
| 656 |
+
0,
|
| 657 |
+
0,
|
| 658 |
+
0,
|
| 659 |
+
0,
|
| 660 |
+
0,
|
| 661 |
+
0,
|
| 662 |
+
0,
|
| 663 |
+
0,
|
| 664 |
+
0,
|
| 665 |
+
0,
|
| 666 |
+
0,
|
| 667 |
+
0,
|
| 668 |
+
0,
|
| 669 |
+
0,
|
| 670 |
+
0,
|
| 671 |
+
0,
|
| 672 |
+
0,
|
| 673 |
+
0,
|
| 674 |
+
0,
|
| 675 |
+
0,
|
| 676 |
+
0,
|
| 677 |
+
0
|
| 678 |
+
],
|
| 679 |
+
[
|
| 680 |
+
0,
|
| 681 |
+
0,
|
| 682 |
+
0,
|
| 683 |
+
0,
|
| 684 |
+
0,
|
| 685 |
+
0,
|
| 686 |
+
0,
|
| 687 |
+
0,
|
| 688 |
+
0,
|
| 689 |
+
0,
|
| 690 |
+
0,
|
| 691 |
+
0,
|
| 692 |
+
0,
|
| 693 |
+
0,
|
| 694 |
+
0,
|
| 695 |
+
0,
|
| 696 |
+
0,
|
| 697 |
+
0,
|
| 698 |
+
0,
|
| 699 |
+
0,
|
| 700 |
+
0,
|
| 701 |
+
0,
|
| 702 |
+
0,
|
| 703 |
+
0,
|
| 704 |
+
0,
|
| 705 |
+
0,
|
| 706 |
+
0,
|
| 707 |
+
0,
|
| 708 |
+
0,
|
| 709 |
+
0,
|
| 710 |
+
0,
|
| 711 |
+
0,
|
| 712 |
+
0,
|
| 713 |
+
0,
|
| 714 |
+
0
|
| 715 |
+
],
|
| 716 |
+
[
|
| 717 |
+
0,
|
| 718 |
+
0,
|
| 719 |
+
0,
|
| 720 |
+
0,
|
| 721 |
+
12,
|
| 722 |
+
0,
|
| 723 |
+
0,
|
| 724 |
+
0,
|
| 725 |
+
0,
|
| 726 |
+
0,
|
| 727 |
+
0,
|
| 728 |
+
0,
|
| 729 |
+
0,
|
| 730 |
+
0,
|
| 731 |
+
0,
|
| 732 |
+
0,
|
| 733 |
+
0,
|
| 734 |
+
0,
|
| 735 |
+
0,
|
| 736 |
+
0,
|
| 737 |
+
0,
|
| 738 |
+
0,
|
| 739 |
+
0,
|
| 740 |
+
0,
|
| 741 |
+
0,
|
| 742 |
+
0,
|
| 743 |
+
0,
|
| 744 |
+
0,
|
| 745 |
+
0,
|
| 746 |
+
0,
|
| 747 |
+
0,
|
| 748 |
+
0,
|
| 749 |
+
0,
|
| 750 |
+
0,
|
| 751 |
+
0
|
| 752 |
+
],
|
| 753 |
+
[
|
| 754 |
+
0,
|
| 755 |
+
0,
|
| 756 |
+
0,
|
| 757 |
+
0,
|
| 758 |
+
0,
|
| 759 |
+
0,
|
| 760 |
+
0,
|
| 761 |
+
0,
|
| 762 |
+
0,
|
| 763 |
+
0,
|
| 764 |
+
0,
|
| 765 |
+
0,
|
| 766 |
+
0,
|
| 767 |
+
0,
|
| 768 |
+
0,
|
| 769 |
+
0,
|
| 770 |
+
0,
|
| 771 |
+
0,
|
| 772 |
+
0,
|
| 773 |
+
0,
|
| 774 |
+
0,
|
| 775 |
+
0,
|
| 776 |
+
0,
|
| 777 |
+
0,
|
| 778 |
+
0,
|
| 779 |
+
0,
|
| 780 |
+
0,
|
| 781 |
+
0,
|
| 782 |
+
0,
|
| 783 |
+
0,
|
| 784 |
+
0,
|
| 785 |
+
0,
|
| 786 |
+
0,
|
| 787 |
+
0,
|
| 788 |
+
0
|
| 789 |
+
],
|
| 790 |
+
[
|
| 791 |
+
0,
|
| 792 |
+
0,
|
| 793 |
+
0,
|
| 794 |
+
0,
|
| 795 |
+
1,
|
| 796 |
+
0,
|
| 797 |
+
0,
|
| 798 |
+
0,
|
| 799 |
+
0,
|
| 800 |
+
0,
|
| 801 |
+
0,
|
| 802 |
+
0,
|
| 803 |
+
0,
|
| 804 |
+
0,
|
| 805 |
+
0,
|
| 806 |
+
0,
|
| 807 |
+
0,
|
| 808 |
+
0,
|
| 809 |
+
0,
|
| 810 |
+
0,
|
| 811 |
+
0,
|
| 812 |
+
0,
|
| 813 |
+
0,
|
| 814 |
+
0,
|
| 815 |
+
0,
|
| 816 |
+
0,
|
| 817 |
+
0,
|
| 818 |
+
0,
|
| 819 |
+
0,
|
| 820 |
+
0,
|
| 821 |
+
0,
|
| 822 |
+
0,
|
| 823 |
+
0,
|
| 824 |
+
0,
|
| 825 |
+
0
|
| 826 |
+
],
|
| 827 |
+
[
|
| 828 |
+
0,
|
| 829 |
+
0,
|
| 830 |
+
0,
|
| 831 |
+
0,
|
| 832 |
+
4,
|
| 833 |
+
0,
|
| 834 |
+
0,
|
| 835 |
+
0,
|
| 836 |
+
0,
|
| 837 |
+
0,
|
| 838 |
+
0,
|
| 839 |
+
0,
|
| 840 |
+
0,
|
| 841 |
+
0,
|
| 842 |
+
0,
|
| 843 |
+
0,
|
| 844 |
+
7,
|
| 845 |
+
0,
|
| 846 |
+
0,
|
| 847 |
+
0,
|
| 848 |
+
0,
|
| 849 |
+
0,
|
| 850 |
+
0,
|
| 851 |
+
0,
|
| 852 |
+
0,
|
| 853 |
+
0,
|
| 854 |
+
0,
|
| 855 |
+
0,
|
| 856 |
+
0,
|
| 857 |
+
0,
|
| 858 |
+
0,
|
| 859 |
+
0,
|
| 860 |
+
0,
|
| 861 |
+
2,
|
| 862 |
+
0
|
| 863 |
+
],
|
| 864 |
+
[
|
| 865 |
+
0,
|
| 866 |
+
0,
|
| 867 |
+
0,
|
| 868 |
+
0,
|
| 869 |
+
0,
|
| 870 |
+
0,
|
| 871 |
+
0,
|
| 872 |
+
0,
|
| 873 |
+
0,
|
| 874 |
+
0,
|
| 875 |
+
0,
|
| 876 |
+
0,
|
| 877 |
+
0,
|
| 878 |
+
0,
|
| 879 |
+
0,
|
| 880 |
+
0,
|
| 881 |
+
0,
|
| 882 |
+
0,
|
| 883 |
+
0,
|
| 884 |
+
0,
|
| 885 |
+
0,
|
| 886 |
+
0,
|
| 887 |
+
0,
|
| 888 |
+
0,
|
| 889 |
+
0,
|
| 890 |
+
0,
|
| 891 |
+
0,
|
| 892 |
+
0,
|
| 893 |
+
0,
|
| 894 |
+
0,
|
| 895 |
+
0,
|
| 896 |
+
0,
|
| 897 |
+
0,
|
| 898 |
+
0,
|
| 899 |
+
0
|
| 900 |
+
],
|
| 901 |
+
[
|
| 902 |
+
0,
|
| 903 |
+
0,
|
| 904 |
+
0,
|
| 905 |
+
0,
|
| 906 |
+
3,
|
| 907 |
+
0,
|
| 908 |
+
0,
|
| 909 |
+
0,
|
| 910 |
+
0,
|
| 911 |
+
0,
|
| 912 |
+
2,
|
| 913 |
+
0,
|
| 914 |
+
0,
|
| 915 |
+
0,
|
| 916 |
+
0,
|
| 917 |
+
0,
|
| 918 |
+
0,
|
| 919 |
+
0,
|
| 920 |
+
0,
|
| 921 |
+
0,
|
| 922 |
+
0,
|
| 923 |
+
0,
|
| 924 |
+
0,
|
| 925 |
+
0,
|
| 926 |
+
0,
|
| 927 |
+
0,
|
| 928 |
+
0,
|
| 929 |
+
0,
|
| 930 |
+
0,
|
| 931 |
+
0,
|
| 932 |
+
0,
|
| 933 |
+
0,
|
| 934 |
+
0,
|
| 935 |
+
0,
|
| 936 |
+
0
|
| 937 |
+
],
|
| 938 |
+
[
|
| 939 |
+
0,
|
| 940 |
+
0,
|
| 941 |
+
0,
|
| 942 |
+
0,
|
| 943 |
+
0,
|
| 944 |
+
0,
|
| 945 |
+
0,
|
| 946 |
+
0,
|
| 947 |
+
0,
|
| 948 |
+
0,
|
| 949 |
+
0,
|
| 950 |
+
0,
|
| 951 |
+
0,
|
| 952 |
+
0,
|
| 953 |
+
0,
|
| 954 |
+
0,
|
| 955 |
+
0,
|
| 956 |
+
0,
|
| 957 |
+
0,
|
| 958 |
+
0,
|
| 959 |
+
0,
|
| 960 |
+
0,
|
| 961 |
+
0,
|
| 962 |
+
0,
|
| 963 |
+
0,
|
| 964 |
+
0,
|
| 965 |
+
0,
|
| 966 |
+
0,
|
| 967 |
+
0,
|
| 968 |
+
0,
|
| 969 |
+
0,
|
| 970 |
+
0,
|
| 971 |
+
0,
|
| 972 |
+
0,
|
| 973 |
+
0
|
| 974 |
+
],
|
| 975 |
+
[
|
| 976 |
+
0,
|
| 977 |
+
0,
|
| 978 |
+
0,
|
| 979 |
+
0,
|
| 980 |
+
8,
|
| 981 |
+
0,
|
| 982 |
+
0,
|
| 983 |
+
0,
|
| 984 |
+
0,
|
| 985 |
+
1,
|
| 986 |
+
0,
|
| 987 |
+
0,
|
| 988 |
+
0,
|
| 989 |
+
0,
|
| 990 |
+
0,
|
| 991 |
+
0,
|
| 992 |
+
0,
|
| 993 |
+
0,
|
| 994 |
+
0,
|
| 995 |
+
0,
|
| 996 |
+
0,
|
| 997 |
+
0,
|
| 998 |
+
0,
|
| 999 |
+
0,
|
| 1000 |
+
0,
|
| 1001 |
+
0,
|
| 1002 |
+
0,
|
| 1003 |
+
0,
|
| 1004 |
+
0,
|
| 1005 |
+
0,
|
| 1006 |
+
0,
|
| 1007 |
+
0,
|
| 1008 |
+
0,
|
| 1009 |
+
1,
|
| 1010 |
+
0
|
| 1011 |
+
],
|
| 1012 |
+
[
|
| 1013 |
+
0,
|
| 1014 |
+
0,
|
| 1015 |
+
0,
|
| 1016 |
+
0,
|
| 1017 |
+
1,
|
| 1018 |
+
0,
|
| 1019 |
+
0,
|
| 1020 |
+
0,
|
| 1021 |
+
0,
|
| 1022 |
+
0,
|
| 1023 |
+
0,
|
| 1024 |
+
0,
|
| 1025 |
+
0,
|
| 1026 |
+
0,
|
| 1027 |
+
0,
|
| 1028 |
+
0,
|
| 1029 |
+
0,
|
| 1030 |
+
0,
|
| 1031 |
+
0,
|
| 1032 |
+
0,
|
| 1033 |
+
0,
|
| 1034 |
+
0,
|
| 1035 |
+
0,
|
| 1036 |
+
0,
|
| 1037 |
+
0,
|
| 1038 |
+
0,
|
| 1039 |
+
0,
|
| 1040 |
+
0,
|
| 1041 |
+
0,
|
| 1042 |
+
0,
|
| 1043 |
+
0,
|
| 1044 |
+
0,
|
| 1045 |
+
0,
|
| 1046 |
+
0,
|
| 1047 |
+
0
|
| 1048 |
+
],
|
| 1049 |
+
[
|
| 1050 |
+
0,
|
| 1051 |
+
0,
|
| 1052 |
+
0,
|
| 1053 |
+
0,
|
| 1054 |
+
1,
|
| 1055 |
+
0,
|
| 1056 |
+
0,
|
| 1057 |
+
0,
|
| 1058 |
+
0,
|
| 1059 |
+
0,
|
| 1060 |
+
0,
|
| 1061 |
+
0,
|
| 1062 |
+
0,
|
| 1063 |
+
0,
|
| 1064 |
+
0,
|
| 1065 |
+
0,
|
| 1066 |
+
0,
|
| 1067 |
+
0,
|
| 1068 |
+
0,
|
| 1069 |
+
0,
|
| 1070 |
+
0,
|
| 1071 |
+
0,
|
| 1072 |
+
2,
|
| 1073 |
+
0,
|
| 1074 |
+
0,
|
| 1075 |
+
0,
|
| 1076 |
+
0,
|
| 1077 |
+
0,
|
| 1078 |
+
0,
|
| 1079 |
+
0,
|
| 1080 |
+
0,
|
| 1081 |
+
0,
|
| 1082 |
+
0,
|
| 1083 |
+
0,
|
| 1084 |
+
1
|
| 1085 |
+
],
|
| 1086 |
+
[
|
| 1087 |
+
0,
|
| 1088 |
+
0,
|
| 1089 |
+
0,
|
| 1090 |
+
0,
|
| 1091 |
+
0,
|
| 1092 |
+
0,
|
| 1093 |
+
0,
|
| 1094 |
+
0,
|
| 1095 |
+
0,
|
| 1096 |
+
0,
|
| 1097 |
+
0,
|
| 1098 |
+
0,
|
| 1099 |
+
0,
|
| 1100 |
+
0,
|
| 1101 |
+
0,
|
| 1102 |
+
0,
|
| 1103 |
+
0,
|
| 1104 |
+
0,
|
| 1105 |
+
0,
|
| 1106 |
+
0,
|
| 1107 |
+
0,
|
| 1108 |
+
0,
|
| 1109 |
+
0,
|
| 1110 |
+
0,
|
| 1111 |
+
0,
|
| 1112 |
+
0,
|
| 1113 |
+
0,
|
| 1114 |
+
0,
|
| 1115 |
+
0,
|
| 1116 |
+
0,
|
| 1117 |
+
0,
|
| 1118 |
+
0,
|
| 1119 |
+
0,
|
| 1120 |
+
0,
|
| 1121 |
+
0
|
| 1122 |
+
],
|
| 1123 |
+
[
|
| 1124 |
+
0,
|
| 1125 |
+
0,
|
| 1126 |
+
0,
|
| 1127 |
+
0,
|
| 1128 |
+
0,
|
| 1129 |
+
0,
|
| 1130 |
+
2,
|
| 1131 |
+
0,
|
| 1132 |
+
0,
|
| 1133 |
+
0,
|
| 1134 |
+
0,
|
| 1135 |
+
0,
|
| 1136 |
+
0,
|
| 1137 |
+
0,
|
| 1138 |
+
0,
|
| 1139 |
+
0,
|
| 1140 |
+
0,
|
| 1141 |
+
0,
|
| 1142 |
+
0,
|
| 1143 |
+
0,
|
| 1144 |
+
0,
|
| 1145 |
+
0,
|
| 1146 |
+
0,
|
| 1147 |
+
0,
|
| 1148 |
+
0,
|
| 1149 |
+
0,
|
| 1150 |
+
0,
|
| 1151 |
+
0,
|
| 1152 |
+
0,
|
| 1153 |
+
0,
|
| 1154 |
+
0,
|
| 1155 |
+
0,
|
| 1156 |
+
0,
|
| 1157 |
+
0,
|
| 1158 |
+
1
|
| 1159 |
+
],
|
| 1160 |
+
[
|
| 1161 |
+
0,
|
| 1162 |
+
0,
|
| 1163 |
+
0,
|
| 1164 |
+
0,
|
| 1165 |
+
3,
|
| 1166 |
+
0,
|
| 1167 |
+
0,
|
| 1168 |
+
0,
|
| 1169 |
+
0,
|
| 1170 |
+
0,
|
| 1171 |
+
0,
|
| 1172 |
+
0,
|
| 1173 |
+
0,
|
| 1174 |
+
0,
|
| 1175 |
+
0,
|
| 1176 |
+
0,
|
| 1177 |
+
0,
|
| 1178 |
+
0,
|
| 1179 |
+
0,
|
| 1180 |
+
0,
|
| 1181 |
+
0,
|
| 1182 |
+
0,
|
| 1183 |
+
0,
|
| 1184 |
+
0,
|
| 1185 |
+
0,
|
| 1186 |
+
0,
|
| 1187 |
+
0,
|
| 1188 |
+
1,
|
| 1189 |
+
0,
|
| 1190 |
+
0,
|
| 1191 |
+
0,
|
| 1192 |
+
0,
|
| 1193 |
+
0,
|
| 1194 |
+
0,
|
| 1195 |
+
0
|
| 1196 |
+
],
|
| 1197 |
+
[
|
| 1198 |
+
0,
|
| 1199 |
+
0,
|
| 1200 |
+
0,
|
| 1201 |
+
0,
|
| 1202 |
+
1,
|
| 1203 |
+
0,
|
| 1204 |
+
0,
|
| 1205 |
+
0,
|
| 1206 |
+
0,
|
| 1207 |
+
0,
|
| 1208 |
+
0,
|
| 1209 |
+
0,
|
| 1210 |
+
0,
|
| 1211 |
+
0,
|
| 1212 |
+
0,
|
| 1213 |
+
0,
|
| 1214 |
+
0,
|
| 1215 |
+
0,
|
| 1216 |
+
0,
|
| 1217 |
+
0,
|
| 1218 |
+
0,
|
| 1219 |
+
0,
|
| 1220 |
+
0,
|
| 1221 |
+
0,
|
| 1222 |
+
0,
|
| 1223 |
+
0,
|
| 1224 |
+
0,
|
| 1225 |
+
0,
|
| 1226 |
+
0,
|
| 1227 |
+
0,
|
| 1228 |
+
0,
|
| 1229 |
+
0,
|
| 1230 |
+
0,
|
| 1231 |
+
0,
|
| 1232 |
+
0
|
| 1233 |
+
],
|
| 1234 |
+
[
|
| 1235 |
+
0,
|
| 1236 |
+
0,
|
| 1237 |
+
0,
|
| 1238 |
+
0,
|
| 1239 |
+
0,
|
| 1240 |
+
0,
|
| 1241 |
+
0,
|
| 1242 |
+
0,
|
| 1243 |
+
0,
|
| 1244 |
+
0,
|
| 1245 |
+
0,
|
| 1246 |
+
0,
|
| 1247 |
+
0,
|
| 1248 |
+
0,
|
| 1249 |
+
0,
|
| 1250 |
+
0,
|
| 1251 |
+
0,
|
| 1252 |
+
0,
|
| 1253 |
+
0,
|
| 1254 |
+
0,
|
| 1255 |
+
0,
|
| 1256 |
+
0,
|
| 1257 |
+
1,
|
| 1258 |
+
0,
|
| 1259 |
+
0,
|
| 1260 |
+
0,
|
| 1261 |
+
0,
|
| 1262 |
+
1,
|
| 1263 |
+
0,
|
| 1264 |
+
0,
|
| 1265 |
+
0,
|
| 1266 |
+
0,
|
| 1267 |
+
0,
|
| 1268 |
+
0,
|
| 1269 |
+
3
|
| 1270 |
+
],
|
| 1271 |
+
[
|
| 1272 |
+
0,
|
| 1273 |
+
0,
|
| 1274 |
+
0,
|
| 1275 |
+
0,
|
| 1276 |
+
1,
|
| 1277 |
+
0,
|
| 1278 |
+
0,
|
| 1279 |
+
0,
|
| 1280 |
+
0,
|
| 1281 |
+
0,
|
| 1282 |
+
0,
|
| 1283 |
+
0,
|
| 1284 |
+
0,
|
| 1285 |
+
0,
|
| 1286 |
+
0,
|
| 1287 |
+
0,
|
| 1288 |
+
0,
|
| 1289 |
+
0,
|
| 1290 |
+
0,
|
| 1291 |
+
0,
|
| 1292 |
+
0,
|
| 1293 |
+
0,
|
| 1294 |
+
0,
|
| 1295 |
+
0,
|
| 1296 |
+
0,
|
| 1297 |
+
0,
|
| 1298 |
+
0,
|
| 1299 |
+
0,
|
| 1300 |
+
0,
|
| 1301 |
+
0,
|
| 1302 |
+
0,
|
| 1303 |
+
0,
|
| 1304 |
+
0,
|
| 1305 |
+
0,
|
| 1306 |
+
0
|
| 1307 |
+
],
|
| 1308 |
+
[
|
| 1309 |
+
0,
|
| 1310 |
+
0,
|
| 1311 |
+
0,
|
| 1312 |
+
0,
|
| 1313 |
+
0,
|
| 1314 |
+
0,
|
| 1315 |
+
0,
|
| 1316 |
+
0,
|
| 1317 |
+
0,
|
| 1318 |
+
0,
|
| 1319 |
+
0,
|
| 1320 |
+
0,
|
| 1321 |
+
0,
|
| 1322 |
+
0,
|
| 1323 |
+
0,
|
| 1324 |
+
0,
|
| 1325 |
+
0,
|
| 1326 |
+
0,
|
| 1327 |
+
0,
|
| 1328 |
+
0,
|
| 1329 |
+
0,
|
| 1330 |
+
0,
|
| 1331 |
+
0,
|
| 1332 |
+
0,
|
| 1333 |
+
0,
|
| 1334 |
+
0,
|
| 1335 |
+
0,
|
| 1336 |
+
0,
|
| 1337 |
+
0,
|
| 1338 |
+
0,
|
| 1339 |
+
0,
|
| 1340 |
+
0,
|
| 1341 |
+
0,
|
| 1342 |
+
0,
|
| 1343 |
+
0
|
| 1344 |
+
],
|
| 1345 |
+
[
|
| 1346 |
+
0,
|
| 1347 |
+
0,
|
| 1348 |
+
0,
|
| 1349 |
+
0,
|
| 1350 |
+
0,
|
| 1351 |
+
0,
|
| 1352 |
+
0,
|
| 1353 |
+
0,
|
| 1354 |
+
0,
|
| 1355 |
+
0,
|
| 1356 |
+
0,
|
| 1357 |
+
0,
|
| 1358 |
+
0,
|
| 1359 |
+
0,
|
| 1360 |
+
0,
|
| 1361 |
+
0,
|
| 1362 |
+
0,
|
| 1363 |
+
0,
|
| 1364 |
+
0,
|
| 1365 |
+
0,
|
| 1366 |
+
0,
|
| 1367 |
+
0,
|
| 1368 |
+
0,
|
| 1369 |
+
0,
|
| 1370 |
+
0,
|
| 1371 |
+
0,
|
| 1372 |
+
0,
|
| 1373 |
+
0,
|
| 1374 |
+
0,
|
| 1375 |
+
0,
|
| 1376 |
+
0,
|
| 1377 |
+
0,
|
| 1378 |
+
0,
|
| 1379 |
+
0,
|
| 1380 |
+
2
|
| 1381 |
+
],
|
| 1382 |
+
[
|
| 1383 |
+
0,
|
| 1384 |
+
0,
|
| 1385 |
+
0,
|
| 1386 |
+
0,
|
| 1387 |
+
1,
|
| 1388 |
+
0,
|
| 1389 |
+
0,
|
| 1390 |
+
0,
|
| 1391 |
+
0,
|
| 1392 |
+
0,
|
| 1393 |
+
0,
|
| 1394 |
+
0,
|
| 1395 |
+
0,
|
| 1396 |
+
0,
|
| 1397 |
+
0,
|
| 1398 |
+
0,
|
| 1399 |
+
0,
|
| 1400 |
+
0,
|
| 1401 |
+
0,
|
| 1402 |
+
0,
|
| 1403 |
+
0,
|
| 1404 |
+
0,
|
| 1405 |
+
0,
|
| 1406 |
+
0,
|
| 1407 |
+
0,
|
| 1408 |
+
0,
|
| 1409 |
+
0,
|
| 1410 |
+
0,
|
| 1411 |
+
0,
|
| 1412 |
+
0,
|
| 1413 |
+
0,
|
| 1414 |
+
0,
|
| 1415 |
+
0,
|
| 1416 |
+
0,
|
| 1417 |
+
0
|
| 1418 |
+
],
|
| 1419 |
+
[
|
| 1420 |
+
0,
|
| 1421 |
+
0,
|
| 1422 |
+
0,
|
| 1423 |
+
0,
|
| 1424 |
+
0,
|
| 1425 |
+
0,
|
| 1426 |
+
0,
|
| 1427 |
+
0,
|
| 1428 |
+
0,
|
| 1429 |
+
0,
|
| 1430 |
+
0,
|
| 1431 |
+
0,
|
| 1432 |
+
0,
|
| 1433 |
+
0,
|
| 1434 |
+
0,
|
| 1435 |
+
0,
|
| 1436 |
+
0,
|
| 1437 |
+
0,
|
| 1438 |
+
0,
|
| 1439 |
+
0,
|
| 1440 |
+
0,
|
| 1441 |
+
0,
|
| 1442 |
+
0,
|
| 1443 |
+
0,
|
| 1444 |
+
0,
|
| 1445 |
+
0,
|
| 1446 |
+
0,
|
| 1447 |
+
0,
|
| 1448 |
+
0,
|
| 1449 |
+
0,
|
| 1450 |
+
0,
|
| 1451 |
+
0,
|
| 1452 |
+
0,
|
| 1453 |
+
0,
|
| 1454 |
+
0
|
| 1455 |
+
],
|
| 1456 |
+
[
|
| 1457 |
+
0,
|
| 1458 |
+
0,
|
| 1459 |
+
0,
|
| 1460 |
+
0,
|
| 1461 |
+
10,
|
| 1462 |
+
0,
|
| 1463 |
+
0,
|
| 1464 |
+
0,
|
| 1465 |
+
0,
|
| 1466 |
+
0,
|
| 1467 |
+
0,
|
| 1468 |
+
0,
|
| 1469 |
+
0,
|
| 1470 |
+
0,
|
| 1471 |
+
0,
|
| 1472 |
+
0,
|
| 1473 |
+
0,
|
| 1474 |
+
0,
|
| 1475 |
+
0,
|
| 1476 |
+
0,
|
| 1477 |
+
0,
|
| 1478 |
+
0,
|
| 1479 |
+
0,
|
| 1480 |
+
0,
|
| 1481 |
+
0,
|
| 1482 |
+
0,
|
| 1483 |
+
0,
|
| 1484 |
+
0,
|
| 1485 |
+
0,
|
| 1486 |
+
0,
|
| 1487 |
+
0,
|
| 1488 |
+
0,
|
| 1489 |
+
0,
|
| 1490 |
+
0,
|
| 1491 |
+
4
|
| 1492 |
+
],
|
| 1493 |
+
[
|
| 1494 |
+
0,
|
| 1495 |
+
0,
|
| 1496 |
+
0,
|
| 1497 |
+
0,
|
| 1498 |
+
4,
|
| 1499 |
+
0,
|
| 1500 |
+
7,
|
| 1501 |
+
0,
|
| 1502 |
+
0,
|
| 1503 |
+
0,
|
| 1504 |
+
0,
|
| 1505 |
+
0,
|
| 1506 |
+
0,
|
| 1507 |
+
0,
|
| 1508 |
+
0,
|
| 1509 |
+
0,
|
| 1510 |
+
0,
|
| 1511 |
+
0,
|
| 1512 |
+
0,
|
| 1513 |
+
0,
|
| 1514 |
+
0,
|
| 1515 |
+
0,
|
| 1516 |
+
0,
|
| 1517 |
+
0,
|
| 1518 |
+
0,
|
| 1519 |
+
0,
|
| 1520 |
+
0,
|
| 1521 |
+
1,
|
| 1522 |
+
0,
|
| 1523 |
+
0,
|
| 1524 |
+
0,
|
| 1525 |
+
0,
|
| 1526 |
+
0,
|
| 1527 |
+
0,
|
| 1528 |
+
126
|
| 1529 |
+
]
|
| 1530 |
+
]
|
| 1531 |
+
}
|
runs/20250928_131527/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fe6abcdbb83ea5ae3d75b585cb12a7ce3a054f5e269d8d2c204cb01e732e94b1
|
| 3 |
+
size 2154772
|
runs/20250928_131527/models/labels.txt
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
ACCOUNT#negative
|
| 2 |
+
CARD#negative
|
| 3 |
+
CARD#neutral
|
| 4 |
+
CARD#positive
|
| 5 |
+
CUSTOMER_SUPPORT#negative
|
| 6 |
+
CUSTOMER_SUPPORT#neutral
|
| 7 |
+
CUSTOMER_SUPPORT#positive
|
| 8 |
+
DISCOUNT#negative
|
| 9 |
+
DISCOUNT#neutral
|
| 10 |
+
DISCOUNT#positive
|
| 11 |
+
INTEREST_RATE#negative
|
| 12 |
+
INTEREST_RATE#neutral
|
| 13 |
+
INTEREST_RATE#positive
|
| 14 |
+
INTERNET_BANKING#negative
|
| 15 |
+
INTERNET_BANKING#neutral
|
| 16 |
+
INTERNET_BANKING#positive
|
| 17 |
+
LOAN#negative
|
| 18 |
+
LOAN#positive
|
| 19 |
+
MONEY_TRANSFER#negative
|
| 20 |
+
MONEY_TRANSFER#positive
|
| 21 |
+
OTHER#negative
|
| 22 |
+
OTHER#neutral
|
| 23 |
+
OTHER#positive
|
| 24 |
+
PAYMENT#negative
|
| 25 |
+
PAYMENT#positive
|
| 26 |
+
PROMOTION#negative
|
| 27 |
+
PROMOTION#neutral
|
| 28 |
+
PROMOTION#positive
|
| 29 |
+
SAVING#negative
|
| 30 |
+
SAVING#neutral
|
| 31 |
+
SAVING#positive
|
| 32 |
+
SECURITY#neutral
|
| 33 |
+
SECURITY#positive
|
| 34 |
+
TRADEMARK#negative
|
| 35 |
+
TRADEMARK#positive
|
runs/20250928_131527/models/model.joblib
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fe6abcdbb83ea5ae3d75b585cb12a7ce3a054f5e269d8d2c204cb01e732e94b1
|
| 3 |
+
size 2154772
|
runs/20250928_131527/training.log
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2025-09-28 13:15:27,107 - INFO - Starting training run: 20250928_131527
|
| 2 |
+
2025-09-28 13:15:27,107 - INFO - Model: svc_linear
|
| 3 |
+
2025-09-28 13:15:27,107 - INFO - Max features: 20000
|
| 4 |
+
2025-09-28 13:15:27,107 - INFO - N-gram range: (1, 2)
|
| 5 |
+
2025-09-28 13:15:27,107 - INFO - Loading UTS2017_Bank aspect sentiment dataset...
|
| 6 |
+
2025-09-28 13:15:31,618 - INFO - Train samples: 1581
|
| 7 |
+
2025-09-28 13:15:31,618 - INFO - Test samples: 396
|
| 8 |
+
2025-09-28 13:15:31,618 - INFO - Unique labels: 35
|
| 9 |
+
2025-09-28 13:15:31,618 - INFO - Label distribution (train): {np.str_('ACCOUNT#negative'): np.int64(3), np.str_('CARD#negative'): np.int64(47), np.str_('CARD#neutral'): np.int64(1), np.str_('CARD#positive'): np.int64(10), np.str_('CUSTOMER_SUPPORT#negative'): np.int64(288), np.str_('CUSTOMER_SUPPORT#neutral'): np.int64(4), np.str_('CUSTOMER_SUPPORT#positive'): np.int64(328), np.str_('DISCOUNT#negative'): np.int64(13), np.str_('DISCOUNT#neutral'): np.int64(3), np.str_('DISCOUNT#positive'): np.int64(19), np.str_('INTEREST_RATE#negative'): np.int64(45), np.str_('INTEREST_RATE#neutral'): np.int64(1), np.str_('INTEREST_RATE#positive'): np.int64(4), np.str_('INTERNET_BANKING#negative'): np.int64(36), np.str_('INTERNET_BANKING#neutral'): np.int64(2), np.str_('INTERNET_BANKING#positive'): np.int64(19), np.str_('LOAN#negative'): np.int64(48), np.str_('LOAN#positive'): np.int64(13), np.str_('MONEY_TRANSFER#negative'): np.int64(24), np.str_('MONEY_TRANSFER#positive'): np.int64(5), np.str_('OTHER#negative'): np.int64(25), np.str_('OTHER#neutral'): np.int64(3), np.str_('OTHER#positive'): np.int64(26), np.str_('PAYMENT#negative'): np.int64(4), np.str_('PAYMENT#positive'): np.int64(8), np.str_('PROMOTION#negative'): np.int64(13), np.str_('PROMOTION#neutral'): np.int64(2), np.str_('PROMOTION#positive'): np.int64(28), np.str_('SAVING#negative'): np.int64(5), np.str_('SAVING#neutral'): np.int64(1), np.str_('SAVING#positive'): np.int64(4), np.str_('SECURITY#neutral'): np.int64(1), np.str_('SECURITY#positive'): np.int64(1), np.str_('TRADEMARK#negative'): np.int64(33), np.str_('TRADEMARK#positive'): np.int64(514)}
|
| 10 |
+
2025-09-28 13:15:31,618 - INFO - Label distribution (test): {np.str_('ACCOUNT#negative'): np.int64(2), np.str_('CARD#negative'): np.int64(7), np.str_('CARD#neutral'): np.int64(0), np.str_('CARD#positive'): np.int64(1), np.str_('CUSTOMER_SUPPORT#negative'): np.int64(75), np.str_('CUSTOMER_SUPPORT#neutral'): np.int64(1), np.str_('CUSTOMER_SUPPORT#positive'): np.int64(78), np.str_('DISCOUNT#negative'): np.int64(5), np.str_('DISCOUNT#neutral'): np.int64(1), np.str_('DISCOUNT#positive'): np.int64(0), np.str_('INTEREST_RATE#negative'): np.int64(10), np.str_('INTEREST_RATE#neutral'): np.int64(0), np.str_('INTEREST_RATE#positive'): np.int64(0), np.str_('INTERNET_BANKING#negative'): np.int64(12), np.str_('INTERNET_BANKING#neutral'): np.int64(0), np.str_('INTERNET_BANKING#positive'): np.int64(1), np.str_('LOAN#negative'): np.int64(13), np.str_('LOAN#positive'): np.int64(0), np.str_('MONEY_TRANSFER#negative'): np.int64(5), np.str_('MONEY_TRANSFER#positive'): np.int64(0), np.str_('OTHER#negative'): np.int64(10), np.str_('OTHER#neutral'): np.int64(1), np.str_('OTHER#positive'): np.int64(4), np.str_('PAYMENT#negative'): np.int64(0), np.str_('PAYMENT#positive'): np.int64(3), np.str_('PROMOTION#negative'): np.int64(4), np.str_('PROMOTION#neutral'): np.int64(1), np.str_('PROMOTION#positive'): np.int64(5), np.str_('SAVING#negative'): np.int64(1), np.str_('SAVING#neutral'): np.int64(0), np.str_('SAVING#positive'): np.int64(2), np.str_('SECURITY#neutral'): np.int64(1), np.str_('SECURITY#positive'): np.int64(0), np.str_('TRADEMARK#negative'): np.int64(14), np.str_('TRADEMARK#positive'): np.int64(138)}
|
| 11 |
+
2025-09-28 13:15:31,619 - INFO - Selected classifier: SVC
|
| 12 |
+
2025-09-28 13:15:31,619 - INFO - ============================================================
|
| 13 |
+
2025-09-28 13:15:31,619 - INFO - Training: UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2
|
| 14 |
+
2025-09-28 13:15:31,619 - INFO - ============================================================
|
| 15 |
+
2025-09-28 13:15:31,619 - INFO - Creating pipeline with max_features=20000, ngram_range=(1, 2)
|
| 16 |
+
2025-09-28 13:15:31,619 - INFO - Training model...
|
| 17 |
+
2025-09-28 13:15:39,357 - INFO - Training completed in 7.74 seconds
|
| 18 |
+
2025-09-28 13:15:39,357 - INFO - Evaluating on training set...
|
| 19 |
+
2025-09-28 13:15:39,803 - INFO - Training accuracy: 0.9431
|
| 20 |
+
2025-09-28 13:15:39,803 - INFO - Evaluating on test set...
|
| 21 |
+
2025-09-28 13:15:39,914 - INFO - Test accuracy: 0.7172
|
| 22 |
+
2025-09-28 13:15:39,914 - INFO - Prediction time: 0.11 seconds
|
| 23 |
+
2025-09-28 13:15:39,914 - INFO - Classification Report:
|
| 24 |
+
2025-09-28 13:15:39,918 - INFO - precision recall f1-score support
|
| 25 |
+
|
| 26 |
+
ACCOUNT#negative 0.00 0.00 0.00 2
|
| 27 |
+
CARD#negative 1.00 0.14 0.25 7
|
| 28 |
+
CARD#positive 0.00 0.00 0.00 1
|
| 29 |
+
CUSTOMER_SUPPORT#negative 0.49 0.96 0.65 75
|
| 30 |
+
CUSTOMER_SUPPORT#neutral 0.00 0.00 0.00 1
|
| 31 |
+
CUSTOMER_SUPPORT#positive 0.86 0.90 0.88 78
|
| 32 |
+
DISCOUNT#negative 0.00 0.00 0.00 5
|
| 33 |
+
DISCOUNT#neutral 0.00 0.00 0.00 1
|
| 34 |
+
DISCOUNT#positive 0.00 0.00 0.00 0
|
| 35 |
+
INTEREST_RATE#negative 0.56 0.50 0.53 10
|
| 36 |
+
INTERNET_BANKING#negative 0.00 0.00 0.00 12
|
| 37 |
+
INTERNET_BANKING#positive 0.00 0.00 0.00 1
|
| 38 |
+
LOAN#negative 0.88 0.54 0.67 13
|
| 39 |
+
MONEY_TRANSFER#negative 0.00 0.00 0.00 5
|
| 40 |
+
OTHER#negative 0.00 0.00 0.00 10
|
| 41 |
+
OTHER#neutral 0.00 0.00 0.00 1
|
| 42 |
+
OTHER#positive 0.67 0.50 0.57 4
|
| 43 |
+
PAYMENT#positive 0.00 0.00 0.00 3
|
| 44 |
+
PROMOTION#negative 0.00 0.00 0.00 4
|
| 45 |
+
PROMOTION#neutral 0.00 0.00 0.00 1
|
| 46 |
+
PROMOTION#positive 0.33 0.20 0.25 5
|
| 47 |
+
SAVING#negative 0.00 0.00 0.00 1
|
| 48 |
+
SAVING#positive 0.00 0.00 0.00 2
|
| 49 |
+
SECURITY#negative 0.00 0.00 0.00 1
|
| 50 |
+
SECURITY#neutral 0.00 0.00 0.00 1
|
| 51 |
+
TRADEMARK#negative 0.00 0.00 0.00 14
|
| 52 |
+
TRADEMARK#positive 0.89 0.91 0.90 138
|
| 53 |
+
|
| 54 |
+
accuracy 0.72 396
|
| 55 |
+
macro avg 0.21 0.17 0.17 396
|
| 56 |
+
weighted avg 0.65 0.72 0.66 396
|
| 57 |
+
|
| 58 |
+
2025-09-28 13:15:39,922 - INFO - Confusion Matrix shape: (35, 35)
|
| 59 |
+
2025-09-28 13:15:40,052 - INFO - Model saved to runs/20250928_131527/models/model.joblib
|
| 60 |
+
2025-09-28 13:15:40,181 - INFO - Model also saved as runs/20250928_131527/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib
|
| 61 |
+
2025-09-28 13:15:40,181 - INFO - Label mapping saved to runs/20250928_131527/models/labels.txt
|
| 62 |
+
2025-09-28 13:15:40,182 - INFO - Metadata saved to runs/20250928_131527/metadata.json
|
runs/20250928_131716/metadata.json
ADDED
|
@@ -0,0 +1,1531 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"timestamp": "20250928_131716",
|
| 3 |
+
"config_name": "UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2",
|
| 4 |
+
"model_name": "svc_linear",
|
| 5 |
+
"classifier": "SVC",
|
| 6 |
+
"max_features": 20000,
|
| 7 |
+
"ngram_range": [
|
| 8 |
+
1,
|
| 9 |
+
2
|
| 10 |
+
],
|
| 11 |
+
"split_ratio": 0.2,
|
| 12 |
+
"n_samples": null,
|
| 13 |
+
"train_samples": 1581,
|
| 14 |
+
"test_samples": 396,
|
| 15 |
+
"unique_labels": 35,
|
| 16 |
+
"labels": [
|
| 17 |
+
"ACCOUNT#negative",
|
| 18 |
+
"CARD#negative",
|
| 19 |
+
"CARD#neutral",
|
| 20 |
+
"CARD#positive",
|
| 21 |
+
"CUSTOMER_SUPPORT#negative",
|
| 22 |
+
"CUSTOMER_SUPPORT#neutral",
|
| 23 |
+
"CUSTOMER_SUPPORT#positive",
|
| 24 |
+
"DISCOUNT#negative",
|
| 25 |
+
"DISCOUNT#neutral",
|
| 26 |
+
"DISCOUNT#positive",
|
| 27 |
+
"INTEREST_RATE#negative",
|
| 28 |
+
"INTEREST_RATE#neutral",
|
| 29 |
+
"INTEREST_RATE#positive",
|
| 30 |
+
"INTERNET_BANKING#negative",
|
| 31 |
+
"INTERNET_BANKING#neutral",
|
| 32 |
+
"INTERNET_BANKING#positive",
|
| 33 |
+
"LOAN#negative",
|
| 34 |
+
"LOAN#positive",
|
| 35 |
+
"MONEY_TRANSFER#negative",
|
| 36 |
+
"MONEY_TRANSFER#positive",
|
| 37 |
+
"OTHER#negative",
|
| 38 |
+
"OTHER#neutral",
|
| 39 |
+
"OTHER#positive",
|
| 40 |
+
"PAYMENT#negative",
|
| 41 |
+
"PAYMENT#positive",
|
| 42 |
+
"PROMOTION#negative",
|
| 43 |
+
"PROMOTION#neutral",
|
| 44 |
+
"PROMOTION#positive",
|
| 45 |
+
"SAVING#negative",
|
| 46 |
+
"SAVING#neutral",
|
| 47 |
+
"SAVING#positive",
|
| 48 |
+
"SECURITY#neutral",
|
| 49 |
+
"SECURITY#positive",
|
| 50 |
+
"TRADEMARK#negative",
|
| 51 |
+
"TRADEMARK#positive"
|
| 52 |
+
],
|
| 53 |
+
"train_accuracy": 0.9430740037950665,
|
| 54 |
+
"test_accuracy": 0.7171717171717171,
|
| 55 |
+
"train_time": 7.706280946731567,
|
| 56 |
+
"prediction_time": 0.10988211631774902,
|
| 57 |
+
"classification_report": {
|
| 58 |
+
"ACCOUNT#negative": {
|
| 59 |
+
"precision": 0.0,
|
| 60 |
+
"recall": 0.0,
|
| 61 |
+
"f1-score": 0.0,
|
| 62 |
+
"support": 2.0
|
| 63 |
+
},
|
| 64 |
+
"CARD#negative": {
|
| 65 |
+
"precision": 1.0,
|
| 66 |
+
"recall": 0.14285714285714285,
|
| 67 |
+
"f1-score": 0.25,
|
| 68 |
+
"support": 7.0
|
| 69 |
+
},
|
| 70 |
+
"CARD#positive": {
|
| 71 |
+
"precision": 0.0,
|
| 72 |
+
"recall": 0.0,
|
| 73 |
+
"f1-score": 0.0,
|
| 74 |
+
"support": 1.0
|
| 75 |
+
},
|
| 76 |
+
"CUSTOMER_SUPPORT#negative": {
|
| 77 |
+
"precision": 0.4931506849315068,
|
| 78 |
+
"recall": 0.96,
|
| 79 |
+
"f1-score": 0.6515837104072398,
|
| 80 |
+
"support": 75.0
|
| 81 |
+
},
|
| 82 |
+
"CUSTOMER_SUPPORT#neutral": {
|
| 83 |
+
"precision": 0.0,
|
| 84 |
+
"recall": 0.0,
|
| 85 |
+
"f1-score": 0.0,
|
| 86 |
+
"support": 1.0
|
| 87 |
+
},
|
| 88 |
+
"CUSTOMER_SUPPORT#positive": {
|
| 89 |
+
"precision": 0.8641975308641975,
|
| 90 |
+
"recall": 0.8974358974358975,
|
| 91 |
+
"f1-score": 0.8805031446540881,
|
| 92 |
+
"support": 78.0
|
| 93 |
+
},
|
| 94 |
+
"DISCOUNT#negative": {
|
| 95 |
+
"precision": 0.0,
|
| 96 |
+
"recall": 0.0,
|
| 97 |
+
"f1-score": 0.0,
|
| 98 |
+
"support": 5.0
|
| 99 |
+
},
|
| 100 |
+
"DISCOUNT#neutral": {
|
| 101 |
+
"precision": 0.0,
|
| 102 |
+
"recall": 0.0,
|
| 103 |
+
"f1-score": 0.0,
|
| 104 |
+
"support": 1.0
|
| 105 |
+
},
|
| 106 |
+
"DISCOUNT#positive": {
|
| 107 |
+
"precision": 0.0,
|
| 108 |
+
"recall": 0.0,
|
| 109 |
+
"f1-score": 0.0,
|
| 110 |
+
"support": 0.0
|
| 111 |
+
},
|
| 112 |
+
"INTEREST_RATE#negative": {
|
| 113 |
+
"precision": 0.5555555555555556,
|
| 114 |
+
"recall": 0.5,
|
| 115 |
+
"f1-score": 0.5263157894736842,
|
| 116 |
+
"support": 10.0
|
| 117 |
+
},
|
| 118 |
+
"INTERNET_BANKING#negative": {
|
| 119 |
+
"precision": 0.0,
|
| 120 |
+
"recall": 0.0,
|
| 121 |
+
"f1-score": 0.0,
|
| 122 |
+
"support": 12.0
|
| 123 |
+
},
|
| 124 |
+
"INTERNET_BANKING#positive": {
|
| 125 |
+
"precision": 0.0,
|
| 126 |
+
"recall": 0.0,
|
| 127 |
+
"f1-score": 0.0,
|
| 128 |
+
"support": 1.0
|
| 129 |
+
},
|
| 130 |
+
"LOAN#negative": {
|
| 131 |
+
"precision": 0.875,
|
| 132 |
+
"recall": 0.5384615384615384,
|
| 133 |
+
"f1-score": 0.6666666666666666,
|
| 134 |
+
"support": 13.0
|
| 135 |
+
},
|
| 136 |
+
"MONEY_TRANSFER#negative": {
|
| 137 |
+
"precision": 0.0,
|
| 138 |
+
"recall": 0.0,
|
| 139 |
+
"f1-score": 0.0,
|
| 140 |
+
"support": 5.0
|
| 141 |
+
},
|
| 142 |
+
"OTHER#negative": {
|
| 143 |
+
"precision": 0.0,
|
| 144 |
+
"recall": 0.0,
|
| 145 |
+
"f1-score": 0.0,
|
| 146 |
+
"support": 10.0
|
| 147 |
+
},
|
| 148 |
+
"OTHER#neutral": {
|
| 149 |
+
"precision": 0.0,
|
| 150 |
+
"recall": 0.0,
|
| 151 |
+
"f1-score": 0.0,
|
| 152 |
+
"support": 1.0
|
| 153 |
+
},
|
| 154 |
+
"OTHER#positive": {
|
| 155 |
+
"precision": 0.6666666666666666,
|
| 156 |
+
"recall": 0.5,
|
| 157 |
+
"f1-score": 0.5714285714285714,
|
| 158 |
+
"support": 4.0
|
| 159 |
+
},
|
| 160 |
+
"PAYMENT#positive": {
|
| 161 |
+
"precision": 0.0,
|
| 162 |
+
"recall": 0.0,
|
| 163 |
+
"f1-score": 0.0,
|
| 164 |
+
"support": 3.0
|
| 165 |
+
},
|
| 166 |
+
"PROMOTION#negative": {
|
| 167 |
+
"precision": 0.0,
|
| 168 |
+
"recall": 0.0,
|
| 169 |
+
"f1-score": 0.0,
|
| 170 |
+
"support": 4.0
|
| 171 |
+
},
|
| 172 |
+
"PROMOTION#neutral": {
|
| 173 |
+
"precision": 0.0,
|
| 174 |
+
"recall": 0.0,
|
| 175 |
+
"f1-score": 0.0,
|
| 176 |
+
"support": 1.0
|
| 177 |
+
},
|
| 178 |
+
"PROMOTION#positive": {
|
| 179 |
+
"precision": 0.3333333333333333,
|
| 180 |
+
"recall": 0.2,
|
| 181 |
+
"f1-score": 0.25,
|
| 182 |
+
"support": 5.0
|
| 183 |
+
},
|
| 184 |
+
"SAVING#negative": {
|
| 185 |
+
"precision": 0.0,
|
| 186 |
+
"recall": 0.0,
|
| 187 |
+
"f1-score": 0.0,
|
| 188 |
+
"support": 1.0
|
| 189 |
+
},
|
| 190 |
+
"SAVING#positive": {
|
| 191 |
+
"precision": 0.0,
|
| 192 |
+
"recall": 0.0,
|
| 193 |
+
"f1-score": 0.0,
|
| 194 |
+
"support": 2.0
|
| 195 |
+
},
|
| 196 |
+
"SECURITY#negative": {
|
| 197 |
+
"precision": 0.0,
|
| 198 |
+
"recall": 0.0,
|
| 199 |
+
"f1-score": 0.0,
|
| 200 |
+
"support": 1.0
|
| 201 |
+
},
|
| 202 |
+
"SECURITY#neutral": {
|
| 203 |
+
"precision": 0.0,
|
| 204 |
+
"recall": 0.0,
|
| 205 |
+
"f1-score": 0.0,
|
| 206 |
+
"support": 1.0
|
| 207 |
+
},
|
| 208 |
+
"TRADEMARK#negative": {
|
| 209 |
+
"precision": 0.0,
|
| 210 |
+
"recall": 0.0,
|
| 211 |
+
"f1-score": 0.0,
|
| 212 |
+
"support": 14.0
|
| 213 |
+
},
|
| 214 |
+
"TRADEMARK#positive": {
|
| 215 |
+
"precision": 0.8936170212765957,
|
| 216 |
+
"recall": 0.9130434782608695,
|
| 217 |
+
"f1-score": 0.9032258064516129,
|
| 218 |
+
"support": 138.0
|
| 219 |
+
},
|
| 220 |
+
"accuracy": 0.7171717171717171,
|
| 221 |
+
"macro avg": {
|
| 222 |
+
"precision": 0.21042669602325392,
|
| 223 |
+
"recall": 0.17228881692649806,
|
| 224 |
+
"f1-score": 0.1740638403363653,
|
| 225 |
+
"support": 396.0
|
| 226 |
+
},
|
| 227 |
+
"weighted avg": {
|
| 228 |
+
"precision": 0.6464059257634583,
|
| 229 |
+
"recall": 0.7171717171717171,
|
| 230 |
+
"f1-score": 0.6601230396489957,
|
| 231 |
+
"support": 396.0
|
| 232 |
+
}
|
| 233 |
+
},
|
| 234 |
+
"confusion_matrix": [
|
| 235 |
+
[
|
| 236 |
+
0,
|
| 237 |
+
0,
|
| 238 |
+
0,
|
| 239 |
+
0,
|
| 240 |
+
1,
|
| 241 |
+
0,
|
| 242 |
+
1,
|
| 243 |
+
0,
|
| 244 |
+
0,
|
| 245 |
+
0,
|
| 246 |
+
0,
|
| 247 |
+
0,
|
| 248 |
+
0,
|
| 249 |
+
0,
|
| 250 |
+
0,
|
| 251 |
+
0,
|
| 252 |
+
0,
|
| 253 |
+
0,
|
| 254 |
+
0,
|
| 255 |
+
0,
|
| 256 |
+
0,
|
| 257 |
+
0,
|
| 258 |
+
0,
|
| 259 |
+
0,
|
| 260 |
+
0,
|
| 261 |
+
0,
|
| 262 |
+
0,
|
| 263 |
+
0,
|
| 264 |
+
0,
|
| 265 |
+
0,
|
| 266 |
+
0,
|
| 267 |
+
0,
|
| 268 |
+
0,
|
| 269 |
+
0,
|
| 270 |
+
0
|
| 271 |
+
],
|
| 272 |
+
[
|
| 273 |
+
0,
|
| 274 |
+
1,
|
| 275 |
+
0,
|
| 276 |
+
0,
|
| 277 |
+
5,
|
| 278 |
+
0,
|
| 279 |
+
0,
|
| 280 |
+
0,
|
| 281 |
+
0,
|
| 282 |
+
0,
|
| 283 |
+
1,
|
| 284 |
+
0,
|
| 285 |
+
0,
|
| 286 |
+
0,
|
| 287 |
+
0,
|
| 288 |
+
0,
|
| 289 |
+
0,
|
| 290 |
+
0,
|
| 291 |
+
0,
|
| 292 |
+
0,
|
| 293 |
+
0,
|
| 294 |
+
0,
|
| 295 |
+
0,
|
| 296 |
+
0,
|
| 297 |
+
0,
|
| 298 |
+
0,
|
| 299 |
+
0,
|
| 300 |
+
0,
|
| 301 |
+
0,
|
| 302 |
+
0,
|
| 303 |
+
0,
|
| 304 |
+
0,
|
| 305 |
+
0,
|
| 306 |
+
0,
|
| 307 |
+
0
|
| 308 |
+
],
|
| 309 |
+
[
|
| 310 |
+
0,
|
| 311 |
+
0,
|
| 312 |
+
0,
|
| 313 |
+
0,
|
| 314 |
+
0,
|
| 315 |
+
0,
|
| 316 |
+
0,
|
| 317 |
+
0,
|
| 318 |
+
0,
|
| 319 |
+
0,
|
| 320 |
+
0,
|
| 321 |
+
0,
|
| 322 |
+
0,
|
| 323 |
+
0,
|
| 324 |
+
0,
|
| 325 |
+
0,
|
| 326 |
+
0,
|
| 327 |
+
0,
|
| 328 |
+
0,
|
| 329 |
+
0,
|
| 330 |
+
0,
|
| 331 |
+
0,
|
| 332 |
+
0,
|
| 333 |
+
0,
|
| 334 |
+
0,
|
| 335 |
+
0,
|
| 336 |
+
0,
|
| 337 |
+
0,
|
| 338 |
+
0,
|
| 339 |
+
0,
|
| 340 |
+
0,
|
| 341 |
+
0,
|
| 342 |
+
0,
|
| 343 |
+
0,
|
| 344 |
+
0
|
| 345 |
+
],
|
| 346 |
+
[
|
| 347 |
+
0,
|
| 348 |
+
0,
|
| 349 |
+
0,
|
| 350 |
+
0,
|
| 351 |
+
0,
|
| 352 |
+
0,
|
| 353 |
+
0,
|
| 354 |
+
0,
|
| 355 |
+
0,
|
| 356 |
+
0,
|
| 357 |
+
0,
|
| 358 |
+
0,
|
| 359 |
+
0,
|
| 360 |
+
0,
|
| 361 |
+
0,
|
| 362 |
+
0,
|
| 363 |
+
0,
|
| 364 |
+
0,
|
| 365 |
+
0,
|
| 366 |
+
0,
|
| 367 |
+
0,
|
| 368 |
+
0,
|
| 369 |
+
0,
|
| 370 |
+
0,
|
| 371 |
+
0,
|
| 372 |
+
0,
|
| 373 |
+
0,
|
| 374 |
+
0,
|
| 375 |
+
0,
|
| 376 |
+
0,
|
| 377 |
+
0,
|
| 378 |
+
0,
|
| 379 |
+
0,
|
| 380 |
+
0,
|
| 381 |
+
1
|
| 382 |
+
],
|
| 383 |
+
[
|
| 384 |
+
0,
|
| 385 |
+
0,
|
| 386 |
+
0,
|
| 387 |
+
0,
|
| 388 |
+
72,
|
| 389 |
+
0,
|
| 390 |
+
1,
|
| 391 |
+
0,
|
| 392 |
+
0,
|
| 393 |
+
0,
|
| 394 |
+
1,
|
| 395 |
+
0,
|
| 396 |
+
0,
|
| 397 |
+
0,
|
| 398 |
+
0,
|
| 399 |
+
0,
|
| 400 |
+
1,
|
| 401 |
+
0,
|
| 402 |
+
0,
|
| 403 |
+
0,
|
| 404 |
+
0,
|
| 405 |
+
0,
|
| 406 |
+
0,
|
| 407 |
+
0,
|
| 408 |
+
0,
|
| 409 |
+
0,
|
| 410 |
+
0,
|
| 411 |
+
0,
|
| 412 |
+
0,
|
| 413 |
+
0,
|
| 414 |
+
0,
|
| 415 |
+
0,
|
| 416 |
+
0,
|
| 417 |
+
0,
|
| 418 |
+
0
|
| 419 |
+
],
|
| 420 |
+
[
|
| 421 |
+
0,
|
| 422 |
+
0,
|
| 423 |
+
0,
|
| 424 |
+
0,
|
| 425 |
+
1,
|
| 426 |
+
0,
|
| 427 |
+
0,
|
| 428 |
+
0,
|
| 429 |
+
0,
|
| 430 |
+
0,
|
| 431 |
+
0,
|
| 432 |
+
0,
|
| 433 |
+
0,
|
| 434 |
+
0,
|
| 435 |
+
0,
|
| 436 |
+
0,
|
| 437 |
+
0,
|
| 438 |
+
0,
|
| 439 |
+
0,
|
| 440 |
+
0,
|
| 441 |
+
0,
|
| 442 |
+
0,
|
| 443 |
+
0,
|
| 444 |
+
0,
|
| 445 |
+
0,
|
| 446 |
+
0,
|
| 447 |
+
0,
|
| 448 |
+
0,
|
| 449 |
+
0,
|
| 450 |
+
0,
|
| 451 |
+
0,
|
| 452 |
+
0,
|
| 453 |
+
0,
|
| 454 |
+
0,
|
| 455 |
+
0
|
| 456 |
+
],
|
| 457 |
+
[
|
| 458 |
+
0,
|
| 459 |
+
0,
|
| 460 |
+
0,
|
| 461 |
+
0,
|
| 462 |
+
6,
|
| 463 |
+
0,
|
| 464 |
+
70,
|
| 465 |
+
0,
|
| 466 |
+
0,
|
| 467 |
+
0,
|
| 468 |
+
0,
|
| 469 |
+
0,
|
| 470 |
+
0,
|
| 471 |
+
0,
|
| 472 |
+
0,
|
| 473 |
+
0,
|
| 474 |
+
0,
|
| 475 |
+
0,
|
| 476 |
+
0,
|
| 477 |
+
0,
|
| 478 |
+
0,
|
| 479 |
+
0,
|
| 480 |
+
0,
|
| 481 |
+
0,
|
| 482 |
+
0,
|
| 483 |
+
0,
|
| 484 |
+
0,
|
| 485 |
+
0,
|
| 486 |
+
0,
|
| 487 |
+
0,
|
| 488 |
+
0,
|
| 489 |
+
0,
|
| 490 |
+
0,
|
| 491 |
+
0,
|
| 492 |
+
2
|
| 493 |
+
],
|
| 494 |
+
[
|
| 495 |
+
0,
|
| 496 |
+
0,
|
| 497 |
+
0,
|
| 498 |
+
0,
|
| 499 |
+
5,
|
| 500 |
+
0,
|
| 501 |
+
0,
|
| 502 |
+
0,
|
| 503 |
+
0,
|
| 504 |
+
0,
|
| 505 |
+
0,
|
| 506 |
+
0,
|
| 507 |
+
0,
|
| 508 |
+
0,
|
| 509 |
+
0,
|
| 510 |
+
0,
|
| 511 |
+
0,
|
| 512 |
+
0,
|
| 513 |
+
0,
|
| 514 |
+
0,
|
| 515 |
+
0,
|
| 516 |
+
0,
|
| 517 |
+
0,
|
| 518 |
+
0,
|
| 519 |
+
0,
|
| 520 |
+
0,
|
| 521 |
+
0,
|
| 522 |
+
0,
|
| 523 |
+
0,
|
| 524 |
+
0,
|
| 525 |
+
0,
|
| 526 |
+
0,
|
| 527 |
+
0,
|
| 528 |
+
0,
|
| 529 |
+
0
|
| 530 |
+
],
|
| 531 |
+
[
|
| 532 |
+
0,
|
| 533 |
+
0,
|
| 534 |
+
0,
|
| 535 |
+
0,
|
| 536 |
+
1,
|
| 537 |
+
0,
|
| 538 |
+
0,
|
| 539 |
+
0,
|
| 540 |
+
0,
|
| 541 |
+
0,
|
| 542 |
+
0,
|
| 543 |
+
0,
|
| 544 |
+
0,
|
| 545 |
+
0,
|
| 546 |
+
0,
|
| 547 |
+
0,
|
| 548 |
+
0,
|
| 549 |
+
0,
|
| 550 |
+
0,
|
| 551 |
+
0,
|
| 552 |
+
0,
|
| 553 |
+
0,
|
| 554 |
+
0,
|
| 555 |
+
0,
|
| 556 |
+
0,
|
| 557 |
+
0,
|
| 558 |
+
0,
|
| 559 |
+
0,
|
| 560 |
+
0,
|
| 561 |
+
0,
|
| 562 |
+
0,
|
| 563 |
+
0,
|
| 564 |
+
0,
|
| 565 |
+
0,
|
| 566 |
+
0
|
| 567 |
+
],
|
| 568 |
+
[
|
| 569 |
+
0,
|
| 570 |
+
0,
|
| 571 |
+
0,
|
| 572 |
+
0,
|
| 573 |
+
0,
|
| 574 |
+
0,
|
| 575 |
+
0,
|
| 576 |
+
0,
|
| 577 |
+
0,
|
| 578 |
+
0,
|
| 579 |
+
0,
|
| 580 |
+
0,
|
| 581 |
+
0,
|
| 582 |
+
0,
|
| 583 |
+
0,
|
| 584 |
+
0,
|
| 585 |
+
0,
|
| 586 |
+
0,
|
| 587 |
+
0,
|
| 588 |
+
0,
|
| 589 |
+
0,
|
| 590 |
+
0,
|
| 591 |
+
0,
|
| 592 |
+
0,
|
| 593 |
+
0,
|
| 594 |
+
0,
|
| 595 |
+
0,
|
| 596 |
+
0,
|
| 597 |
+
0,
|
| 598 |
+
0,
|
| 599 |
+
0,
|
| 600 |
+
0,
|
| 601 |
+
0,
|
| 602 |
+
0,
|
| 603 |
+
0
|
| 604 |
+
],
|
| 605 |
+
[
|
| 606 |
+
0,
|
| 607 |
+
0,
|
| 608 |
+
0,
|
| 609 |
+
0,
|
| 610 |
+
4,
|
| 611 |
+
0,
|
| 612 |
+
0,
|
| 613 |
+
0,
|
| 614 |
+
0,
|
| 615 |
+
0,
|
| 616 |
+
5,
|
| 617 |
+
0,
|
| 618 |
+
0,
|
| 619 |
+
0,
|
| 620 |
+
0,
|
| 621 |
+
0,
|
| 622 |
+
0,
|
| 623 |
+
0,
|
| 624 |
+
0,
|
| 625 |
+
0,
|
| 626 |
+
0,
|
| 627 |
+
0,
|
| 628 |
+
0,
|
| 629 |
+
0,
|
| 630 |
+
0,
|
| 631 |
+
0,
|
| 632 |
+
0,
|
| 633 |
+
0,
|
| 634 |
+
0,
|
| 635 |
+
0,
|
| 636 |
+
0,
|
| 637 |
+
0,
|
| 638 |
+
0,
|
| 639 |
+
0,
|
| 640 |
+
1
|
| 641 |
+
],
|
| 642 |
+
[
|
| 643 |
+
0,
|
| 644 |
+
0,
|
| 645 |
+
0,
|
| 646 |
+
0,
|
| 647 |
+
0,
|
| 648 |
+
0,
|
| 649 |
+
0,
|
| 650 |
+
0,
|
| 651 |
+
0,
|
| 652 |
+
0,
|
| 653 |
+
0,
|
| 654 |
+
0,
|
| 655 |
+
0,
|
| 656 |
+
0,
|
| 657 |
+
0,
|
| 658 |
+
0,
|
| 659 |
+
0,
|
| 660 |
+
0,
|
| 661 |
+
0,
|
| 662 |
+
0,
|
| 663 |
+
0,
|
| 664 |
+
0,
|
| 665 |
+
0,
|
| 666 |
+
0,
|
| 667 |
+
0,
|
| 668 |
+
0,
|
| 669 |
+
0,
|
| 670 |
+
0,
|
| 671 |
+
0,
|
| 672 |
+
0,
|
| 673 |
+
0,
|
| 674 |
+
0,
|
| 675 |
+
0,
|
| 676 |
+
0,
|
| 677 |
+
0
|
| 678 |
+
],
|
| 679 |
+
[
|
| 680 |
+
0,
|
| 681 |
+
0,
|
| 682 |
+
0,
|
| 683 |
+
0,
|
| 684 |
+
0,
|
| 685 |
+
0,
|
| 686 |
+
0,
|
| 687 |
+
0,
|
| 688 |
+
0,
|
| 689 |
+
0,
|
| 690 |
+
0,
|
| 691 |
+
0,
|
| 692 |
+
0,
|
| 693 |
+
0,
|
| 694 |
+
0,
|
| 695 |
+
0,
|
| 696 |
+
0,
|
| 697 |
+
0,
|
| 698 |
+
0,
|
| 699 |
+
0,
|
| 700 |
+
0,
|
| 701 |
+
0,
|
| 702 |
+
0,
|
| 703 |
+
0,
|
| 704 |
+
0,
|
| 705 |
+
0,
|
| 706 |
+
0,
|
| 707 |
+
0,
|
| 708 |
+
0,
|
| 709 |
+
0,
|
| 710 |
+
0,
|
| 711 |
+
0,
|
| 712 |
+
0,
|
| 713 |
+
0,
|
| 714 |
+
0
|
| 715 |
+
],
|
| 716 |
+
[
|
| 717 |
+
0,
|
| 718 |
+
0,
|
| 719 |
+
0,
|
| 720 |
+
0,
|
| 721 |
+
12,
|
| 722 |
+
0,
|
| 723 |
+
0,
|
| 724 |
+
0,
|
| 725 |
+
0,
|
| 726 |
+
0,
|
| 727 |
+
0,
|
| 728 |
+
0,
|
| 729 |
+
0,
|
| 730 |
+
0,
|
| 731 |
+
0,
|
| 732 |
+
0,
|
| 733 |
+
0,
|
| 734 |
+
0,
|
| 735 |
+
0,
|
| 736 |
+
0,
|
| 737 |
+
0,
|
| 738 |
+
0,
|
| 739 |
+
0,
|
| 740 |
+
0,
|
| 741 |
+
0,
|
| 742 |
+
0,
|
| 743 |
+
0,
|
| 744 |
+
0,
|
| 745 |
+
0,
|
| 746 |
+
0,
|
| 747 |
+
0,
|
| 748 |
+
0,
|
| 749 |
+
0,
|
| 750 |
+
0,
|
| 751 |
+
0
|
| 752 |
+
],
|
| 753 |
+
[
|
| 754 |
+
0,
|
| 755 |
+
0,
|
| 756 |
+
0,
|
| 757 |
+
0,
|
| 758 |
+
0,
|
| 759 |
+
0,
|
| 760 |
+
0,
|
| 761 |
+
0,
|
| 762 |
+
0,
|
| 763 |
+
0,
|
| 764 |
+
0,
|
| 765 |
+
0,
|
| 766 |
+
0,
|
| 767 |
+
0,
|
| 768 |
+
0,
|
| 769 |
+
0,
|
| 770 |
+
0,
|
| 771 |
+
0,
|
| 772 |
+
0,
|
| 773 |
+
0,
|
| 774 |
+
0,
|
| 775 |
+
0,
|
| 776 |
+
0,
|
| 777 |
+
0,
|
| 778 |
+
0,
|
| 779 |
+
0,
|
| 780 |
+
0,
|
| 781 |
+
0,
|
| 782 |
+
0,
|
| 783 |
+
0,
|
| 784 |
+
0,
|
| 785 |
+
0,
|
| 786 |
+
0,
|
| 787 |
+
0,
|
| 788 |
+
0
|
| 789 |
+
],
|
| 790 |
+
[
|
| 791 |
+
0,
|
| 792 |
+
0,
|
| 793 |
+
0,
|
| 794 |
+
0,
|
| 795 |
+
1,
|
| 796 |
+
0,
|
| 797 |
+
0,
|
| 798 |
+
0,
|
| 799 |
+
0,
|
| 800 |
+
0,
|
| 801 |
+
0,
|
| 802 |
+
0,
|
| 803 |
+
0,
|
| 804 |
+
0,
|
| 805 |
+
0,
|
| 806 |
+
0,
|
| 807 |
+
0,
|
| 808 |
+
0,
|
| 809 |
+
0,
|
| 810 |
+
0,
|
| 811 |
+
0,
|
| 812 |
+
0,
|
| 813 |
+
0,
|
| 814 |
+
0,
|
| 815 |
+
0,
|
| 816 |
+
0,
|
| 817 |
+
0,
|
| 818 |
+
0,
|
| 819 |
+
0,
|
| 820 |
+
0,
|
| 821 |
+
0,
|
| 822 |
+
0,
|
| 823 |
+
0,
|
| 824 |
+
0,
|
| 825 |
+
0
|
| 826 |
+
],
|
| 827 |
+
[
|
| 828 |
+
0,
|
| 829 |
+
0,
|
| 830 |
+
0,
|
| 831 |
+
0,
|
| 832 |
+
4,
|
| 833 |
+
0,
|
| 834 |
+
0,
|
| 835 |
+
0,
|
| 836 |
+
0,
|
| 837 |
+
0,
|
| 838 |
+
0,
|
| 839 |
+
0,
|
| 840 |
+
0,
|
| 841 |
+
0,
|
| 842 |
+
0,
|
| 843 |
+
0,
|
| 844 |
+
7,
|
| 845 |
+
0,
|
| 846 |
+
0,
|
| 847 |
+
0,
|
| 848 |
+
0,
|
| 849 |
+
0,
|
| 850 |
+
0,
|
| 851 |
+
0,
|
| 852 |
+
0,
|
| 853 |
+
0,
|
| 854 |
+
0,
|
| 855 |
+
0,
|
| 856 |
+
0,
|
| 857 |
+
0,
|
| 858 |
+
0,
|
| 859 |
+
0,
|
| 860 |
+
0,
|
| 861 |
+
2,
|
| 862 |
+
0
|
| 863 |
+
],
|
| 864 |
+
[
|
| 865 |
+
0,
|
| 866 |
+
0,
|
| 867 |
+
0,
|
| 868 |
+
0,
|
| 869 |
+
0,
|
| 870 |
+
0,
|
| 871 |
+
0,
|
| 872 |
+
0,
|
| 873 |
+
0,
|
| 874 |
+
0,
|
| 875 |
+
0,
|
| 876 |
+
0,
|
| 877 |
+
0,
|
| 878 |
+
0,
|
| 879 |
+
0,
|
| 880 |
+
0,
|
| 881 |
+
0,
|
| 882 |
+
0,
|
| 883 |
+
0,
|
| 884 |
+
0,
|
| 885 |
+
0,
|
| 886 |
+
0,
|
| 887 |
+
0,
|
| 888 |
+
0,
|
| 889 |
+
0,
|
| 890 |
+
0,
|
| 891 |
+
0,
|
| 892 |
+
0,
|
| 893 |
+
0,
|
| 894 |
+
0,
|
| 895 |
+
0,
|
| 896 |
+
0,
|
| 897 |
+
0,
|
| 898 |
+
0,
|
| 899 |
+
0
|
| 900 |
+
],
|
| 901 |
+
[
|
| 902 |
+
0,
|
| 903 |
+
0,
|
| 904 |
+
0,
|
| 905 |
+
0,
|
| 906 |
+
3,
|
| 907 |
+
0,
|
| 908 |
+
0,
|
| 909 |
+
0,
|
| 910 |
+
0,
|
| 911 |
+
0,
|
| 912 |
+
2,
|
| 913 |
+
0,
|
| 914 |
+
0,
|
| 915 |
+
0,
|
| 916 |
+
0,
|
| 917 |
+
0,
|
| 918 |
+
0,
|
| 919 |
+
0,
|
| 920 |
+
0,
|
| 921 |
+
0,
|
| 922 |
+
0,
|
| 923 |
+
0,
|
| 924 |
+
0,
|
| 925 |
+
0,
|
| 926 |
+
0,
|
| 927 |
+
0,
|
| 928 |
+
0,
|
| 929 |
+
0,
|
| 930 |
+
0,
|
| 931 |
+
0,
|
| 932 |
+
0,
|
| 933 |
+
0,
|
| 934 |
+
0,
|
| 935 |
+
0,
|
| 936 |
+
0
|
| 937 |
+
],
|
| 938 |
+
[
|
| 939 |
+
0,
|
| 940 |
+
0,
|
| 941 |
+
0,
|
| 942 |
+
0,
|
| 943 |
+
0,
|
| 944 |
+
0,
|
| 945 |
+
0,
|
| 946 |
+
0,
|
| 947 |
+
0,
|
| 948 |
+
0,
|
| 949 |
+
0,
|
| 950 |
+
0,
|
| 951 |
+
0,
|
| 952 |
+
0,
|
| 953 |
+
0,
|
| 954 |
+
0,
|
| 955 |
+
0,
|
| 956 |
+
0,
|
| 957 |
+
0,
|
| 958 |
+
0,
|
| 959 |
+
0,
|
| 960 |
+
0,
|
| 961 |
+
0,
|
| 962 |
+
0,
|
| 963 |
+
0,
|
| 964 |
+
0,
|
| 965 |
+
0,
|
| 966 |
+
0,
|
| 967 |
+
0,
|
| 968 |
+
0,
|
| 969 |
+
0,
|
| 970 |
+
0,
|
| 971 |
+
0,
|
| 972 |
+
0,
|
| 973 |
+
0
|
| 974 |
+
],
|
| 975 |
+
[
|
| 976 |
+
0,
|
| 977 |
+
0,
|
| 978 |
+
0,
|
| 979 |
+
0,
|
| 980 |
+
8,
|
| 981 |
+
0,
|
| 982 |
+
0,
|
| 983 |
+
0,
|
| 984 |
+
0,
|
| 985 |
+
1,
|
| 986 |
+
0,
|
| 987 |
+
0,
|
| 988 |
+
0,
|
| 989 |
+
0,
|
| 990 |
+
0,
|
| 991 |
+
0,
|
| 992 |
+
0,
|
| 993 |
+
0,
|
| 994 |
+
0,
|
| 995 |
+
0,
|
| 996 |
+
0,
|
| 997 |
+
0,
|
| 998 |
+
0,
|
| 999 |
+
0,
|
| 1000 |
+
0,
|
| 1001 |
+
0,
|
| 1002 |
+
0,
|
| 1003 |
+
0,
|
| 1004 |
+
0,
|
| 1005 |
+
0,
|
| 1006 |
+
0,
|
| 1007 |
+
0,
|
| 1008 |
+
0,
|
| 1009 |
+
1,
|
| 1010 |
+
0
|
| 1011 |
+
],
|
| 1012 |
+
[
|
| 1013 |
+
0,
|
| 1014 |
+
0,
|
| 1015 |
+
0,
|
| 1016 |
+
0,
|
| 1017 |
+
1,
|
| 1018 |
+
0,
|
| 1019 |
+
0,
|
| 1020 |
+
0,
|
| 1021 |
+
0,
|
| 1022 |
+
0,
|
| 1023 |
+
0,
|
| 1024 |
+
0,
|
| 1025 |
+
0,
|
| 1026 |
+
0,
|
| 1027 |
+
0,
|
| 1028 |
+
0,
|
| 1029 |
+
0,
|
| 1030 |
+
0,
|
| 1031 |
+
0,
|
| 1032 |
+
0,
|
| 1033 |
+
0,
|
| 1034 |
+
0,
|
| 1035 |
+
0,
|
| 1036 |
+
0,
|
| 1037 |
+
0,
|
| 1038 |
+
0,
|
| 1039 |
+
0,
|
| 1040 |
+
0,
|
| 1041 |
+
0,
|
| 1042 |
+
0,
|
| 1043 |
+
0,
|
| 1044 |
+
0,
|
| 1045 |
+
0,
|
| 1046 |
+
0,
|
| 1047 |
+
0
|
| 1048 |
+
],
|
| 1049 |
+
[
|
| 1050 |
+
0,
|
| 1051 |
+
0,
|
| 1052 |
+
0,
|
| 1053 |
+
0,
|
| 1054 |
+
1,
|
| 1055 |
+
0,
|
| 1056 |
+
0,
|
| 1057 |
+
0,
|
| 1058 |
+
0,
|
| 1059 |
+
0,
|
| 1060 |
+
0,
|
| 1061 |
+
0,
|
| 1062 |
+
0,
|
| 1063 |
+
0,
|
| 1064 |
+
0,
|
| 1065 |
+
0,
|
| 1066 |
+
0,
|
| 1067 |
+
0,
|
| 1068 |
+
0,
|
| 1069 |
+
0,
|
| 1070 |
+
0,
|
| 1071 |
+
0,
|
| 1072 |
+
2,
|
| 1073 |
+
0,
|
| 1074 |
+
0,
|
| 1075 |
+
0,
|
| 1076 |
+
0,
|
| 1077 |
+
0,
|
| 1078 |
+
0,
|
| 1079 |
+
0,
|
| 1080 |
+
0,
|
| 1081 |
+
0,
|
| 1082 |
+
0,
|
| 1083 |
+
0,
|
| 1084 |
+
1
|
| 1085 |
+
],
|
| 1086 |
+
[
|
| 1087 |
+
0,
|
| 1088 |
+
0,
|
| 1089 |
+
0,
|
| 1090 |
+
0,
|
| 1091 |
+
0,
|
| 1092 |
+
0,
|
| 1093 |
+
0,
|
| 1094 |
+
0,
|
| 1095 |
+
0,
|
| 1096 |
+
0,
|
| 1097 |
+
0,
|
| 1098 |
+
0,
|
| 1099 |
+
0,
|
| 1100 |
+
0,
|
| 1101 |
+
0,
|
| 1102 |
+
0,
|
| 1103 |
+
0,
|
| 1104 |
+
0,
|
| 1105 |
+
0,
|
| 1106 |
+
0,
|
| 1107 |
+
0,
|
| 1108 |
+
0,
|
| 1109 |
+
0,
|
| 1110 |
+
0,
|
| 1111 |
+
0,
|
| 1112 |
+
0,
|
| 1113 |
+
0,
|
| 1114 |
+
0,
|
| 1115 |
+
0,
|
| 1116 |
+
0,
|
| 1117 |
+
0,
|
| 1118 |
+
0,
|
| 1119 |
+
0,
|
| 1120 |
+
0,
|
| 1121 |
+
0
|
| 1122 |
+
],
|
| 1123 |
+
[
|
| 1124 |
+
0,
|
| 1125 |
+
0,
|
| 1126 |
+
0,
|
| 1127 |
+
0,
|
| 1128 |
+
0,
|
| 1129 |
+
0,
|
| 1130 |
+
2,
|
| 1131 |
+
0,
|
| 1132 |
+
0,
|
| 1133 |
+
0,
|
| 1134 |
+
0,
|
| 1135 |
+
0,
|
| 1136 |
+
0,
|
| 1137 |
+
0,
|
| 1138 |
+
0,
|
| 1139 |
+
0,
|
| 1140 |
+
0,
|
| 1141 |
+
0,
|
| 1142 |
+
0,
|
| 1143 |
+
0,
|
| 1144 |
+
0,
|
| 1145 |
+
0,
|
| 1146 |
+
0,
|
| 1147 |
+
0,
|
| 1148 |
+
0,
|
| 1149 |
+
0,
|
| 1150 |
+
0,
|
| 1151 |
+
0,
|
| 1152 |
+
0,
|
| 1153 |
+
0,
|
| 1154 |
+
0,
|
| 1155 |
+
0,
|
| 1156 |
+
0,
|
| 1157 |
+
0,
|
| 1158 |
+
1
|
| 1159 |
+
],
|
| 1160 |
+
[
|
| 1161 |
+
0,
|
| 1162 |
+
0,
|
| 1163 |
+
0,
|
| 1164 |
+
0,
|
| 1165 |
+
3,
|
| 1166 |
+
0,
|
| 1167 |
+
0,
|
| 1168 |
+
0,
|
| 1169 |
+
0,
|
| 1170 |
+
0,
|
| 1171 |
+
0,
|
| 1172 |
+
0,
|
| 1173 |
+
0,
|
| 1174 |
+
0,
|
| 1175 |
+
0,
|
| 1176 |
+
0,
|
| 1177 |
+
0,
|
| 1178 |
+
0,
|
| 1179 |
+
0,
|
| 1180 |
+
0,
|
| 1181 |
+
0,
|
| 1182 |
+
0,
|
| 1183 |
+
0,
|
| 1184 |
+
0,
|
| 1185 |
+
0,
|
| 1186 |
+
0,
|
| 1187 |
+
0,
|
| 1188 |
+
1,
|
| 1189 |
+
0,
|
| 1190 |
+
0,
|
| 1191 |
+
0,
|
| 1192 |
+
0,
|
| 1193 |
+
0,
|
| 1194 |
+
0,
|
| 1195 |
+
0
|
| 1196 |
+
],
|
| 1197 |
+
[
|
| 1198 |
+
0,
|
| 1199 |
+
0,
|
| 1200 |
+
0,
|
| 1201 |
+
0,
|
| 1202 |
+
1,
|
| 1203 |
+
0,
|
| 1204 |
+
0,
|
| 1205 |
+
0,
|
| 1206 |
+
0,
|
| 1207 |
+
0,
|
| 1208 |
+
0,
|
| 1209 |
+
0,
|
| 1210 |
+
0,
|
| 1211 |
+
0,
|
| 1212 |
+
0,
|
| 1213 |
+
0,
|
| 1214 |
+
0,
|
| 1215 |
+
0,
|
| 1216 |
+
0,
|
| 1217 |
+
0,
|
| 1218 |
+
0,
|
| 1219 |
+
0,
|
| 1220 |
+
0,
|
| 1221 |
+
0,
|
| 1222 |
+
0,
|
| 1223 |
+
0,
|
| 1224 |
+
0,
|
| 1225 |
+
0,
|
| 1226 |
+
0,
|
| 1227 |
+
0,
|
| 1228 |
+
0,
|
| 1229 |
+
0,
|
| 1230 |
+
0,
|
| 1231 |
+
0,
|
| 1232 |
+
0
|
| 1233 |
+
],
|
| 1234 |
+
[
|
| 1235 |
+
0,
|
| 1236 |
+
0,
|
| 1237 |
+
0,
|
| 1238 |
+
0,
|
| 1239 |
+
0,
|
| 1240 |
+
0,
|
| 1241 |
+
0,
|
| 1242 |
+
0,
|
| 1243 |
+
0,
|
| 1244 |
+
0,
|
| 1245 |
+
0,
|
| 1246 |
+
0,
|
| 1247 |
+
0,
|
| 1248 |
+
0,
|
| 1249 |
+
0,
|
| 1250 |
+
0,
|
| 1251 |
+
0,
|
| 1252 |
+
0,
|
| 1253 |
+
0,
|
| 1254 |
+
0,
|
| 1255 |
+
0,
|
| 1256 |
+
0,
|
| 1257 |
+
1,
|
| 1258 |
+
0,
|
| 1259 |
+
0,
|
| 1260 |
+
0,
|
| 1261 |
+
0,
|
| 1262 |
+
1,
|
| 1263 |
+
0,
|
| 1264 |
+
0,
|
| 1265 |
+
0,
|
| 1266 |
+
0,
|
| 1267 |
+
0,
|
| 1268 |
+
0,
|
| 1269 |
+
3
|
| 1270 |
+
],
|
| 1271 |
+
[
|
| 1272 |
+
0,
|
| 1273 |
+
0,
|
| 1274 |
+
0,
|
| 1275 |
+
0,
|
| 1276 |
+
1,
|
| 1277 |
+
0,
|
| 1278 |
+
0,
|
| 1279 |
+
0,
|
| 1280 |
+
0,
|
| 1281 |
+
0,
|
| 1282 |
+
0,
|
| 1283 |
+
0,
|
| 1284 |
+
0,
|
| 1285 |
+
0,
|
| 1286 |
+
0,
|
| 1287 |
+
0,
|
| 1288 |
+
0,
|
| 1289 |
+
0,
|
| 1290 |
+
0,
|
| 1291 |
+
0,
|
| 1292 |
+
0,
|
| 1293 |
+
0,
|
| 1294 |
+
0,
|
| 1295 |
+
0,
|
| 1296 |
+
0,
|
| 1297 |
+
0,
|
| 1298 |
+
0,
|
| 1299 |
+
0,
|
| 1300 |
+
0,
|
| 1301 |
+
0,
|
| 1302 |
+
0,
|
| 1303 |
+
0,
|
| 1304 |
+
0,
|
| 1305 |
+
0,
|
| 1306 |
+
0
|
| 1307 |
+
],
|
| 1308 |
+
[
|
| 1309 |
+
0,
|
| 1310 |
+
0,
|
| 1311 |
+
0,
|
| 1312 |
+
0,
|
| 1313 |
+
0,
|
| 1314 |
+
0,
|
| 1315 |
+
0,
|
| 1316 |
+
0,
|
| 1317 |
+
0,
|
| 1318 |
+
0,
|
| 1319 |
+
0,
|
| 1320 |
+
0,
|
| 1321 |
+
0,
|
| 1322 |
+
0,
|
| 1323 |
+
0,
|
| 1324 |
+
0,
|
| 1325 |
+
0,
|
| 1326 |
+
0,
|
| 1327 |
+
0,
|
| 1328 |
+
0,
|
| 1329 |
+
0,
|
| 1330 |
+
0,
|
| 1331 |
+
0,
|
| 1332 |
+
0,
|
| 1333 |
+
0,
|
| 1334 |
+
0,
|
| 1335 |
+
0,
|
| 1336 |
+
0,
|
| 1337 |
+
0,
|
| 1338 |
+
0,
|
| 1339 |
+
0,
|
| 1340 |
+
0,
|
| 1341 |
+
0,
|
| 1342 |
+
0,
|
| 1343 |
+
0
|
| 1344 |
+
],
|
| 1345 |
+
[
|
| 1346 |
+
0,
|
| 1347 |
+
0,
|
| 1348 |
+
0,
|
| 1349 |
+
0,
|
| 1350 |
+
0,
|
| 1351 |
+
0,
|
| 1352 |
+
0,
|
| 1353 |
+
0,
|
| 1354 |
+
0,
|
| 1355 |
+
0,
|
| 1356 |
+
0,
|
| 1357 |
+
0,
|
| 1358 |
+
0,
|
| 1359 |
+
0,
|
| 1360 |
+
0,
|
| 1361 |
+
0,
|
| 1362 |
+
0,
|
| 1363 |
+
0,
|
| 1364 |
+
0,
|
| 1365 |
+
0,
|
| 1366 |
+
0,
|
| 1367 |
+
0,
|
| 1368 |
+
0,
|
| 1369 |
+
0,
|
| 1370 |
+
0,
|
| 1371 |
+
0,
|
| 1372 |
+
0,
|
| 1373 |
+
0,
|
| 1374 |
+
0,
|
| 1375 |
+
0,
|
| 1376 |
+
0,
|
| 1377 |
+
0,
|
| 1378 |
+
0,
|
| 1379 |
+
0,
|
| 1380 |
+
2
|
| 1381 |
+
],
|
| 1382 |
+
[
|
| 1383 |
+
0,
|
| 1384 |
+
0,
|
| 1385 |
+
0,
|
| 1386 |
+
0,
|
| 1387 |
+
1,
|
| 1388 |
+
0,
|
| 1389 |
+
0,
|
| 1390 |
+
0,
|
| 1391 |
+
0,
|
| 1392 |
+
0,
|
| 1393 |
+
0,
|
| 1394 |
+
0,
|
| 1395 |
+
0,
|
| 1396 |
+
0,
|
| 1397 |
+
0,
|
| 1398 |
+
0,
|
| 1399 |
+
0,
|
| 1400 |
+
0,
|
| 1401 |
+
0,
|
| 1402 |
+
0,
|
| 1403 |
+
0,
|
| 1404 |
+
0,
|
| 1405 |
+
0,
|
| 1406 |
+
0,
|
| 1407 |
+
0,
|
| 1408 |
+
0,
|
| 1409 |
+
0,
|
| 1410 |
+
0,
|
| 1411 |
+
0,
|
| 1412 |
+
0,
|
| 1413 |
+
0,
|
| 1414 |
+
0,
|
| 1415 |
+
0,
|
| 1416 |
+
0,
|
| 1417 |
+
0
|
| 1418 |
+
],
|
| 1419 |
+
[
|
| 1420 |
+
0,
|
| 1421 |
+
0,
|
| 1422 |
+
0,
|
| 1423 |
+
0,
|
| 1424 |
+
0,
|
| 1425 |
+
0,
|
| 1426 |
+
0,
|
| 1427 |
+
0,
|
| 1428 |
+
0,
|
| 1429 |
+
0,
|
| 1430 |
+
0,
|
| 1431 |
+
0,
|
| 1432 |
+
0,
|
| 1433 |
+
0,
|
| 1434 |
+
0,
|
| 1435 |
+
0,
|
| 1436 |
+
0,
|
| 1437 |
+
0,
|
| 1438 |
+
0,
|
| 1439 |
+
0,
|
| 1440 |
+
0,
|
| 1441 |
+
0,
|
| 1442 |
+
0,
|
| 1443 |
+
0,
|
| 1444 |
+
0,
|
| 1445 |
+
0,
|
| 1446 |
+
0,
|
| 1447 |
+
0,
|
| 1448 |
+
0,
|
| 1449 |
+
0,
|
| 1450 |
+
0,
|
| 1451 |
+
0,
|
| 1452 |
+
0,
|
| 1453 |
+
0,
|
| 1454 |
+
0
|
| 1455 |
+
],
|
| 1456 |
+
[
|
| 1457 |
+
0,
|
| 1458 |
+
0,
|
| 1459 |
+
0,
|
| 1460 |
+
0,
|
| 1461 |
+
10,
|
| 1462 |
+
0,
|
| 1463 |
+
0,
|
| 1464 |
+
0,
|
| 1465 |
+
0,
|
| 1466 |
+
0,
|
| 1467 |
+
0,
|
| 1468 |
+
0,
|
| 1469 |
+
0,
|
| 1470 |
+
0,
|
| 1471 |
+
0,
|
| 1472 |
+
0,
|
| 1473 |
+
0,
|
| 1474 |
+
0,
|
| 1475 |
+
0,
|
| 1476 |
+
0,
|
| 1477 |
+
0,
|
| 1478 |
+
0,
|
| 1479 |
+
0,
|
| 1480 |
+
0,
|
| 1481 |
+
0,
|
| 1482 |
+
0,
|
| 1483 |
+
0,
|
| 1484 |
+
0,
|
| 1485 |
+
0,
|
| 1486 |
+
0,
|
| 1487 |
+
0,
|
| 1488 |
+
0,
|
| 1489 |
+
0,
|
| 1490 |
+
0,
|
| 1491 |
+
4
|
| 1492 |
+
],
|
| 1493 |
+
[
|
| 1494 |
+
0,
|
| 1495 |
+
0,
|
| 1496 |
+
0,
|
| 1497 |
+
0,
|
| 1498 |
+
4,
|
| 1499 |
+
0,
|
| 1500 |
+
7,
|
| 1501 |
+
0,
|
| 1502 |
+
0,
|
| 1503 |
+
0,
|
| 1504 |
+
0,
|
| 1505 |
+
0,
|
| 1506 |
+
0,
|
| 1507 |
+
0,
|
| 1508 |
+
0,
|
| 1509 |
+
0,
|
| 1510 |
+
0,
|
| 1511 |
+
0,
|
| 1512 |
+
0,
|
| 1513 |
+
0,
|
| 1514 |
+
0,
|
| 1515 |
+
0,
|
| 1516 |
+
0,
|
| 1517 |
+
0,
|
| 1518 |
+
0,
|
| 1519 |
+
0,
|
| 1520 |
+
0,
|
| 1521 |
+
1,
|
| 1522 |
+
0,
|
| 1523 |
+
0,
|
| 1524 |
+
0,
|
| 1525 |
+
0,
|
| 1526 |
+
0,
|
| 1527 |
+
0,
|
| 1528 |
+
126
|
| 1529 |
+
]
|
| 1530 |
+
]
|
| 1531 |
+
}
|
runs/20250928_131716/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:502ef69d34545f9ad39573ed199b1ef331cb6fa17eb5977d3841e7596125b2a5
|
| 3 |
+
size 2154772
|
runs/20250928_131716/models/labels.txt
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
ACCOUNT#negative
|
| 2 |
+
CARD#negative
|
| 3 |
+
CARD#neutral
|
| 4 |
+
CARD#positive
|
| 5 |
+
CUSTOMER_SUPPORT#negative
|
| 6 |
+
CUSTOMER_SUPPORT#neutral
|
| 7 |
+
CUSTOMER_SUPPORT#positive
|
| 8 |
+
DISCOUNT#negative
|
| 9 |
+
DISCOUNT#neutral
|
| 10 |
+
DISCOUNT#positive
|
| 11 |
+
INTEREST_RATE#negative
|
| 12 |
+
INTEREST_RATE#neutral
|
| 13 |
+
INTEREST_RATE#positive
|
| 14 |
+
INTERNET_BANKING#negative
|
| 15 |
+
INTERNET_BANKING#neutral
|
| 16 |
+
INTERNET_BANKING#positive
|
| 17 |
+
LOAN#negative
|
| 18 |
+
LOAN#positive
|
| 19 |
+
MONEY_TRANSFER#negative
|
| 20 |
+
MONEY_TRANSFER#positive
|
| 21 |
+
OTHER#negative
|
| 22 |
+
OTHER#neutral
|
| 23 |
+
OTHER#positive
|
| 24 |
+
PAYMENT#negative
|
| 25 |
+
PAYMENT#positive
|
| 26 |
+
PROMOTION#negative
|
| 27 |
+
PROMOTION#neutral
|
| 28 |
+
PROMOTION#positive
|
| 29 |
+
SAVING#negative
|
| 30 |
+
SAVING#neutral
|
| 31 |
+
SAVING#positive
|
| 32 |
+
SECURITY#neutral
|
| 33 |
+
SECURITY#positive
|
| 34 |
+
TRADEMARK#negative
|
| 35 |
+
TRADEMARK#positive
|
runs/20250928_131716/models/model.joblib
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:502ef69d34545f9ad39573ed199b1ef331cb6fa17eb5977d3841e7596125b2a5
|
| 3 |
+
size 2154772
|
runs/20250928_131716/training.log
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2025-09-28 13:17:16,505 - INFO - Starting training run: 20250928_131716
|
| 2 |
+
2025-09-28 13:17:16,505 - INFO - Model: svc_linear
|
| 3 |
+
2025-09-28 13:17:16,505 - INFO - Max features: 20000
|
| 4 |
+
2025-09-28 13:17:16,505 - INFO - N-gram range: (1, 2)
|
| 5 |
+
2025-09-28 13:17:16,505 - INFO - Loading UTS2017_Bank aspect sentiment dataset...
|
| 6 |
+
2025-09-28 13:17:21,253 - INFO - Train samples: 1581
|
| 7 |
+
2025-09-28 13:17:21,253 - INFO - Test samples: 396
|
| 8 |
+
2025-09-28 13:17:21,253 - INFO - Unique labels: 35
|
| 9 |
+
2025-09-28 13:17:21,253 - INFO - Label distribution (train): {np.str_('ACCOUNT#negative'): np.int64(3), np.str_('CARD#negative'): np.int64(47), np.str_('CARD#neutral'): np.int64(1), np.str_('CARD#positive'): np.int64(10), np.str_('CUSTOMER_SUPPORT#negative'): np.int64(288), np.str_('CUSTOMER_SUPPORT#neutral'): np.int64(4), np.str_('CUSTOMER_SUPPORT#positive'): np.int64(328), np.str_('DISCOUNT#negative'): np.int64(13), np.str_('DISCOUNT#neutral'): np.int64(3), np.str_('DISCOUNT#positive'): np.int64(19), np.str_('INTEREST_RATE#negative'): np.int64(45), np.str_('INTEREST_RATE#neutral'): np.int64(1), np.str_('INTEREST_RATE#positive'): np.int64(4), np.str_('INTERNET_BANKING#negative'): np.int64(36), np.str_('INTERNET_BANKING#neutral'): np.int64(2), np.str_('INTERNET_BANKING#positive'): np.int64(19), np.str_('LOAN#negative'): np.int64(48), np.str_('LOAN#positive'): np.int64(13), np.str_('MONEY_TRANSFER#negative'): np.int64(24), np.str_('MONEY_TRANSFER#positive'): np.int64(5), np.str_('OTHER#negative'): np.int64(25), np.str_('OTHER#neutral'): np.int64(3), np.str_('OTHER#positive'): np.int64(26), np.str_('PAYMENT#negative'): np.int64(4), np.str_('PAYMENT#positive'): np.int64(8), np.str_('PROMOTION#negative'): np.int64(13), np.str_('PROMOTION#neutral'): np.int64(2), np.str_('PROMOTION#positive'): np.int64(28), np.str_('SAVING#negative'): np.int64(5), np.str_('SAVING#neutral'): np.int64(1), np.str_('SAVING#positive'): np.int64(4), np.str_('SECURITY#neutral'): np.int64(1), np.str_('SECURITY#positive'): np.int64(1), np.str_('TRADEMARK#negative'): np.int64(33), np.str_('TRADEMARK#positive'): np.int64(514)}
|
| 10 |
+
2025-09-28 13:17:21,253 - INFO - Label distribution (test): {np.str_('ACCOUNT#negative'): np.int64(2), np.str_('CARD#negative'): np.int64(7), np.str_('CARD#neutral'): np.int64(0), np.str_('CARD#positive'): np.int64(1), np.str_('CUSTOMER_SUPPORT#negative'): np.int64(75), np.str_('CUSTOMER_SUPPORT#neutral'): np.int64(1), np.str_('CUSTOMER_SUPPORT#positive'): np.int64(78), np.str_('DISCOUNT#negative'): np.int64(5), np.str_('DISCOUNT#neutral'): np.int64(1), np.str_('DISCOUNT#positive'): np.int64(0), np.str_('INTEREST_RATE#negative'): np.int64(10), np.str_('INTEREST_RATE#neutral'): np.int64(0), np.str_('INTEREST_RATE#positive'): np.int64(0), np.str_('INTERNET_BANKING#negative'): np.int64(12), np.str_('INTERNET_BANKING#neutral'): np.int64(0), np.str_('INTERNET_BANKING#positive'): np.int64(1), np.str_('LOAN#negative'): np.int64(13), np.str_('LOAN#positive'): np.int64(0), np.str_('MONEY_TRANSFER#negative'): np.int64(5), np.str_('MONEY_TRANSFER#positive'): np.int64(0), np.str_('OTHER#negative'): np.int64(10), np.str_('OTHER#neutral'): np.int64(1), np.str_('OTHER#positive'): np.int64(4), np.str_('PAYMENT#negative'): np.int64(0), np.str_('PAYMENT#positive'): np.int64(3), np.str_('PROMOTION#negative'): np.int64(4), np.str_('PROMOTION#neutral'): np.int64(1), np.str_('PROMOTION#positive'): np.int64(5), np.str_('SAVING#negative'): np.int64(1), np.str_('SAVING#neutral'): np.int64(0), np.str_('SAVING#positive'): np.int64(2), np.str_('SECURITY#neutral'): np.int64(1), np.str_('SECURITY#positive'): np.int64(0), np.str_('TRADEMARK#negative'): np.int64(14), np.str_('TRADEMARK#positive'): np.int64(138)}
|
| 11 |
+
2025-09-28 13:17:21,253 - INFO - Selected classifier: SVC
|
| 12 |
+
2025-09-28 13:17:21,253 - INFO - ============================================================
|
| 13 |
+
2025-09-28 13:17:21,253 - INFO - Training: UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2
|
| 14 |
+
2025-09-28 13:17:21,253 - INFO - ============================================================
|
| 15 |
+
2025-09-28 13:17:21,253 - INFO - Creating pipeline with max_features=20000, ngram_range=(1, 2)
|
| 16 |
+
2025-09-28 13:17:21,253 - INFO - Training model...
|
| 17 |
+
2025-09-28 13:17:28,959 - INFO - Training completed in 7.71 seconds
|
| 18 |
+
2025-09-28 13:17:28,959 - INFO - Evaluating on training set...
|
| 19 |
+
2025-09-28 13:17:29,420 - INFO - Training accuracy: 0.9431
|
| 20 |
+
2025-09-28 13:17:29,420 - INFO - Evaluating on test set...
|
| 21 |
+
2025-09-28 13:17:29,530 - INFO - Test accuracy: 0.7172
|
| 22 |
+
2025-09-28 13:17:29,530 - INFO - Prediction time: 0.11 seconds
|
| 23 |
+
2025-09-28 13:17:29,530 - INFO - Classification Report:
|
| 24 |
+
2025-09-28 13:17:29,535 - INFO - precision recall f1-score support
|
| 25 |
+
|
| 26 |
+
ACCOUNT#negative 0.00 0.00 0.00 2
|
| 27 |
+
CARD#negative 1.00 0.14 0.25 7
|
| 28 |
+
CARD#positive 0.00 0.00 0.00 1
|
| 29 |
+
CUSTOMER_SUPPORT#negative 0.49 0.96 0.65 75
|
| 30 |
+
CUSTOMER_SUPPORT#neutral 0.00 0.00 0.00 1
|
| 31 |
+
CUSTOMER_SUPPORT#positive 0.86 0.90 0.88 78
|
| 32 |
+
DISCOUNT#negative 0.00 0.00 0.00 5
|
| 33 |
+
DISCOUNT#neutral 0.00 0.00 0.00 1
|
| 34 |
+
DISCOUNT#positive 0.00 0.00 0.00 0
|
| 35 |
+
INTEREST_RATE#negative 0.56 0.50 0.53 10
|
| 36 |
+
INTERNET_BANKING#negative 0.00 0.00 0.00 12
|
| 37 |
+
INTERNET_BANKING#positive 0.00 0.00 0.00 1
|
| 38 |
+
LOAN#negative 0.88 0.54 0.67 13
|
| 39 |
+
MONEY_TRANSFER#negative 0.00 0.00 0.00 5
|
| 40 |
+
OTHER#negative 0.00 0.00 0.00 10
|
| 41 |
+
OTHER#neutral 0.00 0.00 0.00 1
|
| 42 |
+
OTHER#positive 0.67 0.50 0.57 4
|
| 43 |
+
PAYMENT#positive 0.00 0.00 0.00 3
|
| 44 |
+
PROMOTION#negative 0.00 0.00 0.00 4
|
| 45 |
+
PROMOTION#neutral 0.00 0.00 0.00 1
|
| 46 |
+
PROMOTION#positive 0.33 0.20 0.25 5
|
| 47 |
+
SAVING#negative 0.00 0.00 0.00 1
|
| 48 |
+
SAVING#positive 0.00 0.00 0.00 2
|
| 49 |
+
SECURITY#negative 0.00 0.00 0.00 1
|
| 50 |
+
SECURITY#neutral 0.00 0.00 0.00 1
|
| 51 |
+
TRADEMARK#negative 0.00 0.00 0.00 14
|
| 52 |
+
TRADEMARK#positive 0.89 0.91 0.90 138
|
| 53 |
+
|
| 54 |
+
accuracy 0.72 396
|
| 55 |
+
macro avg 0.21 0.17 0.17 396
|
| 56 |
+
weighted avg 0.65 0.72 0.66 396
|
| 57 |
+
|
| 58 |
+
2025-09-28 13:17:29,539 - INFO - Confusion Matrix shape: (35, 35)
|
| 59 |
+
2025-09-28 13:17:29,672 - INFO - Model saved to runs/20250928_131716/models/model.joblib
|
| 60 |
+
2025-09-28 13:17:29,803 - INFO - Model also saved as runs/20250928_131716/models/UTS2017_Bank_AspectSentiment_SVC_feat20k_ngram1-2.joblib
|
| 61 |
+
2025-09-28 13:17:29,927 - INFO - Model exported as ./uts2017_sentiment_20250928_131716.joblib
|
| 62 |
+
2025-09-28 13:17:29,927 - INFO - Label mapping saved to runs/20250928_131716/models/labels.txt
|
| 63 |
+
2025-09-28 13:17:29,928 - INFO - Metadata saved to runs/20250928_131716/metadata.json
|
uts2017_sentiment_20250928_131716.joblib
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:502ef69d34545f9ad39573ed199b1ef331cb6fa17eb5977d3841e7596125b2a5
|
| 3 |
+
size 2154772
|