DineEase_Review_Rating_Consistency_Final__Khushal_Ramaniipynb

DineEase — Review Rating Consistency (Final Submission)¶

In [1]:

# Setup: imports and environment check
import os, sys
print('Python', sys.version)
import numpy as np, pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score, mean_squared_error, confusion_matrix
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
print('pandas', pd.__version__, 'numpy', np.__version__, 'tensorflow', tf.__version__)
# For reproducibility
RANDOM_SEED = 42
np.random.seed(RANDOM_SEED)
import warnings
warnings.filterwarnings('ignore')

Python 3.12.11 (main, Jun 4 2025, 08:56:18) [GCC 11.4.0]
pandas 2.2.2 numpy 2.0.2 tensorflow 2.19.0

1) Data loading and initial inspection¶

In [2]:

DATA_PATH = "/content/Reviews.csv"
df = pd.read_csv(DATA_PATH)
print("Dataset path:", DATA_PATH)
print("Shape:", df.shape)
print("\nColumn types:")
print(df.dtypes)
print("\nMissing values per column:\n", df.isnull().sum())
display(df.head(8))

Dataset path: /content/Reviews.csv
Shape: (27762, 2)

Column types:
rating float64
review_text object
dtype: object

Missing values per column:
rating 0
review_text 14
dtype: int64

rating
review_text

0
1.0
Their service is worst, pricing in menu is dif...

1
5.0
really appreciate their quality and timing . I...

2
4.0
Went there on a Friday night, the place was su...

3
4.0
A very decent place serving good food.\r\nOrde...

4
5.0
One of the BEST places for steaks in the city....

5
5.0
Really lovely place for steaks and sizzlers. T...

6
5.0
This place ia for ultimate steak lovers!\r\nBo...

7
5.0
It's a shame if you haven't tried Once Upon a ...

2) Exploratory Data Analysis (EDA)¶

In [3]:

# Confirm summary statistics
print("Total rows:", len(df))
print("Missing review_text count:", df['review_text'].isnull().sum())

# Rating distribution
plt.figure(figsize=(8,4))
df['rating'].value_counts().sort_index().plot(kind='bar')
plt.title('Rating distribution (counts by star)')
plt.xlabel('Rating')
plt.ylabel('Count')
plt.show()

# Review length distribution (characters and words)
df['review_text_filled'] = df['review_text'].fillna('')
df['char_len'] = df['review_text_filled'].apply(len)
df['word_len'] = df['review_text_filled'].apply(lambda x: len(str(x).split()))
plt.figure(figsize=(8,4))
plt.hist(df['char_len'], bins=50)
plt.title('Review character length distribution')
plt.xlabel('Characters')
plt.ylabel('Count')
plt.show()

plt.figure(figsize=(8,4))
plt.hist(df['word_len'], bins=50)
plt.title('Review word count distribution')
plt.xlabel('Words')
plt.ylabel('Count')
plt.show()

# Show shortest and longest reviews
display(df.sort_values('char_len').head(6)[['rating','review_text','char_len']])
display(df.sort_values('char_len', ascending=False).head(6)[['rating','review_text','char_len']])

Total rows: 27762
Missing review_text count: 14

rating
review_text
char_len

6527
5.0
NaN
0

27402
3.0
NaN
0

27403
5.0
NaN
0

27404
5.0
NaN
0

26650
5.0
NaN
0

26655
5.0
NaN
0

rating
review_text
char_len

3916
4.0
ÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃ...
39304

14881
4.0
ÃÃÃÃÃÃÃÃÂ£ÃÃÃÃÃÃÃ...
14721

14885
4.0
ÃÃÃÃÃÃÃÃÂ£ÃÃÃÃÃÃÃ...
14721

5837
3.0
ÃÃÃÃÃÃÃÃÂ£ÃÃÃÃÃÃÃ...
12299

27610
4.0
ÃÃÃÃÃÃÃÃÂ£ÃÃÃÃÃÃÃ...
11318

27630
4.0
ÃÃÃÃÃÃÃÃÂ£ÃÃÃÃÃÃÃ...
11318

3) Text preprocessing¶

**Preprocessing steps and reasoning **

Normalize Unicode and lowercase text to reduce vocabulary size.

Remove control characters and extra whitespace.

Preserve punctuation for now (punctuation can bear sentiment cues), but remove extremely non-printable characters.

Create clean_text column used for vectorizers and tokenizers.

In [4]:

import re, unicodedata
def clean_text(s):
if pd.isnull(s):
return ''
s = str(s)
s = unicodedata.normalize('NFKC', s)
s = s.lower()
s = re.sub(r'\s+', ' ', s).strip()
s = re.sub(r'[\x00-\x1f\x7f-\x9f]', '', s)
return s

df['clean_text'] = df['review_text_filled'].apply(clean_text)
display(df[['review_text','clean_text']].head(8))

review_text
clean_text

0
Their service is worst, pricing in menu is dif...
their service is worst, pricing in menu is dif...

1
really appreciate their quality and timing . I...
really appreciate their quality and timing . i...

2
Went there on a Friday night, the place was su...
went there on a friday night, the place was su...

3
A very decent place serving good food.\r\nOrde...
a very decent place serving good food. ordered...

4
One of the BEST places for steaks in the city....
one of the best places for steaks in the city....

5
Really lovely place for steaks and sizzlers. T...
really lovely place for steaks and sizzlers. t...

6
This place ia for ultimate steak lovers!\r\nBo...
this place ia for ultimate steak lovers! both ...

7
It's a shame if you haven't tried Once Upon a ...
it's a shame if you haven't tried once upon a ...

4) Label creation and Train/Validation split¶

Labeling choices and rationale (Markdown):

The dataset contains numerical rating (1.0–5.0). We create an integer label rating_int by rounding the star rating to the nearest integer and clipping to 1–5. This is consistent with the case study's goal of aligning stars and text.

We will train both:

Classification models to predict rating_int (discrete 1–5)

Regression model to predict continuous rating and detect nuanced differences

We perform an 80/20 stratified split on rating_int to preserve class distribution.

In [5]:

# Create integer rating label
df['rating_int'] = df['rating'].round().astype(int).clip(1,5)
df['rating_int'].value_counts().sort_index()

Out[5]:

count

rating_int

1
3126

2
2113

3
3762

4
10643

5
8118

dtype: int64

In [6]:

# Train/validation split (stratify by rating_int)
train_df, val_df = train_test_split(df, test_size=0.20, random_state=RANDOM_SEED, stratify=df['rating_int'])
print('Train size:', train_df.shape, 'Val size:', val_df.shape)
print('\nTrain rating distribution:')
print(train_df['rating_int'].value_counts(normalize=True).sort_index())
print('\nVal rating distribution:')
print(val_df['rating_int'].value_counts(normalize=True).sort_index())

Train size: (22209, 7) Val size: (5553, 7)

Train rating distribution:
rating_int
1 0.112612
2 0.076095
3 0.135531
4 0.383358
5 0.292404
Name: proportion, dtype: float64

Val rating distribution:
rating_int
1 0.112552
2 0.076175
3 0.135422
4 0.383396
5 0.292455
Name: proportion, dtype: float64

Split rationale: Stratified splitting ensures the class balance in train/validation sets matches the original dataset, reducing evaluation bias.

5) Baseline — TF-IDF + Logistic Regression (Classification)¶

In [7]:

tfidf = TfidfVectorizer(max_features=25000, ngram_range=(1,2), min_df=5)
X_train_tfidf = tfidf.fit_transform(train_df['clean_text'])
X_val_tfidf = tfidf.transform(val_df['clean_text'])

lr = LogisticRegression(max_iter=1000, class_weight='balanced', random_state=RANDOM_SEED)
lr.fit(X_train_tfidf, train_df['rating_int'])
pred_val_lr = lr.predict(X_val_tfidf)
print('Validation accuracy (LR):', accuracy_score(val_df['rating_int'], pred_val_lr))
print('\nClassification report (LR):\n', classification_report(val_df['rating_int'], pred_val_lr, digits=3))

Validation accuracy (LR): 0.8328831262380695

Classification report (LR):
precision recall f1-score support

1 0.841 0.909 0.874 625
2 0.801 0.801 0.801 423
3 0.757 0.795 0.776 752
4 0.866 0.806 0.835 2129
5 0.834 0.864 0.849 1624

accuracy 0.833 5553
macro avg 0.820 0.835 0.827 5553
weighted avg 0.834 0.833 0.833 5553

Random Forest on TF-IDF (optional ensemble)¶

Random Forest can capture non-linear interactions between TF-IDF features but is heavier. We include it as an ensemble candidate.

In [8]:

rf = RandomForestClassifier(n_estimators=200, random_state=RANDOM_SEED, n_jobs=-1)
rf.fit(X_train_tfidf, train_df['rating_int'])
pred_val_rf = rf.predict(X_val_tfidf)
print('Validation accuracy (RF):', accuracy_score(val_df['rating_int'], pred_val_rf))

Validation accuracy (RF): 0.8775436700882406

6) Deep Learning approach — Tokenizer + Embedding + Bidirectional LSTM (Regression)¶

Rationale (Markdown):

Regression output predicts continuous rating, allowing nuanced differences between stars.

A BiLSTM can capture sequence information and context compared to bag-of-words models.

We will train a lightweight model suitable for Colab GPU. For production, consider fine-tuning a transformer.

In [12]:

# Tokenize and prepare sequences
MAX_VOCAB = 30000
MAX_LEN = 150
tokenizer = keras.preprocessing.text.Tokenizer(num_words=MAX_VOCAB, oov_token='<OOV>')
tokenizer.fit_on_texts(train_df['clean_text'])

X_train_seq = tokenizer.texts_to_sequences(train_df['clean_text'])
X_val_seq = tokenizer.texts_to_sequences(val_df['clean_text'])

X_train_pad = keras.preprocessing.sequence.pad_sequences(X_train_seq, maxlen=MAX_LEN, padding='post', truncating='post')
X_val_pad = keras.preprocessing.sequence.pad_sequences(X_val_seq, maxlen=MAX_LEN, padding='post', truncating='post')

y_train_reg = train_df['rating'].values.astype('float32')
y_val_reg = val_df['rating'].values.astype('float32')

print('Shapes:', X_train_pad.shape, X_val_pad.shape, y_train_reg.shape)

Shapes: (22209, 150) (5553, 150) (22209,)

In [14]:

# make sure MAX_VOCAB and MAX_LEN are correct ints
MAX_VOCAB = 30000
MAX_LEN = 150
emb_dim = 128

model = keras.Sequential([
layers.Embedding(input_dim=MAX_VOCAB, output_dim=emb_dim, input_length=MAX_LEN), # input_length forces build
layers.Bidirectional(layers.LSTM(64)),
layers.Dropout(0.3),
layers.Dense(64, activation='relu'),
layers.Dropout(0.2),
layers.Dense(1, activation='linear')
])

model.build(input_shape=(None, MAX_LEN)) # <-- add this
model.summary()

Model: "sequential_3"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type) ┃ Output Shape ┃ Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ embedding_3 (Embedding) │ (None, 150, 128) │ 3,840,000 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ bidirectional_3 (Bidirectional) │ (None, 128) │ 98,816 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_6 (Dropout) │ (None, 128) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_6 (Dense) │ (None, 64) │ 8,256 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dropout_7 (Dropout) │ (None, 64) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_7 (Dense) │ (None, 1) │ 65 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

Total params: 3,947,137 (15.06 MB)

Trainable params: 3,947,137 (15.06 MB)

Non-trainable params: 0 (0.00 B)

In [17]:

# --- Compile
import tensorflow as tf
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
loss='mse', # regression: mean squared error
metrics=[tf.keras.metrics.RootMeanSquaredError(name='rmse')]
)

# optional: callbacks (early stopping to avoid overfitting)
callbacks = [
tf.keras.callbacks.EarlyStopping(monitor='val_rmse', patience=3, restore_best_weights=True)
]

# --- Train ---
EPOCHS = 12
history = model.fit(
X_train_pad, y_train_reg,
validation_data=(X_val_pad, y_val_reg),
epochs=EPOCHS,
batch_size=128,
callbacks=callbacks
)

history = model.fit(X_train_pad, y_train_reg, validation_data=(X_val_pad, y_val_reg), epochs=EPOCHS, batch_size=128)

Epoch 1/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 7s 23ms/step - loss: 0.2155 - rmse: 0.4642 - val_loss: 0.2376 - val_rmse: 0.4875
Epoch 2/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.1957 - rmse: 0.4424 - val_loss: 0.2320 - val_rmse: 0.4817
Epoch 3/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 23ms/step - loss: 0.1826 - rmse: 0.4273 - val_loss: 0.2372 - val_rmse: 0.4870
Epoch 4/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.1668 - rmse: 0.4084 - val_loss: 0.2241 - val_rmse: 0.4733
Epoch 5/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.1541 - rmse: 0.3925 - val_loss: 0.2253 - val_rmse: 0.4746
Epoch 6/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 5s 30ms/step - loss: 0.1462 - rmse: 0.3824 - val_loss: 0.2233 - val_rmse: 0.4725
Epoch 7/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 21ms/step - loss: 0.1412 - rmse: 0.3757 - val_loss: 0.2192 - val_rmse: 0.4682
Epoch 8/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 5s 21ms/step - loss: 0.1339 - rmse: 0.3659 - val_loss: 0.2208 - val_rmse: 0.4699
Epoch 9/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 23ms/step - loss: 0.1304 - rmse: 0.3611 - val_loss: 0.2339 - val_rmse: 0.4836
Epoch 10/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 21ms/step - loss: 0.1228 - rmse: 0.3503 - val_loss: 0.2242 - val_rmse: 0.4735
Epoch 1/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 21ms/step - loss: 0.1349 - rmse: 0.3672 - val_loss: 0.2179 - val_rmse: 0.4668
Epoch 2/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 23ms/step - loss: 0.1286 - rmse: 0.3586 - val_loss: 0.2177 - val_rmse: 0.4665
Epoch 3/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 21ms/step - loss: 0.1203 - rmse: 0.3467 - val_loss: 0.2208 - val_rmse: 0.4699
Epoch 4/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.1174 - rmse: 0.3427 - val_loss: 0.2183 - val_rmse: 0.4672
Epoch 5/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 23ms/step - loss: 0.1128 - rmse: 0.3358 - val_loss: 0.2126 - val_rmse: 0.4611
Epoch 6/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 5s 20ms/step - loss: 0.1045 - rmse: 0.3233 - val_loss: 0.2102 - val_rmse: 0.4585
Epoch 7/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.1064 - rmse: 0.3262 - val_loss: 0.2100 - val_rmse: 0.4582
Epoch 8/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 24ms/step - loss: 0.1007 - rmse: 0.3174 - val_loss: 0.2117 - val_rmse: 0.4601
Epoch 9/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.0948 - rmse: 0.3079 - val_loss: 0.2046 - val_rmse: 0.4524
Epoch 10/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 20ms/step - loss: 0.0882 - rmse: 0.2970 - val_loss: 0.2056 - val_rmse: 0.4534
Epoch 11/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 22ms/step - loss: 0.0862 - rmse: 0.2935 - val_loss: 0.2038 - val_rmse: 0.4514
Epoch 12/12
174/174 ━━━━━━━━━━━━━━━━━━━━ 4s 22ms/step - loss: 0.0843 - rmse: 0.2903 - val_loss: 0.2069 - val_rmse: 0.4548

7) Evaluation and Mismatch Detection¶

In [19]:

# Predict and evaluate
val_pred_reg = model.predict(X_val_pad).flatten()

# Compute RMSE manually for compatibility
rmse = np.sqrt(mean_squared_error(y_val_reg, val_pred_reg))
print('Validation RMSE (regression):', rmse)

# Rounded predictions
val_pred_round = np.rint(val_pred_reg).astype(int)
val_pred_round = np.clip(val_pred_round, 1, 5)
print('\nRounded prediction accuracy:', accuracy_score(val_df['rating_int'], val_pred_round))
print('\nClassification report (rounded predictions):\n', classification_report(val_df['rating_int'], val_pred_round, digits=3))

174/174 ━━━━━━━━━━━━━━━━━━━━ 2s 10ms/step
Validation RMSE (regression): 0.4548082984309827

Rounded prediction accuracy: 0.8292814694759589

Classification report (rounded predictions):
precision recall f1-score support

1 0.876 0.894 0.885 625
2 0.733 0.721 0.727 423
3 0.689 0.774 0.729 752
4 0.865 0.810 0.837 2129
5 0.863 0.884 0.873 1624

accuracy 0.829 5553
macro avg 0.805 0.817 0.810 5553
weighted avg 0.832 0.829 0.830 5553

In [20]:

# Flag mismatches where |rounded_pred - actual| >= 1
val_df_eval = val_df.copy().reset_index(drop=True)
val_df_eval['pred_rating_cont'] = val_pred_reg
val_df_eval['pred_rating_round'] = val_pred_round
val_df_eval['abs_diff'] = (val_df_eval['pred_rating_round'] - val_df_eval['rating_int']).abs()
mismatches = val_df_eval[val_df_eval['abs_diff'] >= 1.0].sort_values(by='abs_diff', ascending=False)
print('Total validation rows:', val_df_eval.shape[0])
print('Flagged mismatches (|diff|>=1):', mismatches.shape[0])
display(mismatches[['rating','rating_int','review_text','pred_rating_cont','pred_rating_round','abs_diff']].head(20))

Total validation rows: 5553
Flagged mismatches (|diff|>=1): 948

rating
rating_int
review_text
pred_rating_cont
pred_rating_round
abs_diff

2236
1.0
1
We tried reaching the number provided for the ...
4.608463
5
4

313
5.0
5
but rotis is not given he missed the one item
1.007297
1
4

3881
5.0
5
yaar paaji kamaal kardia!\r\nEk north india ka...
1.404939
1
4

999
1.0
1
If you have plenty of money a must try to wast...
3.617565
4
3

2214
1.0
1
I donÃÃÃÃÃÃÃÃt mind having a h...
3.920128
4
3

308
1.0
1
I didn't taken any delivery
4.214668
4
3

1104
1.0
1
mx closed The food is okay, though one might ...
4.007859
4
3

4502
1.0
1
So it was like a camel. I had to recook spaghe...
4.114514
4
3

2900
1.0
1
Food day by day is becoming worse .... I have ...
3.885747
4
3

4758
1.0
1
Iate delivery
3.866074
4
3

2408
3.5
4
I went to Paprica(Infusion) with my team and s...
1.287852
1
3

1394
1.0
1
Delayed
4.170616
4
3

1421
5.0
5
very tasty and well prepared food
2.496819
2
3

4941
1.0
1
i ordered crispy corn,gobi manchurian,crispy b...
3.748415
4
3

1483
1.0
1
Delayed
4.170616
4
3

3834
3.5
4
I went to Paprica(Infusion) with my team and s...
1.287852
1
3

3490
5.0
5
Quantity is very less
2.380265
2
3

1586
3.5
4
The last time I complained about the food and ...
1.060114
1
3

3995
4.0
4
ÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃÃ...
1.027788
1
3

1790
4.0
4
Let me just start by saying zee having its own...
1.117955
1
3

Mismatch analysis (Markdown):

The table above lists top mismatches. These should be reviewed manually.

Common reasons for mismatches include sarcasm, short reviews, multi-aspect feedback (positive about food but negative about service), or rating typos.