Model Card for Lkkash/distilbert-book-reviews

Model Details

Model Description

• Developed by: Lkkash
• Model type: DistilBERT-based binary sequence classification model
• Language(s) (NLP): English
• License: Apache license 2.0
• Finetuned from model: distilbert-base-uncased

Model Sources

• Repository: https://huggingface.co/distilbert/distilbert-base-uncased (Base model)
• Repository: https://huggingface.co/datasets/Lkkash/book-reviews-from-amazon-and-goodreads (For the dataset)

Uses

This model is intended for sentiment classification of English-language book reviews. Given the text of a book review, the model predicts whether the review expresses a good/positive or bad/negative sentiment. The model can be used for: Classifying individual book reviews by sentiment. Automatically categorizing large collections of book reviews. Exploring sentiment trends in book-review datasets. Educational and research projects involving NLP and text classification. The model is intended primarily for English-language book reviews similar to those in its training data.

Direct Use

This model is intended for sentiment classification of English-language book reviews. Given the text of a book review, the model predicts whether the review expresses a good/positive or bad/negative sentiment. The model can be used for: Classifying individual book reviews by sentiment. Automatically categorizing large collections of book reviews. Exploring sentiment trends in book-review datasets. Educational and research projects involving NLP and text classification. The model is intended primarily for English-language book reviews similar to those in its training data.

Downstream Use

The model can be integrated into larger applications or data-processing pipelines that work with book reviews. Potential downstream applications include: Book-review sentiment analysis tools. Recommendation or review-analysis systems. Dashboards showing positive and negative review trends. Automated organization or filtering of book reviews. NLP research and experimentation. For downstream applications, users should evaluate the model on data representative of their specific use case before deployment.

Out-of-Scope Use

This model is not intended for: High-stakes decision-making involving individuals. Determining a person's character, personality, or suitability based on their writing. Moderation or classification of content unrelated to book reviews. Sentiment analysis of languages other than English without additional evaluation. Making factual judgments about the quality or value of a book. Generating book reviews or other text. Use as a general-purpose sentiment classifier without task-specific evaluation. The model may perform poorly on text that differs substantially from its training data, including very short text, languages other than English, domains unrelated to books, sarcasm, ambiguous statements, or reviews containing mixed or nuanced sentiments. Predictions should therefore be treated as model outputs rather than definitive judgments.

Bias, Risks, and Limitations

This model may reflect biases present in the Amazon and Goodreads reviews used for training, including differences in writing styles, reviewer preferences, and book genres. It has not been evaluated across different demographic groups or review sources. The model may perform poorly on sarcasm, irony, mixed or ambiguous reviews, very short text, non-English text, or domains outside book reviews. Inputs longer than 128 tokens may also be truncated. The model predicts the sentiment expressed in a review and should not be interpreted as an objective measure of book quality. It was evaluated on a held-out validation split from the same dataset, so real-world performance may differ. Outputs should be reviewed before being used in important decisions.

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

How to Get Started with the Model

Run the model directly from this code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

MODEL_NAME = "Lkkash/distilbert-book-reviews"

model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_NAME,
    num_labels=2
)

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

text = """
I don't know why i did not understand the book properly, like i don't get what the book was supposed to convey.
"""

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    padding=True
)

with torch.inference_mode():
    outputs = model(**inputs)

logits = outputs.logits

probabilities = torch.softmax(logits, dim=1)

print("\nOutput Probabilities:")
print(f"Bad:  {probabilities[0][0].item() * 100:.2f}%")
print(f"Good: {probabilities[0][1].item() * 100:.2f}%")

prediction = torch.argmax(logits, dim=1).item()

id_to_label = {
    0: "Bad",
    1: "Good"
}

print("\nPrediction:")
print(id_to_label[prediction])

Training Details

Training Data

Dataset used - https://huggingface.co/datasets/Lkkash/book-reviews-from-amazon-and-goodreads The model was fine-tuned on a dataset containing approximately 107,000 book reviews collected from Amazon and Goodreads. The training data was loaded from books_only_reviews_clean.csv. Only the text and label columns were used for training. The label column was renamed to labels for compatibility with the Hugging Face Trainer. The labels represent binary sentiment: 0 — Bad/negative review 1 — Good/positive review The dataset was divided into training and validation subsets using an 80/20 split with stratified sampling. This resulted in approximately: Training set: 85,600 reviews Validation set: 21,400 reviews Stratified sampling was used to maintain a similar proportion of the two sentiment classes in both subsets. The split used a random seed of 42.

Fine-tuning code

The model was finetuned from this code:

# Important libraries to install
# pip install transformers scikit-learn datasets pandas numpy torch 'accelerate>=1.10'
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from transformers import TrainingArguments, Trainer
from sklearn.metrics import (
    accuracy_score,
    f1_score,
    precision_score,
    recall_score
)
from sklearn.model_selection import train_test_split
from datasets import Dataset
import pandas as pd
import numpy as np
import torch

FILE_NAME = "books_only_reviews_clean.csv"  ## Dataset (saved locally during finetuning)
MODEL_NAME = "distilbert-base-uncased"      ## Base model to finetune
NUM_LABELS = 2                              ## Total number of different labels (0,1 for book-review sentiment)

# Load model and tokenizer
def load_model():
    model = AutoModelForSequenceClassification.from_pretrained(
        MODEL_NAME,
        num_labels=NUM_LABELS,
        problem_type="single_label_classification"
    )
    tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
    return model, tokenizer

model, tokenizer = load_model()

# Load data
df = pd.read_csv(FILE_NAME)
# Keep only the columns we need
df = df[["text", "label"]]

# Rename label -> labels because Hugging Face Trainer expects the target column to be called "labels"
df = df.rename(columns={"label": "labels"})
# Make sure labels are integers
df["labels"] = df["labels"].astype(int)
print(df.head(10))
print(df["labels"].value_counts())


# Tokenization
def tokenize(example):
    return tokenizer(
        example["text"],
        truncation=True,
        padding="max_length",
        max_length=128
    )

# Train / validation split
# Train = The dataset the model learns from to find patterns.
# Validation = The dataset used to tune settings and stop the model from memorizing.
# Test = The final dataset used to unbiasedly grade the finished model.

train_df, val_df = train_test_split(
    df,
    test_size=0.2,
    random_state=42,
    shuffle=True,
    stratify=df["labels"]       # Keeps 0/1 proportions similar
)

# Convert to huggingFace datasets
train_dataset = Dataset.from_pandas(
    train_df,
    preserve_index=False
)

val_dataset = Dataset.from_pandas(
    val_df,
    preserve_index=False
)

# Tokenize
train_dataset = train_dataset.map(
    tokenize,
    batched=True
)

val_dataset = val_dataset.map(
    tokenize,
    batched=True
)

# Tell Hugging Face which columns to return as tensors
train_dataset.set_format(
    type="torch",
    columns=["input_ids", "attention_mask", "labels"]
)

val_dataset.set_format(
    type="torch",
    columns=["input_ids", "attention_mask", "labels"]
)

# Compute metrics
def compute_metrics(eval_pred):
    logits, labels = eval_pred
    # For single-label classification: choose the class with the highest logit
    predictions = np.argmax(logits, axis=1)
    return {
        "accuracy": accuracy_score(labels, predictions),
        "f1": f1_score(
            labels,
            predictions,
            average="binary"
        ),
        "precision": precision_score(
            labels,
            predictions,
            average="binary",
            zero_division=0
        ),
        "recall": recall_score(
            labels,
            predictions,
            average="binary",
            zero_division=0
        )
    }

# Device (mps for mac)
device = torch.device(
    "mps" if torch.backends.mps.is_available() else
    "cuda" if torch.cuda.is_available() else
    "cpu"
)
print("Using device:", device)
model.to(device)

# Training arguments
training_args = TrainingArguments(
    output_dir="./book_sentimental",
    num_train_epochs=2,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=32,
    learning_rate=3e-5,
    weight_decay=0.01,
    eval_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,
    dataloader_num_workers=0,
    dataloader_pin_memory=False,
    logging_steps=100,
    report_to="none",
    fp16=False,
    bf16=False,
    seed=42,
)

# Trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    compute_metrics=compute_metrics
)


# Train the model
trainer.train()

# Save the model
trainer.save_model("./distilbert-book-reviews")
tokenizer.save_pretrained(
    "./distilbert-book-reviews"
)

Training Procedure

The pretrained distilbert-base-uncased model was fine-tuned for binary single-label sequence classification. A classification head with two output labels was used. Training was performed using the Hugging Face Transformers Trainer API with PyTorch.

Preprocessing

Book review text was tokenized using the distilbert-base-uncased tokenizer. The following preprocessing configuration was used: Tokenizer: distilbert-base-uncased Maximum sequence length: 128 tokens Truncation: Enabled Padding: max_length Input features: input_ids and attention_mask Target: Binary labels value (0 or 1) The resulting datasets were converted to PyTorch tensors before training.

Training Hyperparameters

Training regime: FP32 / standard full-precision training Base model: distilbert-base-uncased Number of labels: 2 Number of epochs: 2 Training batch size per device: 16 Evaluation batch size per device: 32 Learning rate: 3e-5 Weight decay: 0.01 Random seed: 42 Evaluation strategy: Once per epoch Checkpoint saving: Once per epoch Best model criterion: Lowest validation loss Mixed precision: Disabled (fp16=False, bf16=False)

Speeds, Sizes, Times

The complete training run took approximately 5,199 seconds (1.44 hours) over two epochs. Training throughput was approximately: Training samples/second: 32.8 Training steps/second: 2.05 Total training steps: 10,658 Final training loss: 0.1365 The model and tokenizer were saved after training.

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on a held-out validation set containing 20% of the book review dataset. The data was split using stratified sampling with a random seed of 42 to preserve the class distribution between the two sentiment classes. The dataset consists of book reviews from Amazon and Goodreads. The target labels are: 0 — Bad/negative review 1 — Good/positive review The validation data was not used to update the model parameters during training. Dataset Card - https://huggingface.co/datasets/Lkkash/book-reviews-from-amazon-and-goodreads

Factors

The primary evaluation factor is the sentiment class of the book review: Bad/negative reviews (0) Good/positive reviews (1) The current evaluation does not separately report performance by review source, genre, author, or other demographic or domain factors.

Metrics

The model was evaluated using four metrics: Accuracy: The proportion of reviews correctly classified across both classes. F1 Score: The harmonic mean of precision and recall, providing a balanced measure of classification performance. Precision: The proportion of reviews predicted as good/positive (1) that were actually good/positive. Recall: The proportion of actual good/positive reviews (1) correctly identified by the model. These metrics were selected to provide both an overall measure of classification accuracy and a more detailed assessment of positive-class prediction performance.

Results

The model was evaluated after each training epoch on the held-out validation set. The best model was selected based on the lowest validation loss. The model achieved the following validation performance across the training epochs:

Epoch	Validation Loss	Accuracy	F1 Score	Precision	Recall
1	0.1428	95.31%	95.50%	96.18%	94.84%
2	0.1741	95.53%	95.76%	95.39%	96.13%

Although the second epoch produced slightly higher accuracy, F1 score, and recall, the validation loss increased from 0.1428 to 0.1741. Since the training configuration selected the best model based on validation loss, the epoch 1 checkpoint was selected as the best model. The final training run took approximately 5,199 seconds (1.44 hours) on an Apple MPS device. The final training loss was 0.1365.

Summary

Technical Specifications

Model Architecture and Objective

This model is based on distilbert-base-uncased, a pretrained DistilBERT transformer model. It was fine-tuned for binary sentiment classification on a dataset of book reviews collected from Amazon and Goodreads.A sequence classification head with two output labels was added to the pretrained DistilBERT model. The model classifies book reviews into two categories: 0 — Bad/negative review 1 — Good/positive review The DistilBERT tokenizer is used to preprocess the review text. Reviews are tokenized with truncation and a maximum sequence length of 128 tokens, with padding applied to produce fixed-length inputs. The dataset was divided into training and validation sets using an 80/20 split with stratified sampling to preserve the class distribution between the two sets. The model was fine-tuned using a single-label classification objective. Model performance was evaluated using accuracy, F1 score, precision, and recall.

Compute Infrastructure

The model was fine-tuned using PyTorch and the Hugging Face Transformers Trainer framework. The training script automatically selects an available Apple MPS device, CUDA GPU, or CPU.

Software

The model was trained using:

Python PyTorch Hugging Face Transformers Hugging Face Datasets Hugging Face Tokenizers Pandas NumPy Scikit-learn

The model was fine-tuned using the Hugging Face Trainer API with the following training configuration:

Base model: distilbert-base-uncased Number of labels: 2 Epochs: 2 Training batch size per device: 16 Evaluation batch size per device: 32 Learning rate: 3e-5 Weight decay: 0.01 Maximum sequence length: 128 Validation split: 20% Random seed: 42 Mixed-precision training: disabled Best model selection: lowest validation loss

Citation

BibTeX:

@misc{laksh_distilbert_book_reviews, author = {Lkkash}, title = {DistilBERT Book Reviews}, year = {2026}, publisher = {Hugging Face}, url = {https://huggingface.co/Lkkash/distilbert-book-reviews} }

APA:

Lkkash. (2026). DistilBERT book reviews [Fine-tuned DistilBERT model]. Hugging Face. https://huggingface.co/Lkkash/distilbert-book-reviews