omaradly/jira-task-duration-classifier

Jira Task Duration Classifier

This repository packages a machine learning API that predicts whether a Jira-style software issue is likely to be Short, Standard, or Long-running based on the issue text and metadata.

The repository includes the trained model, a reproducible training script, cleaned sample data, evaluation artifacts, notebooks, and a FastAPI inference service.

What It Predicts

The model predicts an ordinal duration class, not an exact completion date.

Class	Duration rule used during labeling
Short	3 days or less
Standard	More than 3 days and up to 15 days
Long-running	More than 15 days

The prediction is useful for early planning, triage, backlog sizing, and identifying tasks that may need more discovery before assignment.

Project Highlights

• Model: scikit-learn Pipeline with TF-IDF text features, one-hot categorical features, scaled numeric features, and Logistic Regression.
• API: FastAPI service with /health, /predict, /docs, and /openapi.json.
• Inputs: issue summary, description, priority, issue type, project metadata, date metadata, labels, assignee flag, votes, and watches.
• Output: predicted class plus class probabilities.
• Evaluation: held-out stratified test set with 20,315 examples and 0.80 accuracy.
• Data source: Apache Jira issues dataset from Kaggle.

Repository Contents

task-duration-classifier-hf/
|-- api/
|   `-- main.py
|-- data/
|   `-- processed/
|       |-- final_cleaned_sample.csv
|       `-- jira_issues_cleaned_sample.csv
|-- models/
|   `-- duration_logistic_regression_classifier.joblib
|-- model_tests/
|   |-- classification_report.txt
|   |-- confusion_matrix.png
|   `-- metric_charts/
|-- notebooks/
|   |-- 01-exploratory-data-analysis.ipynb
|   |-- 02-data-cleaning.ipynb
|   |-- 03-feature-engineering.ipynb
|   `-- rebuild_data_notebooks.py
`-- training/
    |-- evaluate_data_filters.py
    `-- model_training.py

How It Works

The model is trained from completed Jira issues. For each issue, the pipeline combines:

• total_text: summary plus description, vectorized with TF-IDF unigrams and bigrams.
• Categorical fields: priority, issue type, project key, project category, created year, and created month.
• Numeric fields: text lengths, word counts, description presence, label count, assignee presence, votes, and watches.

The training script builds a single scikit-learn pipeline:

1. TfidfVectorizer(max_features=10000, stop_words="english", ngram_range=(1, 2), min_df=5, max_df=0.9, sublinear_tf=True)
2. OneHotEncoder(handle_unknown="ignore") for categorical metadata
3. SimpleImputer(strategy="median") and StandardScaler() for numeric metadata
4. LogisticRegression(solver="saga", penalty="l2", max_iter=1200, random_state=42)

The saved artifact is:

models/duration_logistic_regression_classifier.joblib

Data Source And Preparation

The original data source is the Kaggle Apache Jira Issues dataset:

https://www.kaggle.com/datasets/tedlozzo/apaches-jira-issues

The notebooks document the data workflow:

Notebook	Purpose
01-exploratory-data-analysis.ipynb	Inspects raw fields, missingness, timestamps, text lengths, category distributions, and duration spread.
02-data-cleaning.ipynb	Keeps completed issues, validates created/resolution timestamps, normalizes text and category fields, and creates count/flag features.
03-feature-engineering.ipynb	Creates duration_days, assigns duration labels, filters noisy class boundaries, controls project dominance, and balances classes.

Dataset flow from the source project:

Stage	Rows
Raw rows loaded in cleaning notebook	1,149,323
Completed issues after timestamp filtering	945,103
After duplicate removal	937,203
After consistency filtering	258,722
After project/class cap	218,629
Final balanced modeling rows	101,571
Rows per final duration class	33,857
Held-out test rows	20,315

This Hugging Face repository includes sample processed CSVs for inspection. The full raw dataset is not included because it is large and should be downloaded from Kaggle.

Sample class distribution in data/processed/final_cleaned_sample.csv:

Class	Sample rows
Short	27
Standard	37
Long-running	36

Example processed row:

Field	Value
summary	MailConsumer: Move mail after processing
issuetype_name	New Feature
priority_name	Major
project_key	CAMEL
has_description	1
duration_category	Short

Evaluation Metrics

The model was evaluated on a stratified held-out test split of 20,315 examples.

Class	Precision	Recall	F1-score	Support
Long-running	0.79	0.71	0.74	6,771
Short	0.85	0.87	0.86	6,772
Standard	0.76	0.81	0.79	6,772
Accuracy			0.80	20,315
Macro avg	0.80	0.80	0.80	20,315
Weighted avg	0.80	0.80	0.80	20,315

Confusion Matrix

Per-Class Metrics

The strongest class is Short with an F1-score of 0.86. Long-running is the hardest class, mainly because it has lower recall: some long tasks are predicted as shorter categories.

API Usage

Run the API locally from the repository root:

python -m pip install fastapi uvicorn pandas scikit-learn joblib pydantic
python -m uvicorn api.main:app --host 0.0.0.0 --port 8000

Open the interactive API docs:

http://localhost:8000/docs

Endpoints

Method	Endpoint	Description
GET	/	API metadata
GET	/health	Service status and model metadata
POST	/predict	Predicts duration class and probabilities
GET	/docs	Swagger UI
GET	/openapi.json	OpenAPI schema

Example Request

curl -X POST "http://localhost:8000/predict" \
  -H "Content-Type: application/json" \
  -H "Origin: http://localhost:8000" \
  -d '{
    "summary": "Fix CORS error on user endpoint",
    "description": "Frontend requests are failing in production because the API response is missing an Access-Control-Allow-Origin header.",
    "issuetype_name": "Bug",
    "priority_name": "High",
    "project_key": "WEB",
    "project_category_name": "Application",
    "created_year": 2026,
    "created_month": 7,
    "labels_count": 2,
    "has_assignee": 1,
    "votes_votes": 1,
    "watches_watch_count": 4
  }'

Example response shape:

{
  "duration_category": "Standard",
  "probabilities": {
    "Long-running": 0.18,
    "Short": 0.27,
    "Standard": 0.55
  }
}

Reproducing Training

The training script expects the full processed dataset at:

data/processed/final_cleaned.csv

Then run:

python -m pip install pandas scikit-learn joblib matplotlib
python training/model_training.py

Training writes:

models/duration_logistic_regression_classifier.joblib
model_tests/classification_report.txt
model_tests/confusion_matrix.png

The included final_cleaned_sample.csv is intended for inspection and API/schema understanding, not for full model reproduction.

Limitations

• The model learns from historical Apache Jira patterns, so predictions may not transfer cleanly to private teams, different workflows, or non-software project management data.
• Duration is affected by team capacity, priority changes, dependencies, release schedules, and human process factors that are not fully represented in the input features.
• The labels are coarse buckets. The model does not estimate exact completion time.
• Some boundary cases are inherently ambiguous, especially tasks near the 3-day and 15-day thresholds.
• The included sample CSVs are for transparency and demonstration; full training requires rebuilding or downloading the larger processed dataset.