abrar008/gopher_submission

Main Script

• run_full_pipeline.py - Main pipeline orchestrator

Input Data

• phase_1_test.csv - Phase 1 test data (C[1-8] format questions)
• phase_2_test.csv - Phase 2 test data (markdown/mixed format questions)
• train.csv - Training data for Phase 1 XGBoost model

Core Modules

• router.py - Question router (Phase 1/Phase 2/GK detection)
• seperate_values.py - Network data parser
• rule_based_classifier.py - Feature extraction (10 engineered features)
• xgboost_tool.py - XGBoost wrapper class

Pipeline Scripts

• generate_phase1_hybrid.py - Phase 1: XGBoost classification
• generate_phase1_narrated.py - Phase 1: SLM narrative generation
• generate_phase2_hybrid.py - Phase 2: XGBoost classification + mapping
• generate_phase2_narrated.py - Phase 2: SLM narrative/solver

Pre-trained Model

• xgboost_hybrid_model.pkl - Phase 2 XGBoost model (with vectorizer)

Execution

The pipeline supports three execution modes:

Mode 1: Single Mixed File (Recommended for arbitrary test data)

python run_full_pipeline.py test_data.csv

Mode 2: Two Separate Files

python run_full_pipeline.py phase1_questions.csv phase2_questions.csv

Mode 3: Default (Uses hardcoded filenames)

python run_full_pipeline.py
# Uses phase_1_test.csv and phase_2_test.csv

Note: The pipeline automatically routes questions to the appropriate processing path regardless of input format. The router detects:

• Phase 1 format: Questions with explicit C[1-8]: notation
• Phase 2 Network: Questions with network-related keywords but different formats
• Phase 2 GK: General knowledge questions without network context

Input File Format

Input CSV must have at minimum:

• ID column: Unique question identifier
• question column: Question text

Optional columns are ignored.