Repository: danieleschmidt/customer-churn-predictor-mlops
Pull request #1: feat: Implement initial MLOps pipeline for customer churn prediction
Author: login: danieleschmidt
Created: 2025-05-21T20:56:10Z | Merged: 2025-05-21T20:56:24Z

This commit introduces a comprehensive initial MLOps pipeline covering data preprocessing, model training, MLflow integration, prediction, and CI/CD setup using GitHub Actions.

Key features implemented:

1.  **Data Preprocessing (`src/preprocess_data.py`, `scripts/run_preprocessing.py`):**
    *   Handles 'TotalCharges' conversion to numeric and NaN filling.
    *   Performs one-hot encoding for categorical features.
    *   Encodes the 'Churn' target variable.
    *   A script is provided to run preprocessing and save outputs.
    *   Includes a sample `data/raw/customer_data.csv`.

2.  **Model Training (`src/train_model.py`, `scripts/run_training.py`):**
    *   Trains a Logistic Regression model.
    *   Integrates with MLflow for logging parameters, metrics, and the model.
    *   Saves the trained model to `models/churn_model.joblib`.
    *   A script is provided to run the training pipeline.

3.  **Prediction (`src/predict_churn.py`):**
    *   Loads the trained model.
    *   Provides a function to make predictions on new (preprocessed) data.
    *   Includes example usage that attempts to load feature names from processed data.

4.  **Unit Testing (`tests/test_preprocess_data.py`):**
    *   Adds unit tests for the data preprocessing function, covering 'TotalCharges' handling, categorical encoding, and target encoding.

5.  **GitHub Actions CI/CD (`.github/workflows/main.yml`):**
    *   **Test Workflow:** Triggers on push and pull_request to `main`. Sets up Python, installs dependencies, and runs unit tests.
    *   **Train on PR Workflow:** Triggers on pull_request to `main` (after tests pass). Checks out code, sets up Python, installs dependencies, and runs the data preprocessing and model training scripts to simulate CI validation of the training pipeline.

6.  **Dependencies (`requirements.txt`):**
    *   Lists all necessary packages (pandas, scikit-learn, numpy, mlflow, joblib).

This setup provides a foundational structure for further development and operationalization of the churn prediction model.

## Summary by Sourcery

Implement an initial end-to-end MLOps pipeline for customer churn prediction, covering data preprocessing, model training and MLflow integration, prediction utilities, and CI/CD setup using GitHub Actions.

New Features:
- Implement data preprocessing module with TotalCharges numeric conversion, NaN filling, and one-hot encoding
- Add model training module with logistic regression, MLflow logging, and model serialization
- Provide prediction script to load the trained model and generate churn predictions with probabilities
- Include CLI scripts to orchestrate preprocessing and training pipelines

CI:
- Set up GitHub Actions to run unit tests on push and pull requests
- Add PR-specific job to run preprocessing and training scripts for CI validation

Tests:
- Add unit tests for preprocessing to verify TotalCharges handling, categorical encoding, target encoding, and output dimensions

Chores:
- Add requirements.txt with core dependencies


Diff:
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
new file mode 100644
index 0000000..7ecb2eb
--- /dev/null
+++ b/.github/workflows/main.yml
@@ -0,0 +1,61 @@
+name: CI Tests and PR Training
+
+on:
+  push:
+    branches:
+      - main # Triggers on push to main branch
+  pull_request:
+    branches:
+      - main # Triggers on pull request to main branch
+
+jobs:
+  test: # Existing test job
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: '3.8'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+
+      - name: Run unit tests
+        run: |
+          python -m unittest discover tests
+
+  train_on_pr: # New job for PRs
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request' # Ensure this job only runs for pull_request events
+    needs: test # Optionally, make it depend on the test job succeeding
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: '3.8'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+          # Install mlflow here if not already effectively installed for the scripts
+          # The requirements.txt should cover it.
+
+      - name: Run Preprocessing
+        run: |
+          python scripts/run_preprocessing.py
+
+      - name: Run Training
+        run: |
+          # Ensure MLFLOW_TRACKING_URI is not set or is set to a local file path
+          # to prevent errors if it tries to connect to a remote server in CI
+          # For this prompt, default local logging of MLflow is fine.
+          python scripts/run_training.py
diff --git a/data/raw/customer_data.csv b/data/raw/customer_data.csv
new file mode 100644
index 0000000..e2512e7
--- /dev/null
+++ b/data/raw/customer_data.csv
@@ -0,0 +1,11 @@
+customerID,gender,SeniorCitizen,Partner,Dependents,tenure,PhoneService,MultipleLines,InternetService,OnlineSecurity,OnlineBackup,DeviceProtection,TechSupport,StreamingTV,StreamingMovies,Contract,PaperlessBilling,PaymentMethod,MonthlyCharges,TotalCharges,Churn
+7590-VHVEG,Female,0,Yes,No,1,No,No phone service,DSL,No,Yes,No,No,No,No,Month-to-month,Yes,Electronic check,29.85,29.85,No
+5575-GNVDE,Male,0,No,No,34,Yes,No,DSL,Yes,No,Yes,No,No,No,One year,No,Mailed check,56.95,1889.5,No
+3668-QPYBK,Male,0,No,No,2,Yes,No,DSL,Yes,Yes,No,No,No,No,Month-to-month,Yes,Mailed check,53.85,108.15,Yes
+7795-CFOCW,Male,0,No,No,45,No,No phone service,DSL,Yes,No,Yes,Yes,No,No,One year,No,Bank transfer (automatic),42.30,1840.75,No
+9237-HQITU,Female,0,No,No,2,Yes,No,Fiber optic,No,No,No,No,No,No,Month-to-month,Yes,Electronic check,70.70,151.65,Yes
+9305-CDSKC,Female,0,No,No,8,Yes,Yes,Fiber optic,No,No,Yes,No,Yes,Yes,Month-to-month,Yes,Electronic check,99.65,820.5,Yes
+1452-KIOVK,Male,0,No,Yes,22,Yes,Yes,Fiber optic,No,Yes,No,No,Yes,No,Month-to-month,Yes,Credit card (automatic),89.10,1949.4,No
+6713-OKOMC,Female,0,No,No,10,No,No phone service,DSL,Yes,No,No,No,No,No,Month-to-month,No,Mailed check,29.75,301.9,No
+7892-POOKP,Female,0,Yes,No,28,Yes,Yes,Fiber optic,No,No,Yes,Yes,Yes,Yes,Month-to-month,Yes,Electronic check,104.80,3046.05,Yes
+6388-TABGU,Male,0,No,Yes,62,Yes,No,DSL,Yes,Yes,No,No,No,No,One year,No,Bank transfer (automatic),56.15,3487.95,No
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..3d17da4
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,5 @@
+pandas
+scikit-learn
+numpy
+mlflow
+joblib
diff --git a/scripts/run_preprocessing.py b/scripts/run_preprocessing.py
new file mode 100644
index 0000000..e314141
--- /dev/null
+++ b/scripts/run_preprocessing.py
@@ -0,0 +1,29 @@
+import os
+import pandas as pd
+from src.preprocess_data import preprocess # Ensure this import works based on your project structure
+
+def main():
+    # Define paths
+    raw_data_path = 'data/raw/customer_data.csv'
+    processed_features_path = 'data/processed/processed_features.csv'
+    processed_target_path = 'data/processed/processed_target.csv'
+
+    # Create directories if they don't exist
+    os.makedirs(os.path.dirname(processed_features_path), exist_ok=True)
+    os.makedirs(os.path.dirname(processed_target_path), exist_ok=True)
+
+    # Preprocess data
+    print(f"Loading raw data from {raw_data_path}...")
+    X_processed, y_processed = preprocess(raw_data_path)
+
+    # Save processed data
+    print(f"Saving processed features to {processed_features_path}...")
+    X_processed.to_csv(processed_features_path, index=False)
+    
+    print(f"Saving processed target to {processed_target_path}...")
+    pd.DataFrame(y_processed, columns=['Churn']).to_csv(processed_target_path, index=False)
+    
+    print("Preprocessing complete.")
+
+if __name__ == '__main__':
+    main()
diff --git a/scripts/run_training.py b/scripts/run_training.py
new file mode 100644
index 0000000..6542e1c
--- /dev/null
+++ b/scripts/run_training.py
@@ -0,0 +1,26 @@
+import sys
+import os
+
+# Add src directory to Python path to allow direct import of train_model
+# This is useful if running the script directly from the 'scripts' directory or root
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+
+from src.train_model import train_churn_model # Ensure this import works
+
+def main():
+    processed_features_path = 'data/processed/processed_features.csv'
+    processed_target_path = 'data/processed/processed_target.csv'
+
+    print("Starting model training script...")
+    if not (os.path.exists(processed_features_path) and os.path.exists(processed_target_path)):
+        print(f"Error: Processed data not found at {processed_features_path} or {processed_target_path}.")
+        print("Please ensure you have run the preprocessing script successfully.")
+        print("Expected files: processed_features.csv and processed_target.csv in data/processed/")
+        return
+
+    model_path, run_id = train_churn_model(processed_features_path, processed_target_path)
+    print(f"Training complete. Model saved to: {model_path}")
+    print(f"MLflow Run ID: {run_id}")
+
+if __name__ == '__main__':
+    main()
diff --git a/src/predict_churn.py b/src/predict_churn.py
new file mode 100644
index 0000000..c94b497
--- /dev/null
+++ b/src/predict_churn.py
@@ -0,0 +1,171 @@
+import joblib
+import pandas as pd
+import numpy as np
+import os
+
+# Assume the model and preprocessor/column info might be needed.
+# For this prompt, we only load the model.
+# However, a robust prediction script would need to ensure the input_data_dict
+# is transformed into the exact same format as the data used for training.
+# This includes one-hot encoding, scaling, column order, etc.
+# For simplicity, this version assumes input_data_dict is *already*
+# in the post-processed, numerical format expected by the model.
+# A future improvement would be to save and load the preprocessor pipeline.
+
+MODEL_PATH = 'models/churn_model.joblib'
+
+# It's good practice to also save the columns used during training
+# to ensure the input for prediction matches.
+# This could be saved as a list in a .json file or similar.
+# For now, we'll assume this step was done during training or is handled
+# by the caller providing data in the correct one-hot encoded format.
+
+def make_prediction(input_data_dict):
+    """
+    Loads the trained model and makes a churn prediction.
+
+    Args:
+        input_data_dict (dict): A dictionary representing a single customer's
+                                features, in the pre-processed format expected
+                                by the model (i.e., after one-hot encoding).
+                                Example: {'gender_Female': 0, 'gender_Male': 1, ... , 'TotalCharges': 500}
+
+    Returns:
+        tuple: (prediction (0 or 1), probability (float))
+               Returns (None, None) if model not found or error occurs.
+    """
+    if not os.path.exists(MODEL_PATH):
+        print(f"Error: Model not found at {MODEL_PATH}")
+        return None, None
+
+    try:
+        print(f"Loading model from {MODEL_PATH}...")
+        model = joblib.load(MODEL_PATH)
+    except Exception as e:
+        print(f"Error loading model: {e}")
+        return None, None
+
+    try:
+        # Convert the input dictionary to a DataFrame.
+        # The model expects a 2D array-like structure (e.g., a DataFrame or NumPy array).
+        # The order of columns in this DataFrame MUST match the order of features
+        # the model was trained on.
+        # For this example, we'll rely on the dictionary being ordered correctly,
+        # or more robustly, we'd use a list of training columns.
+        
+        # Create a single-row DataFrame.
+        # Important: The column order must match the training data.
+        # A more robust solution would involve saving and loading the column order
+        # from the training phase. For this example, we'll assume the dict provides this.
+        # If input_data_dict keys don't match model's expected features, it will fail.
+        
+        # To make this more robust, we should ideally have the list of feature names
+        # that the model was trained on. Let's assume for now that the input_data_dict
+        # will have all necessary features. A production system would need to load
+        # the `ColumnTransformer` used in preprocessing or at least the list of
+        # `X_processed_df.columns` from `preprocess_data.py`.
+
+        # For now, let's create a DataFrame directly from the dict.
+        # This assumes input_data_dict has the correct features in a flat structure.
+        input_df = pd.DataFrame([input_data_dict])
+        
+        # A more robust way, if you saved `X_processed_df.columns` during/after preprocessing:
+        # Load `X_train_columns.json` (example name)
+        # X_train_cols = load_json('X_train_columns.json') 
+        # input_df = pd.DataFrame(columns=X_train_cols)
+        # input_df = input_df.append(input_data_dict, ignore_index=True).fillna(0) # Fill missing one-hot encoded features with 0
+
+        print(f"Making prediction for input: {input_data_dict}")
+        prediction = model.predict(input_df)
+        probability = model.predict_proba(input_df)
+
+        # probability is an array of shape (n_samples, n_classes).
+        # For binary classification, this is typically [[prob_0, prob_1]].
+        # We want the probability of the positive class (churn=1).
+        churn_probability = probability[0, 1] 
+
+        print(f"Raw prediction: {prediction}")
+        print(f"Raw probability: {probability}")
+        print(f"Predicted class: {prediction[0]}, Churn probability: {churn_probability:.4f}")
+        
+        return int(prediction[0]), float(churn_probability)
+
+    except Exception as e:
+        print(f"Error during prediction: {e}")
+        # This could be due to mismatched feature names/order, or unexpected data types.
+        # Example: if input_data_dict doesn't have all columns model expects based on training.
+        return None, None
+
+if __name__ == '__main__':
+    # Example usage:
+    # This example assumes you know the *exact* feature names after one-hot encoding.
+    # In a real scenario, these feature names would come from your preprocessing step.
+    # You would need to run preprocessing, get the column names from X_processed_df,
+    # and then construct a similar dictionary.
+    
+    print("Starting prediction script directly (for testing purposes)...")
+    if not os.path.exists(MODEL_PATH):
+        print(f"Model {MODEL_PATH} not found. Please train the model first using scripts/run_training.py.")
+    else:
+        # Create a sample input dictionary.
+        # IMPORTANT: This is a placeholder. The actual features and their names
+        # depend on the one-hot encoding of your *specific* training data.
+        # You would need to inspect the columns of 'processed_features.csv'
+        # from the preprocessing step to create a valid example.
+        
+        # Let's try to load the columns from the processed_features.csv to make this example more realistic
+        processed_features_path = 'data/processed/processed_features.csv'
+        if os.path.exists(processed_features_path):
+            try:
+                X_processed_df = pd.read_csv(processed_features_path)
+                sample_feature_names = X_processed_df.columns.tolist()
+                
+                # Create a dummy input dict using the first row of processed data
+                # or by creating a dict with all zeros.
+                if not X_processed_df.empty:
+                    example_input_dict = X_processed_df.iloc[0].to_dict()
+                    print(f"Using features from the first row of {processed_features_path} for example prediction.")
+                else:
+                    print(f"{processed_features_path} is empty. Creating a zeroed-out example dict.")
+                    example_input_dict = {col: 0 for col in sample_feature_names}
+                
+                # Modify a few values for demonstration if needed
+                # Example: if 'TotalCharges' is a feature and you want to set a specific value
+                if 'TotalCharges' in example_input_dict: # This name might be different if it was scaled/transformed
+                    example_input_dict['TotalCharges'] = 2000 
+                # Example: if 'tenure' is a feature
+                if 'tenure' in example_input_dict:
+                     example_input_dict['tenure'] = 24
+
+
+                print(f"Example input data (first 5 items): {dict(list(example_input_dict.items())[:5])}...")
+                prediction, probability = make_prediction(example_input_dict)
+
+                if prediction is not None:
+                    print(f"\nExample Prediction successful:")
+                    print(f"Predicted Churn: {'Yes' if prediction == 1 else 'No'} (Class: {prediction})")
+                    print(f"Probability of Churn: {probability:.4f}")
+                else:
+                    print("\nExample Prediction failed. Check logs for errors.")
+            except Exception as e:
+                print(f"Could not create example input for testing predict_churn.py: {e}")
+                print("This might be because processed_features.csv is not available or is not in the expected format.")
+                print("Please run scripts/run_preprocessing.py first.")
+
+        else:
+            print(f"Cannot run example: Processed features file not found at {processed_features_path}.")
+            print("Please run scripts/run_preprocessing.py to generate it.")
+            print("Falling back to a very basic example dict (likely to fail if features don't match model):")
+            # This basic example is highly unlikely to match the actual model features
+            # after one-hot encoding in a real scenario.
+            basic_example_input = {
+                'gender_Female': 1, 'gender_Male': 0, 'SeniorCitizen': 0, 'Partner_Yes': 1, 'Partner_No': 0,
+                'Dependents_No': 1, 'Dependents_Yes': 0, 'tenure': 10, 'PhoneService_Yes': 1, 'PhoneService_No': 0,
+                # ... many more one-hot encoded features would be here ...
+                'MonthlyCharges': 70.0, 'TotalCharges': 700.0
+            }
+            # It's better to inform the user that this basic example is insufficient.
+            print("The basic_example_input is a placeholder and likely does not match your model's required features.")
+            print("For a meaningful test, ensure processed_features.csv exists or provide a dict with correct one-hot encoded column names.")
+
+```
diff --git a/src/preprocess_data.py b/src/preprocess_data.py
new file mode 100644
index 0000000..9dfcc73
--- /dev/null
+++ b/src/preprocess_data.py
@@ -0,0 +1,58 @@
+import pandas as pd
+from sklearn.preprocessing import LabelEncoder, OneHotEncoder
+from sklearn.compose import ColumnTransformer
+import numpy as np
+
+def preprocess(df_path):
+    """
+    Preprocesses the raw customer data.
+
+    Args:
+        df_path (str): Path to the raw CSV data file.
+
+    Returns:
+        tuple: A tuple containing processed features (X) and target (y).
+    """
+    df = pd.read_csv(df_path)
+
+    # Convert 'TotalCharges' to numeric, coercing errors, and fill NaNs
+    df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
+    df['TotalCharges'] = df['TotalCharges'].fillna(0)
+
+    # Encode target variable 'Churn'
+    label_encoder_churn = LabelEncoder()
+    df['Churn'] = label_encoder_churn.fit_transform(df['Churn'])
+    y = df['Churn']
+
+    # Separate features (X)
+    X = df.drop('Churn', axis=1)
+    X = X.drop('customerID', axis=1) # Drop customerID as it's an identifier
+
+    # Identify categorical and numerical features
+    categorical_features = X.select_dtypes(include=['object']).columns
+    numerical_features = X.select_dtypes(include=np.number).columns
+
+    # Create preprocessing pipelines for categorical and numerical features
+    # One-hot encode categorical features
+    # For numerical features, we'll just pass them through for now, but scaling could be added here.
+    preprocessor = ColumnTransformer(
+        transformers=[
+            ('cat', OneHotEncoder(handle_unknown='ignore', sparse_output=False), categorical_features)
+        ],
+        remainder='passthrough' # Keep numerical columns not specified in transformers
+    )
+
+    X_processed = preprocessor.fit_transform(X)
+    
+    # Get feature names after one-hot encoding
+    # Get feature names from OneHotEncoder
+    ohe_feature_names = preprocessor.named_transformers_['cat'].get_feature_names_out(categorical_features)
+    
+    # Concatenate OHE feature names with remaining numerical feature names
+    # Ensure numerical features are correctly identified and ordered as in X after categorical processing
+    # The remainder='passthrough' puts numerical columns at the end, in their original relative order.
+    processed_feature_names = list(ohe_feature_names) + list(numerical_features)
+
+    X_processed_df = pd.DataFrame(X_processed, columns=processed_feature_names, index=X.index)
+
+    return X_processed_df, y
diff --git a/src/train_model.py b/src/train_model.py
new file mode 100644
index 0000000..36804b9
--- /dev/null
+++ b/src/train_model.py
@@ -0,0 +1,83 @@
+import pandas as pd
+from sklearn.model_selection import train_test_split
+from sklearn.linear_model import LogisticRegression
+from sklearn.metrics import accuracy_score, f1_score
+import mlflow
+import mlflow.sklearn
+import joblib
+import os
+
+def train_churn_model(X_path, y_path):
+    """
+    Trains a churn prediction model, logs it with MLflow, and saves it.
+
+    Args:
+        X_path (str): Path to the processed features CSV file.
+        y_path (str): Path to the processed target CSV file.
+    """
+    print(f"Loading data from {X_path} and {y_path}...")
+    X = pd.read_csv(X_path)
+    y = pd.read_csv(y_path).squeeze() # Use squeeze() to convert DataFrame column to Series
+
+    # Split data
+    print("Splitting data into training and testing sets...")
+    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
+
+    # Initialize MLflow run
+    with mlflow.start_run() as run:
+        run_id = run.info.run_id
+        print(f"MLflow Run ID: {run_id}")
+
+        # Initialize and train Logistic Regression model
+        print("Training Logistic Regression model...")
+        model = LogisticRegression(solver='liblinear', C=1.0, random_state=42) # Example parameters
+        model.fit(X_train, y_train)
+
+        # Make predictions
+        y_pred_test = model.predict(X_test)
+
+        # Calculate metrics
+        accuracy = accuracy_score(y_test, y_pred_test)
+        f1 = f1_score(y_test, y_pred_test)
+
+        print(f"Test Set Accuracy: {accuracy:.4f}")
+        print(f"Test Set F1-score: {f1:.4f}")
+
+        # Log parameters
+        print("Logging model parameters to MLflow...")
+        mlflow.log_param("solver", model.get_params()['solver'])
+        mlflow.log_param("C", model.get_params()['C'])
+        mlflow.log_param("random_state", model.get_params()['random_state'])
+
+        # Log metrics
+        print("Logging model metrics to MLflow...")
+        mlflow.log_metric("accuracy", accuracy)
+        mlflow.log_metric("f1_score", f1)
+
+        # Log model
+        print("Logging model to MLflow...")
+        mlflow.sklearn.log_model(model, "churn_model")
+
+        # Save the trained model
+        model_dir = 'models'
+        os.makedirs(model_dir, exist_ok=True)
+        model_path = os.path.join(model_dir, 'churn_model.joblib')
+        print(f"Saving trained model to {model_path}...")
+        joblib.dump(model, model_path)
+        
+        print("Model training and MLflow logging complete.")
+        return model_path, run_id
+
+if __name__ == '__main__':
+    # This part is for direct script execution testing, if needed.
+    # Typically, this would be called by run_training.py
+    print("Starting model training directly (for testing purposes)...")
+    # Define default paths for X and y, assuming they are in data/processed
+    default_X_path = 'data/processed/processed_features.csv'
+    default_y_path = 'data/processed/processed_target.csv'
+
+    if not (os.path.exists(default_X_path) and os.path.exists(default_y_path)):
+        print(f"Error: Processed data not found at {default_X_path} or {default_y_path}.")
+        print("Please run the preprocessing script first (e.g., scripts/run_preprocessing.py).")
+    else:
+        train_churn_model(default_X_path, default_y_path)
diff --git a/tests/__init__.py b/tests/__init__.py
new file mode 100644
index 0000000..5536bf7
--- /dev/null
+++ b/tests/__init__.py
@@ -0,0 +1,2 @@
+# This file makes the 'tests' directory a Python package.
+# It can be empty.
diff --git a/tests/test_preprocess_data.py b/tests/test_preprocess_data.py
new file mode 100644
index 0000000..c2b93b8
--- /dev/null
+++ b/tests/test_preprocess_data.py
@@ -0,0 +1,101 @@
+import unittest
+import pandas as pd
+from pandas.testing import assert_frame_equal, assert_series_equal
+import numpy as np
+import sys
+import os
+
+# Add src directory to Python path to import preprocess_data
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+from src.preprocess_data import preprocess # Adjust import if necessary
+
+class TestPreprocessData(unittest.TestCase):
+
+    def setUp(self):
+        # Create a small sample DataFrame for testing
+        self.sample_data = {
+            'customerID': ['123-ABC', '456-DEF', '789-GHI', '012-JKL'],
+            'gender': ['Female', 'Male', 'Female', 'Male'],
+            'SeniorCitizen': [0, 1, 0, 0],
+            'Partner': ['Yes', 'No', 'No', 'Yes'],
+            'Dependents': ['No', 'No', 'Yes', 'No'],
+            'tenure': [1, 10, 24, 2],
+            'PhoneService': ['No', 'Yes', 'Yes', 'Yes'],
+            'MultipleLines': ['No phone service', 'No', 'Yes', 'No'],
+            'InternetService': ['DSL', 'Fiber optic', 'DSL', 'No'],
+            'OnlineSecurity': ['No', 'Yes', 'No', 'No internet service'],
+            'OnlineBackup': ['Yes', 'No', 'No', 'No internet service'],
+            'DeviceProtection': ['No', 'Yes', 'No', 'No internet service'],
+            'TechSupport': ['No', 'No', 'Yes', 'No internet service'],
+            'StreamingTV': ['No', 'Yes', 'No', 'No internet service'],
+            'StreamingMovies': ['No', 'Yes', 'No', 'No internet service'],
+            'Contract': ['Month-to-month', 'One year', 'Two year', 'Month-to-month'],
+            'PaperlessBilling': ['Yes', 'No', 'No', 'Yes'],
+            'PaymentMethod': ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Electronic check'],
+            'MonthlyCharges': [29.85, 90.00, 50.00, 20.00],
+            'TotalCharges': ['29.85', '900', '1200.00', ' '], # Includes a space to test coercion
+            'Churn': ['No', 'Yes', 'No', 'Yes']
+        }
+        self.df = pd.DataFrame(self.sample_data)
+        
+        # Create a dummy CSV file for the preprocess function to read
+        self.test_csv_path = 'test_sample_data.csv'
+        self.df.to_csv(self.test_csv_path, index=False)
+
+    def tearDown(self):
+        # Clean up the dummy CSV file
+        if os.path.exists(self.test_csv_path):
+            os.remove(self.test_csv_path)
+
+    def test_total_charges_conversion_and_nan_fill(self):
+        # Call preprocess once to get X_processed
+        X_processed, _ = preprocess(self.test_csv_path) 
+        
+        # Ensure NaN handling was effective for TotalCharges.
+        self.assertNotIn('TotalCharges_nan', X_processed.columns) 
+        
+        # Infer the name of the TotalCharges column after transformation
+        total_charges_col_name = [col for col in X_processed.columns if 'TotalCharges' in col][-1]
+        
+        self.assertTrue(pd.api.types.is_float_dtype(X_processed[total_charges_col_name]))
+        
+        # Check that the space in 'TotalCharges' for '012-JKL' (index 3) was converted to 0.0
+        # This customer is at index 3 in the original self.df and X_processed (assuming index preservation)
+        self.assertEqual(X_processed.loc[3, total_charges_col_name], 0.0)
+        
+        # General check for no NaNs in the column
+        self.assertFalse(X_processed[total_charges_col_name].isnull().any(), "TotalCharges column should not have NaNs after processing.")
+
+
+    def test_categorical_encoding(self):
+        X_processed, _ = preprocess(self.test_csv_path)
+        
+        original_categorical_cols = self.df.select_dtypes(include=['object']).drop(columns=['customerID', 'TotalCharges', 'Churn']).columns
+        for col in original_categorical_cols:
+            self.assertNotIn(col, X_processed.columns)
+
+        self.assertTrue(any(col.startswith('gender_') for col in X_processed.columns))
+        self.assertTrue(any(col.startswith('Contract_') for col in X_processed.columns))
+        self.assertTrue(all(pd.api.types.is_numeric_dtype(X_processed[col]) for col in X_processed.columns))
+
+
+    def test_target_encoding(self):
+        _, y_processed = preprocess(self.test_csv_path)
+        self.assertTrue(pd.api.types.is_integer_dtype(y_processed)) 
+        self.assertEqual(y_processed.nunique(), 2) 
+        expected_y = pd.Series([0, 1, 0, 1], name='Churn')
+        assert_series_equal(y_processed, expected_y, check_dtype=False)
+
+
+    def test_output_shape(self):
+        X_processed, y_processed = preprocess(self.test_csv_path)
+        self.assertEqual(X_processed.shape[0], self.df.shape[0]) 
+        self.assertEqual(y_processed.shape[0], self.df.shape[0]) 
+        self.assertTrue(X_processed.shape[1] > self.df.shape[1] - 2) 
+
+    def test_id_column_dropped(self):
+        X_processed, _ = preprocess(self.test_csv_path)
+        self.assertNotIn('customerID', X_processed.columns)
+
+if __name__ == '__main__':
+    unittest.main()


Reviews:
user: login: sourcery-ai[bot]
body: Hey @danieleschmidt - I've reviewed your changes - here's some feedback:

- Persist your preprocessing artifacts (e.g. the OneHotEncoder or the list of trained feature names) alongside the model so the prediction script can automatically align, fill missing features, and avoid relying on manually ordered input dicts.
- Rather than using sys.path hacks in your run scripts, consider packaging the project (via setup.py/pyproject.toml) and installing it in CI so that imports are cleaner and more robust.
- Pin explicit dependency versions in requirements.txt (e.g. pandas==1.3.5) to ensure consistent CI builds and guard against breaking changes in upstream libraries.

<details>
<summary>Here's what I looked at during the review</summary>

- 🟡 **General issues**: 4 issues found
- 🟢 **Security**: all looks good
- 🟡 **Testing**: 2 issues found
- 🟡 **Complexity**: 1 issue found
- 🟢 **Documentation**: all looks good
</details>

***

<details>
<summary>Sourcery is free for open source - if you like our reviews please consider sharing them ✨</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https://sourcery.ai)
- [Facebook](https://www.facebook.com/sharer/sharer.php?u=https://sourcery.ai)

</details>

<sub>
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
</sub>
state: APPROVED
author_association: NONE
submitted_at: 2025-05-21T21:01:43Z

Review comments:
diff_hunk: @@ -0,0 +1,171 @@
+import joblib
+import pandas as pd
+import numpy as np
+import os
+
+# Assume the model and preprocessor/column info might be needed.
+# For this prompt, we only load the model.
+# However, a robust prediction script would need to ensure the input_data_dict
+# is transformed into the exact same format as the data used for training.
+# This includes one-hot encoding, scaling, column order, etc.
+# For simplicity, this version assumes input_data_dict is *already*
+# in the post-processed, numerical format expected by the model.
+# A future improvement would be to save and load the preprocessor pipeline.
+
+MODEL_PATH = 'models/churn_model.joblib'
+
+# It's good practice to also save the columns used during training
+# to ensure the input for prediction matches.
+# This could be saved as a list in a .json file or similar.
+# For now, we'll assume this step was done during training or is handled
+# by the caller providing data in the correct one-hot encoded format.
+
+def make_prediction(input_data_dict):
+    """
+    Loads the trained model and makes a churn prediction.
+
+    Args:
+        input_data_dict (dict): A dictionary representing a single customer's
+                                features, in the pre-processed format expected
+                                by the model (i.e., after one-hot encoding).
+                                Example: {'gender_Female': 0, 'gender_Male': 1, ... , 'TotalCharges': 500}
+
+    Returns:
+        tuple: (prediction (0 or 1), probability (float))
+               Returns (None, None) if model not found or error occurs.
+    """
+    if not os.path.exists(MODEL_PATH):
+        print(f"Error: Model not found at {MODEL_PATH}")
+        return None, None
+
+    try:
+        print(f"Loading model from {MODEL_PATH}...")
path: src/predict_churn.py
user: login: sourcery-ai[bot]
body: **suggestion:** Use a logging framework instead of print statements

The `logging` module allows configurable output and better message management than `print`.
created_at: 2025-05-21T21:01:42Z
author_association: NONE
diff_hunk: @@ -0,0 +1,58 @@
+import pandas as pd
+from sklearn.preprocessing import LabelEncoder, OneHotEncoder
+from sklearn.compose import ColumnTransformer
+import numpy as np
+
+def preprocess(df_path):
+    """
+    Preprocesses the raw customer data.
+
+    Args:
+        df_path (str): Path to the raw CSV data file.
+
+    Returns:
+        tuple: A tuple containing processed features (X) and target (y).
+    """
+    df = pd.read_csv(df_path)
+
+    # Convert 'TotalCharges' to numeric, coercing errors, and fill NaNs
+    df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
+    df['TotalCharges'] = df['TotalCharges'].fillna(0)
+
+    # Encode target variable 'Churn'
+    label_encoder_churn = LabelEncoder()
+    df['Churn'] = label_encoder_churn.fit_transform(df['Churn'])
+    y = df['Churn']
+
+    # Separate features (X)
+    X = df.drop('Churn', axis=1)
+    X = X.drop('customerID', axis=1) # Drop customerID as it's an identifier
+
+    # Identify categorical and numerical features
+    categorical_features = X.select_dtypes(include=['object']).columns
+    numerical_features = X.select_dtypes(include=np.number).columns
+
+    # Create preprocessing pipelines for categorical and numerical features
+    # One-hot encode categorical features
+    # For numerical features, we'll just pass them through for now, but scaling could be added here.
+    preprocessor = ColumnTransformer(
path: src/preprocess_data.py
user: login: sourcery-ai[bot]
body: **suggestion:** Consider adding scaling for numerical features

Adding a `StandardScaler` to the pipeline for numeric columns will help maintain consistency between training and inference.
created_at: 2025-05-21T21:01:42Z
author_association: NONE
diff_hunk: @@ -0,0 +1,26 @@
+import sys
+import os
+
+# Add src directory to Python path to allow direct import of train_model
+# This is useful if running the script directly from the 'scripts' directory or root
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+
+from src.train_model import train_churn_model # Ensure this import works
path: scripts/run_training.py
user: login: sourcery-ai[bot]
body: **suggestion:** Avoid manipulating sys.path; use proper package installation

Use editable installs and a project configuration file to manage imports instead of modifying sys.path.

```suggestion
import os

from train_model import train_churn_model
```
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,101 @@
+import unittest
+import pandas as pd
+from pandas.testing import assert_frame_equal, assert_series_equal
+import numpy as np
+import sys
+import os
+
+# Add src directory to Python path to import preprocess_data
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+from src.preprocess_data import preprocess # Adjust import if necessary
+
+class TestPreprocessData(unittest.TestCase):
path: tests/test_preprocess_data.py
user: login: sourcery-ai[bot]
body: **suggestion (testing):** Consider adding tests for edge case inputs to `preprocess`.

Test cases could include: empty input CSV, CSV with only headers, DataFrame lacking categorical columns, and DataFrame lacking numerical columns (except 'TotalCharges' after conversion). This will help verify robustness for diverse inputs.
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,101 @@
+import unittest
+import pandas as pd
+from pandas.testing import assert_frame_equal, assert_series_equal
+import numpy as np
+import sys
+import os
+
+# Add src directory to Python path to import preprocess_data
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+from src.preprocess_data import preprocess # Adjust import if necessary
+
+class TestPreprocessData(unittest.TestCase):
+
+    def setUp(self):
+        # Create a small sample DataFrame for testing
+        self.sample_data = {
+            'customerID': ['123-ABC', '456-DEF', '789-GHI', '012-JKL'],
+            'gender': ['Female', 'Male', 'Female', 'Male'],
+            'SeniorCitizen': [0, 1, 0, 0],
+            'Partner': ['Yes', 'No', 'No', 'Yes'],
+            'Dependents': ['No', 'No', 'Yes', 'No'],
+            'tenure': [1, 10, 24, 2],
+            'PhoneService': ['No', 'Yes', 'Yes', 'Yes'],
+            'MultipleLines': ['No phone service', 'No', 'Yes', 'No'],
+            'InternetService': ['DSL', 'Fiber optic', 'DSL', 'No'],
+            'OnlineSecurity': ['No', 'Yes', 'No', 'No internet service'],
+            'OnlineBackup': ['Yes', 'No', 'No', 'No internet service'],
+            'DeviceProtection': ['No', 'Yes', 'No', 'No internet service'],
+            'TechSupport': ['No', 'No', 'Yes', 'No internet service'],
+            'StreamingTV': ['No', 'Yes', 'No', 'No internet service'],
+            'StreamingMovies': ['No', 'Yes', 'No', 'No internet service'],
+            'Contract': ['Month-to-month', 'One year', 'Two year', 'Month-to-month'],
+            'PaperlessBilling': ['Yes', 'No', 'No', 'Yes'],
+            'PaymentMethod': ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Electronic check'],
+            'MonthlyCharges': [29.85, 90.00, 50.00, 20.00],
+            'TotalCharges': ['29.85', '900', '1200.00', ' '], # Includes a space to test coercion
+            'Churn': ['No', 'Yes', 'No', 'Yes']
+        }
+        self.df = pd.DataFrame(self.sample_data)
+        
+        # Create a dummy CSV file for the preprocess function to read
+        self.test_csv_path = 'test_sample_data.csv'
+        self.df.to_csv(self.test_csv_path, index=False)
+
+    def tearDown(self):
+        # Clean up the dummy CSV file
+        if os.path.exists(self.test_csv_path):
+            os.remove(self.test_csv_path)
+
+    def test_total_charges_conversion_and_nan_fill(self):
+        # Call preprocess once to get X_processed
+        X_processed, _ = preprocess(self.test_csv_path) 
+        
+        # Ensure NaN handling was effective for TotalCharges.
+        self.assertNotIn('TotalCharges_nan', X_processed.columns) 
+        
+        # Infer the name of the TotalCharges column after transformation
+        total_charges_col_name = [col for col in X_processed.columns if 'TotalCharges' in col][-1]
+        
+        self.assertTrue(pd.api.types.is_float_dtype(X_processed[total_charges_col_name]))
+        
+        # Check that the space in 'TotalCharges' for '012-JKL' (index 3) was converted to 0.0
+        # This customer is at index 3 in the original self.df and X_processed (assuming index preservation)
+        self.assertEqual(X_processed.loc[3, total_charges_col_name], 0.0)
+        
+        # General check for no NaNs in the column
+        self.assertFalse(X_processed[total_charges_col_name].isnull().any(), "TotalCharges column should not have NaNs after processing.")
+
+
+    def test_categorical_encoding(self):
+        X_processed, _ = preprocess(self.test_csv_path)
+        
+        original_categorical_cols = self.df.select_dtypes(include=['object']).drop(columns=['customerID', 'TotalCharges', 'Churn']).columns
+        for col in original_categorical_cols:
+            self.assertNotIn(col, X_processed.columns)
+
+        self.assertTrue(any(col.startswith('gender_') for col in X_processed.columns))
+        self.assertTrue(any(col.startswith('Contract_') for col in X_processed.columns))
+        self.assertTrue(all(pd.api.types.is_numeric_dtype(X_processed[col]) for col in X_processed.columns))
path: tests/test_preprocess_data.py
user: login: sourcery-ai[bot]
body: **suggestion (testing):** Enhance categorical encoding assertions with specific column counts.

Assert the exact number of one-hot encoded columns for key categorical features, such as 2 for 'gender' and 3 for 'MultipleLines', based on their unique values in the sample data.

```suggestion
        # Assert exact number of one‐hot encoded columns for key categoricals
        # Expect 2 for 'gender' and 3 for 'MultipleLines' based on sample-data unique values
        self.assertEqual(sum(col.startswith('gender_') for col in X_processed.columns), 2)
        self.assertEqual(sum(col.startswith('MultipleLines_') for col in X_processed.columns), 3)
        # Ensure all processed columns are numeric
        self.assertTrue(all(pd.api.types.is_numeric_dtype(X_processed[col]) for col in X_processed.columns))
```
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,101 @@
+import unittest
+import pandas as pd
+from pandas.testing import assert_frame_equal, assert_series_equal
+import numpy as np
+import sys
+import os
+
+# Add src directory to Python path to import preprocess_data
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+from src.preprocess_data import preprocess # Adjust import if necessary
path: tests/test_preprocess_data.py
user: login: sourcery-ai[bot]
body: **suggestion:** Consider refactoring `preprocess` to accept a DataFrame for enhanced testability.

Accepting a DataFrame would allow tests to use in-memory data, reducing reliance on file I/O and making tests simpler and faster. File reading can be handled outside the function, such as in the main script.
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,171 @@
+import joblib
+import pandas as pd
+import numpy as np
+import os
+
+# Assume the model and preprocessor/column info might be needed.
+# For this prompt, we only load the model.
+# However, a robust prediction script would need to ensure the input_data_dict
+# is transformed into the exact same format as the data used for training.
+# This includes one-hot encoding, scaling, column order, etc.
+# For simplicity, this version assumes input_data_dict is *already*
+# in the post-processed, numerical format expected by the model.
+# A future improvement would be to save and load the preprocessor pipeline.
+
+MODEL_PATH = 'models/churn_model.joblib'
+
+# It's good practice to also save the columns used during training
+# to ensure the input for prediction matches.
+# This could be saved as a list in a .json file or similar.
+# For now, we'll assume this step was done during training or is handled
+# by the caller providing data in the correct one-hot encoded format.
+
+def make_prediction(input_data_dict):
path: src/predict_churn.py
user: login: sourcery-ai[bot]
body: **issue (complexity):** Consider refactoring make_prediction to be a pure function, cache the model at module scope, and move demo/example code to a separate script.

```markdown
You can dramatically simplify `make_prediction` by

 1. removing I/O/print logic from the core function  
 2. caching the model at module‐scope so you only load it once  
 3. pushing your “example/demo” code into a separate script

Below is one way to do that with minimal functional changes:

```python
# predict_churn.py
import os
import joblib
import pandas as pd

MODEL_PATH = "models/churn_model.joblib"
_model = None  # module‐level cache


def _get_model():
    global _model
    if _model is None:
        if not os.path.exists(MODEL_PATH):
            raise FileNotFoundError(f"Model not found at {MODEL_PATH}")
        _model = joblib.load(MODEL_PATH)
    return _model


def make_prediction(input_data: dict) -> tuple[int, float]:
    """
    Pure function: convert input_data → (prediction, churn_probability).
    Raises on errors; no prints.
    """
    model = _get_model()
    df = pd.DataFrame([input_data])
    pred = model.predict(df)[0]
    prob = model.predict_proba(df)[0, 1]
    return int(pred), float(prob)
```

Then create a small separate demo script:

```python
# demo_predict.py
from predict_churn import make_prediction
import pandas as pd

if __name__ == "__main__":
    # load feature names from preprocessing, or build your example dict
    X = pd.read_csv("data/processed/processed_features.csv")
    example = X.iloc[0].to_dict() if not X.empty else {col: 0 for col in X.columns}
    example["TotalCharges"] = 2000
    example["tenure"] = 24

    try:
        pred, prob = make_prediction(example)
        print(f"Churn={pred}, P(churn)={prob:.2%}")
    except Exception as e:
        print("Failed:", e)
```

Benefits:

- **make_prediction** is now a pure, side‐effect‐free function (easier to test and reason about).  
- Model loads exactly once.  
- All ad-hoc example/demo code lives in its own script.  
```
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,101 @@
+import unittest
+import pandas as pd
+from pandas.testing import assert_frame_equal, assert_series_equal
+import numpy as np
+import sys
+import os
+
+# Add src directory to Python path to import preprocess_data
+sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
+from src.preprocess_data import preprocess # Adjust import if necessary
+
+class TestPreprocessData(unittest.TestCase):
+
+    def setUp(self):
+        # Create a small sample DataFrame for testing
+        self.sample_data = {
+            'customerID': ['123-ABC', '456-DEF', '789-GHI', '012-JKL'],
+            'gender': ['Female', 'Male', 'Female', 'Male'],
+            'SeniorCitizen': [0, 1, 0, 0],
+            'Partner': ['Yes', 'No', 'No', 'Yes'],
+            'Dependents': ['No', 'No', 'Yes', 'No'],
+            'tenure': [1, 10, 24, 2],
+            'PhoneService': ['No', 'Yes', 'Yes', 'Yes'],
+            'MultipleLines': ['No phone service', 'No', 'Yes', 'No'],
+            'InternetService': ['DSL', 'Fiber optic', 'DSL', 'No'],
+            'OnlineSecurity': ['No', 'Yes', 'No', 'No internet service'],
+            'OnlineBackup': ['Yes', 'No', 'No', 'No internet service'],
+            'DeviceProtection': ['No', 'Yes', 'No', 'No internet service'],
+            'TechSupport': ['No', 'No', 'Yes', 'No internet service'],
+            'StreamingTV': ['No', 'Yes', 'No', 'No internet service'],
+            'StreamingMovies': ['No', 'Yes', 'No', 'No internet service'],
+            'Contract': ['Month-to-month', 'One year', 'Two year', 'Month-to-month'],
+            'PaperlessBilling': ['Yes', 'No', 'No', 'Yes'],
+            'PaymentMethod': ['Electronic check', 'Mailed check', 'Bank transfer (automatic)', 'Electronic check'],
+            'MonthlyCharges': [29.85, 90.00, 50.00, 20.00],
+            'TotalCharges': ['29.85', '900', '1200.00', ' '], # Includes a space to test coercion
+            'Churn': ['No', 'Yes', 'No', 'Yes']
+        }
+        self.df = pd.DataFrame(self.sample_data)
+        
+        # Create a dummy CSV file for the preprocess function to read
+        self.test_csv_path = 'test_sample_data.csv'
+        self.df.to_csv(self.test_csv_path, index=False)
+
+    def tearDown(self):
+        # Clean up the dummy CSV file
+        if os.path.exists(self.test_csv_path):
+            os.remove(self.test_csv_path)
+
+    def test_total_charges_conversion_and_nan_fill(self):
+        # Call preprocess once to get X_processed
+        X_processed, _ = preprocess(self.test_csv_path) 
+        
+        # Ensure NaN handling was effective for TotalCharges.
+        self.assertNotIn('TotalCharges_nan', X_processed.columns) 
+        
+        # Infer the name of the TotalCharges column after transformation
+        total_charges_col_name = [col for col in X_processed.columns if 'TotalCharges' in col][-1]
+        
+        self.assertTrue(pd.api.types.is_float_dtype(X_processed[total_charges_col_name]))
+        
+        # Check that the space in 'TotalCharges' for '012-JKL' (index 3) was converted to 0.0
+        # This customer is at index 3 in the original self.df and X_processed (assuming index preservation)
+        self.assertEqual(X_processed.loc[3, total_charges_col_name], 0.0)
+        
+        # General check for no NaNs in the column
+        self.assertFalse(X_processed[total_charges_col_name].isnull().any(), "TotalCharges column should not have NaNs after processing.")
+
+
+    def test_categorical_encoding(self):
+        X_processed, _ = preprocess(self.test_csv_path)
+        
+        original_categorical_cols = self.df.select_dtypes(include=['object']).drop(columns=['customerID', 'TotalCharges', 'Churn']).columns
+        for col in original_categorical_cols:
+            self.assertNotIn(col, X_processed.columns)
path: tests/test_preprocess_data.py
user: login: sourcery-ai[bot]
body: **issue (code-quality):** Avoid loops in tests. ([`no-loop-in-tests`](https://docs.sourcery.ai/Reference/Rules-and-In-Line-Suggestions/Python/Default-Rules/no-loop-in-tests))

<details><summary>Explanation</summary>Avoid complex code, like loops, in test functions.

Google's software engineering guidelines says:
"Clear tests are trivially correct upon inspection"
To reach that avoid complex code in tests:
* loops
* conditionals

Some ways to fix this:

* Use parametrized tests to get rid of the loop.
* Move the complex logic into helpers.
* Move the complex part into pytest fixtures.

> Complexity is most often introduced in the form of logic. Logic is defined via the imperative parts of programming languages such as operators, loops, and conditionals. When a piece of code contains logic, you need to do a bit of mental computation to determine its result instead of just reading it off of the screen. It doesn't take much logic to make a test more difficult to reason about.

Software Engineering at Google / [Don't Put Logic in Tests](https://abseil.io/resources/swe-book/html/ch12.html#donapostrophet_put_logic_in_tests)
</details>
created_at: 2025-05-21T21:01:43Z
author_association: NONE
diff_hunk: @@ -0,0 +1,171 @@
+import joblib
+import pandas as pd
+import numpy as np
+import os
+
+# Assume the model and preprocessor/column info might be needed.
+# For this prompt, we only load the model.
+# However, a robust prediction script would need to ensure the input_data_dict
+# is transformed into the exact same format as the data used for training.
+# This includes one-hot encoding, scaling, column order, etc.
+# For simplicity, this version assumes input_data_dict is *already*
+# in the post-processed, numerical format expected by the model.
+# A future improvement would be to save and load the preprocessor pipeline.
+
+MODEL_PATH = 'models/churn_model.joblib'
+
+# It's good practice to also save the columns used during training
+# to ensure the input for prediction matches.
+# This could be saved as a list in a .json file or similar.
+# For now, we'll assume this step was done during training or is handled
+# by the caller providing data in the correct one-hot encoded format.
+
+def make_prediction(input_data_dict):
+    """
+    Loads the trained model and makes a churn prediction.
+
+    Args:
+        input_data_dict (dict): A dictionary representing a single customer's
+                                features, in the pre-processed format expected
+                                by the model (i.e., after one-hot encoding).
+                                Example: {'gender_Female': 0, 'gender_Male': 1, ... , 'TotalCharges': 500}
+
+    Returns:
+        tuple: (prediction (0 or 1), probability (float))
+               Returns (None, None) if model not found or error occurs.
+    """
+    if not os.path.exists(MODEL_PATH):
+        print(f"Error: Model not found at {MODEL_PATH}")
+        return None, None
+
+    try:
+        print(f"Loading model from {MODEL_PATH}...")
+        model = joblib.load(MODEL_PATH)
+    except Exception as e:
+        print(f"Error loading model: {e}")
+        return None, None
+
+    try:
+        # Convert the input dictionary to a DataFrame.
+        # The model expects a 2D array-like structure (e.g., a DataFrame or NumPy array).
+        # The order of columns in this DataFrame MUST match the order of features
+        # the model was trained on.
+        # For this example, we'll rely on the dictionary being ordered correctly,
+        # or more robustly, we'd use a list of training columns.
+        
+        # Create a single-row DataFrame.
+        # Important: The column order must match the training data.
+        # A more robust solution would involve saving and loading the column order
+        # from the training phase. For this example, we'll assume the dict provides this.
+        # If input_data_dict keys don't match model's expected features, it will fail.
+        
+        # To make this more robust, we should ideally have the list of feature names
+        # that the model was trained on. Let's assume for now that the input_data_dict
+        # will have all necessary features. A production system would need to load
+        # the `ColumnTransformer` used in preprocessing or at least the list of
+        # `X_processed_df.columns` from `preprocess_data.py`.
+
+        # For now, let's create a DataFrame directly from the dict.
+        # This assumes input_data_dict has the correct features in a flat structure.
+        input_df = pd.DataFrame([input_data_dict])
path: src/predict_churn.py
user: login: sourcery-ai[bot]
body: **issue (code-quality):** Extract code out into function ([`extract-method`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/extract-method/))
created_at: 2025-05-21T21:01:43Z
author_association: NONE

PR comments:
user: login: sourcery-ai[bot]
created_at: 2025-05-21T20:56:14Z
body: <!-- Generated by sourcery-ai[bot]: start review_guide -->

## Reviewer's Guide

This PR establishes an end-to-end MLOps pipeline by adding data preprocessing, model training with MLflow integration, a prediction interface, unit tests, CI/CD workflows, and dependency management.

#### Sequence Diagram for Data Preprocessing and Model Training

```mermaid
sequenceDiagram
    participant RPP as scripts/run_preprocessing.py
    participant PPD as src/preprocess_data.py
    participant RDT as Raw Data (customer_data.csv)
    participant PRD as Processed Data (CSV files)
    participant RTP as scripts/run_training.py
    participant TM as src/train_model.py
    participant MLF as MLflow
    participant MDL as Trained Model (churn_model.joblib)

    RPP->>PPD: preprocess(raw_data_path)
    PPD->>RDT: Reads data
    PPD-->>RPP: X_processed_df, y_processed_series
    RPP->>PRD: Saves processed_features.csv
    RPP->>PRD: Saves processed_target.csv

    RTP->>TM: train_churn_model(features_path, target_path)
    TM->>PRD: Loads processed_features.csv
    TM->>PRD: Loads processed_target.csv
    TM->>MLF: Start Run
    TM->>TM: Train Logistic Regression Model
    TM->>MLF: Log parameters (solver, C, random_state)
    TM->>MLF: Log metrics (accuracy, f1_score)
    TM->>MLF: Log model (churn_model)
    TM->>MDL: Saves churn_model.joblib
    TM-->>RTP: model_path, run_id
```

#### Class Diagram for New MLOps Python Modules

```mermaid
classDiagram
direction LR
class preprocess_data {
    +preprocess(df_path string) tuple~DataFrame, Series~
}
class train_model {
    +train_churn_model(X_path string, y_path string) tuple~string, string~
}
class predict_churn {
    +make_prediction(input_data_dict dict) tuple~int, float~
}
class run_preprocessing_py {
    <<Script>>
    +main() void
}
class run_training_py {
    <<Script>>
    +main() void
}
run_preprocessing_py ..> preprocess_data : uses
run_training_py ..> train_model : uses
```

### File-Level Changes

| Change | Details | Files |
| ------ | ------- | ----- |
| Implemented data preprocessing pipeline | <ul><li>Convert 'TotalCharges' to numeric and fill NaNs</li><li>Label-encode 'Churn' target and drop customer ID</li><li>One-hot encode categorical features via ColumnTransformer</li><li>Provide a script to run preprocessing and save outputs</li></ul> | `src/preprocess_data.py`<br/>`scripts/run_preprocessing.py`<br/>`data/raw/customer_data.csv` |
| Added model training with MLflow logging | <ul><li>Load processed features and split into train/test sets</li><li>Train LogisticRegression and compute accuracy and F1 metrics</li><li>Log parameters, metrics, and model artifact to MLflow</li><li>Save trained model as a joblib file</li><li>Include a script to execute the training step</li></ul> | `src/train_model.py`<br/>`scripts/run_training.py` |
| Created prediction interface script | <ul><li>Load trained model and handle missing model errors</li><li>Convert input dict to DataFrame and ensure feature alignment</li><li>Generate churn prediction and probability outputs</li><li>Include example CLI usage with processed feature loading</li></ul> | `src/predict_churn.py` |
| Added unit tests for preprocessing | <ul><li>Verify 'TotalCharges' conversion and NaN handling</li><li>Check one-hot encoding of categorical columns</li><li>Validate target encoding and output shapes</li><li>Ensure customerID column is dropped</li></ul> | `tests/test_preprocess_data.py` |
| Configured GitHub Actions CI/CD workflows | <ul><li>Run unit tests on push and pull requests to main</li><li>Add a PR-specific job to run preprocessing and training</li><li>Set up Python environment and dependencies in CI</li></ul> | `.github/workflows/main.yml` |
| Managed project dependencies | <ul><li>List required libraries in requirements.txt</li></ul> | `requirements.txt` |

---

<details>
<summary>Tips and commands</summary>

#### Interacting with Sourcery

- **Trigger a new review:** Comment `@sourcery-ai review` on the pull request.
- **Continue discussions:** Reply directly to Sourcery's review comments.
- **Generate a GitHub issue from a review comment:** Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with `@sourcery-ai issue` to create an issue from it.
- **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull
  request title to generate a title at any time. You can also comment
  `@sourcery-ai title` on the pull request to (re-)generate the title at any time.
- **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment `@sourcery-ai summary` on the pull request to
  (re-)generate the summary at any time.
- **Generate reviewer's guide:** Comment `@sourcery-ai guide` on the pull
  request to (re-)generate the reviewer's guide at any time.
- **Resolve all Sourcery comments:** Comment `@sourcery-ai resolve` on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
- **Dismiss all Sourcery reviews:** Comment `@sourcery-ai dismiss` on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  `@sourcery-ai review` to trigger a new review!

#### Customizing Your Experience

Access your [dashboard](https://app.sourcery.ai) to:
- Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
- Change the review language.
- Add, remove or edit custom review instructions.
- Adjust other review settings.

#### Getting Help

- [Contact our support team](mailto:support@sourcery.ai) for questions or feedback.
- Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information.
- Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).

</details>

<!-- Generated by sourcery-ai[bot]: end review_guide -->
author_association: NONE
user: login: danieleschmidt
created_at: 2025-05-21T20:56:19Z
body: LGTM
author_association: OWNER