So here  below is the code please apply above improvement  as you see fit  with out reducing episodes in any way or data computational capacity only affect where its absolutely needed to 



Improvement suggested 

Looking at your RL-enhanced GARCH model results, several of these gradient optimization techniques could indeed help improve performance. Here's how they might apply to your specific situation:

## Most Relevant Solutions for Your RL-GARCH Model

**1. Gradient Clipping**
This is likely your most immediately useful solution. Given that your RL-enhanced model shows much higher prediction errors (MSE jumped from 0.00003957 to 0.00325203), you may be experiencing exploding gradients during the reinforcement learning training process. Gradient clipping would prevent the RL agent from making overly aggressive parameter updates.

**2. Better Weight Initialization**
Your RL model's higher errors suggest the neural network components may not be initializing properly. Using Xavier or He initialization for the neural network layers in your RL agent could provide more stable starting conditions for learning the GARCH parameter adjustments.

**3. Alternative Activation Functions**
If your RL agent uses sigmoid or tanh activations, switching to ReLU or Leaky ReLU could help. Financial time series often have complex, non-linear patterns that ReLU-based functions handle better, potentially improving your agent's ability to learn optimal volatility forecasting strategies.

## Implementation Priorities

**Start with Gradient Clipping:** Given your dramatic increase in prediction errors, this is likely where exploding gradients are hurting performance. Implement gradient clipping with a reasonable threshold (e.g., 1.0 or 0.5) during your RL training.

**Then Try Better Initialization:** Initialize your neural network weights using He initialization if you're using ReLU activations, or Xavier if using tanh/sigmoid.

**Consider Skip Connections:** If your RL agent uses a deep network architecture, residual connections could help maintain gradient flow through the network layers.

## Expected Impact

These techniques should help stabilize your RL training process and potentially bring your prediction metrics closer to the traditional GARCH performance levels. The goal would be to leverage RL's adaptability while maintaining the statistical rigor that makes traditional GARCH effective.

Would you like me to elaborate on implementing any of these specific techniques for your RL-GARCH hybrid model?


Code to which to be applied 


import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_squared_error, mean_absolute_error
from scipy import stats
import warnings
warnings.filterwarnings('ignore')

# RL and GARCH libraries
try:
    from arch import arch_model
    from arch.univariate import GARCH, Normal
except ImportError:
    print("Install arch library: pip install arch")

import gym
from gym import spaces
import random
from collections import deque

class GARCHRLEnvironment(gym.Env):
    """
    Enhanced Reinforcement Learning Environment for GARCH Parameter Optimization
    """
    def __init__(self, returns_data, window_size=50, validation_split=0.2):
        super(GARCHRLEnvironment, self).__init__()
        
        # Split data for validation
        split_idx = int(len(returns_data) * (1 - validation_split))
        self.train_data = returns_data[:split_idx]
        self.validation_data = returns_data[split_idx:]
        
        self.returns_data = self.train_data  # Use training data for environment
        self.window_size = window_size
        self.current_step = 0
        self.max_steps = len(self.returns_data) - window_size
        
        # Constrained action space for stability
        self.action_space = spaces.Box(low=0.01, high=0.99, shape=(3,), dtype=np.float32)
        
        # Enhanced state space
        self.observation_space = spaces.Box(
            low=-np.inf, high=np.inf, 
            shape=(window_size + 8,), dtype=np.float32
        )
        
        self.reset()
    
    def reset(self):
        self.current_step = self.window_size
        return self._get_observation()
    
    def _get_observation(self):
        """Enhanced observation with more market features"""
        start_idx = self.current_step - self.window_size
        end_idx = self.current_step
        
        historical_returns = self.returns_data[start_idx:end_idx]
        
        # Enhanced features
        volatility = np.std(historical_returns)
        mean_return = np.mean(historical_returns)
        skewness = stats.skew(historical_returns)
        kurtosis = stats.kurtosis(historical_returns)
        
        # Rolling statistics
        rolling_vol_5 = np.std(historical_returns[-5:])
        rolling_vol_10 = np.std(historical_returns[-10:])
        vol_ratio = rolling_vol_5 / rolling_vol_10 if rolling_vol_10 > 0 else 1.0
        
        # Persistence measure
        persistence = np.corrcoef(historical_returns[:-1], historical_returns[1:])[0, 1]
        if np.isnan(persistence):
            persistence = 0.0
        
        features = np.concatenate([
            historical_returns,
            [volatility, mean_return, skewness, kurtosis, rolling_vol_5, rolling_vol_10, vol_ratio, persistence]
        ])
        
        return features.astype(np.float32)
    
    def step(self, action):
        # Enhanced parameter mapping with constraints
        omega = action[0] * 0.01  # omega: [0.01, 0.01]
        alpha = action[1] * 0.2   # alpha: [0.01, 0.2]
        beta = action[2] * 0.8    # beta: [0.01, 0.8]
        
        # Strict stationarity constraint
        sum_params = alpha + beta
        if sum_params >= 0.99:
            # Rescale to maintain sum < 0.99
            alpha = alpha * 0.98 / sum_params
            beta = beta * 0.98 / sum_params
        
        reward = self._calculate_reward(omega, alpha, beta)
        
        self.current_step += 1
        done = self.current_step >= self.max_steps
        
        return self._get_observation(), reward, done, {'params': [omega, alpha, beta]}
    
    def _calculate_reward(self, omega, alpha, beta):
        """Enhanced reward function with multiple criteria"""
        try:
            start_idx = max(0, self.current_step - self.window_size)
            end_idx = self.current_step
            train_returns = self.returns_data[start_idx:end_idx]
            
            # Fit GARCH model with fixed parameters
            garch_model = arch_model(train_returns * 100, vol='Garch', p=1, q=1, rescale=False)
            
            # Manual parameter setting (simplified approach)
            if self.current_step < len(self.returns_data):
                actual_return = self.returns_data[self.current_step]
                
                # Simple volatility prediction using GARCH formula
                recent_vol = np.std(train_returns[-10:])
                predicted_vol_squared = omega + alpha * (train_returns[-1] ** 2) + beta * (recent_vol ** 2)
                predicted_vol = np.sqrt(max(predicted_vol_squared, 1e-6))
                
                actual_vol = abs(actual_return)
                
                # Multi-objective reward
                forecast_error = abs(predicted_vol - actual_vol)
                forecast_reward = -forecast_error * 1000
                
                # Stability reward (prefer alpha + beta close to 0.95)
                stability_reward = -abs(alpha + beta - 0.95) * 100
                
                # Parameter constraint rewards
                param_reward = 0
                if omega > 0 and alpha > 0 and beta > 0:
                    param_reward = 10
                
                # Volatility clustering reward
                clustering_reward = 0
                if len(train_returns) >= 2:
                    vol_persistence = np.corrcoef(np.abs(train_returns[:-1]), np.abs(train_returns[1:]))[0, 1]
                    if not np.isnan(vol_persistence) and vol_persistence > 0:
                        clustering_reward = vol_persistence * 50
                
                total_reward = forecast_reward + stability_reward + param_reward + clustering_reward
                return max(total_reward, -1000)  # Cap minimum reward
            
            return 0
            
        except Exception as e:
            return -1000  # Heavy penalty for errors

class EnhancedDQNAgent:
    """Enhanced DQN Agent with better exploration and stability"""
    def __init__(self, state_size, action_size, learning_rate=0.001):
        self.state_size = state_size
        self.action_size = action_size
        self.memory = deque(maxlen=50000)
        self.epsilon = 1.0
        self.epsilon_min = 0.05
        self.epsilon_decay = 0.9995
        self.learning_rate = learning_rate
        self.batch_size = 64
        
        # Better parameter tracking
        self.best_params_history = []
        self.performance_history = []
        
    def remember(self, state, action, reward, next_state, done):
        self.memory.append((state, action, reward, next_state, done))
    
    def act(self, state, training=True):
        if training and np.random.random() <= self.epsilon:
            # Constrained random exploration
            action = np.random.uniform(0.1, 0.9, self.action_size)
            # Ensure stationarity
            if action[1] + action[2] > 0.95:
                action[1] = action[1] * 0.9
                action[2] = action[2] * 0.9
            return action
        
        # Greedy action (simplified - in practice use neural network)
        # Return reasonable default parameters
        return np.array([0.3, 0.4, 0.5])  # These will be scaled in environment
    
    def replay(self):
        if len(self.memory) < self.batch_size:
            return
        
        # Simplified replay - in practice implement proper neural network training
        if self.epsilon > self.epsilon_min:
            self.epsilon *= self.epsilon_decay
        
        # Track learning progress
        recent_rewards = [exp[2] for exp in list(self.memory)[-100:]]
        avg_reward = np.mean(recent_rewards)
        self.performance_history.append(avg_reward)

class EnhancedGARCHRL:
    """Enhanced GARCH Model with RL and Validation"""
    def __init__(self, data_file='garch.xlsx'):
        self.data_file = data_file
        self.data = None
        self.returns = None
        self.scaler = StandardScaler()
        self.best_rl_params = None
        self.traditional_model = None
        self.validation_results = {}
        self.ensemble_weights = None
        
    def load_data(self):
        """Load and validate data"""
        try:
            self.data = pd.read_excel(self.data_file)
            print("Data loaded successfully!")
            print(f"Data shape: {self.data.shape}")
            print(f"Columns: {self.data.columns.tolist()}")
            
            # Handle different possible column names
            date_cols = [col for col in self.data.columns if 'date' in col.lower()]
            if date_cols:
                self.data['date'] = pd.to_datetime(self.data[date_cols[0]])
                self.data.set_index('date', inplace=True)
            
            # Get price column
            price_cols = [col for col in self.data.columns if any(x in col.lower() for x in ['close', 'price', 'adj'])]
            if price_cols:
                self.data['price'] = self.data[price_cols[0]]
            else:
                # Use first numeric column
                numeric_cols = self.data.select_dtypes(include=[np.number]).columns
                if len(numeric_cols) > 0:
                    self.data['price'] = self.data[numeric_cols[0]]
                else:
                    raise ValueError("No numeric price column found")
            
            # Calculate returns and clean data
            self.data['returns'] = self.data['price'].pct_change()
            self.data = self.data.dropna()
            
            # Remove extreme outliers (beyond 5 standard deviations)
            returns_std = self.data['returns'].std()
            returns_mean = self.data['returns'].mean()
            outlier_mask = np.abs(self.data['returns'] - returns_mean) < 5 * returns_std
            self.data = self.data[outlier_mask]
            
            self.returns = self.data['returns'].values
            
            print(f"Returns calculated: {len(self.returns)} observations")
            print(f"Return statistics: Mean={np.mean(self.returns):.6f}, Std={np.std(self.returns):.6f}")
            
            return True
            
        except Exception as e:
            print(f"Error loading data: {str(e)}")
            return False
    
    def traditional_garch(self):
        """Fit traditional GARCH(1,1) with cross-validation"""
        try:
            # Use 80% for training, 20% for validation
            split_idx = int(len(self.returns) * 0.8)
            train_returns = self.returns[:split_idx]
            val_returns = self.returns[split_idx:]
            
            # Scale returns
            scaled_train = train_returns * 100
            
            # Fit GARCH model
            garch_model = arch_model(scaled_train, vol='Garch', p=1, q=1, rescale=False)
            garch_results = garch_model.fit(disp='off')
            
            self.traditional_model = garch_results
            
            # Validate on out-of-sample data
            val_forecasts = []
            val_actuals = []
            
            # Rolling forecasts for validation
            for i in range(len(val_returns)):
                if i == 0:
                    forecast = garch_results.forecast(horizon=1)
                else:
                    # Re-fit with expanding window
                    extended_train = np.concatenate([train_returns, val_returns[:i]])
                    extended_scaled = extended_train * 100
                    temp_model = arch_model(extended_scaled, vol='Garch', p=1, q=1, rescale=False)
                    temp_results = temp_model.fit(disp='off')
                    forecast = temp_results.forecast(horizon=1)
                
                pred_vol = np.sqrt(forecast.variance.values[-1, 0]) / 100
                actual_vol = abs(val_returns[i])
                
                val_forecasts.append(pred_vol)
                val_actuals.append(actual_vol)
            
            # Calculate validation metrics
            val_mse = mean_squared_error(val_actuals, val_forecasts)
            val_mae = mean_absolute_error(val_actuals, val_forecasts)
            
            self.validation_results['traditional'] = {
                'mse': val_mse,
                'mae': val_mae,
                'forecasts': val_forecasts,
                'actuals': val_actuals
            }
            
            print("Traditional GARCH(1,1) Results:")
            print(garch_results.summary())
            
            params = garch_results.params
            omega = params['omega']
            alpha = params['alpha[1]']
            beta = params['beta[1]']
            
            print(f"\nValidation Results:")
            print(f"MSE: {val_mse:.8f}")
            print(f"MAE: {val_mae:.8f}")
            print(f"GARCH Parameters:")
            print(f"Omega: {omega:.6f}")
            print(f"Alpha: {alpha:.6f}")
            print(f"Beta: {beta:.6f}")
            print(f"Persistence (Alpha + Beta): {alpha + beta:.6f}")
            
            return garch_results, forecast
            
        except Exception as e:
            print(f"Error in traditional GARCH: {str(e)}")
            return None, None
    
    def rl_enhanced_garch(self, episodes=2000):
        """Enhanced RL training with validation"""
        print("Starting Enhanced RL-GARCH Training...")
        
        # Create environment with validation split
        env = GARCHRLEnvironment(self.returns, window_size=50, validation_split=0.2)
        
        # Create enhanced agent
        agent = EnhancedDQNAgent(
            state_size=env.observation_space.shape[0],
            action_size=env.action_space.shape[0]
        )
        
        # Training tracking
        episode_rewards = []
        best_params = None
        best_reward = -float('inf')
        param_history = []
        
        # Multi-stage training
        stages = [
            {'episodes': episodes//3, 'window': 30, 'description': 'Stage 1: Short-term patterns'},
            {'episodes': episodes//3, 'window': 50, 'description': 'Stage 2: Medium-term patterns'},
            {'episodes': episodes//3, 'window': 70, 'description': 'Stage 3: Long-term patterns'}
        ]
        
        for stage in stages:
            print(f"\n{stage['description']}")
            env.window_size = stage['window']
            env.max_steps = len(env.returns_data) - stage['window']
            
            for episode in range(stage['episodes']):
                state = env.reset()
                total_reward = 0
                done = False
                episode_params = []
                
                while not done:
                    action = agent.act(state, training=True)
                    next_state, reward, done, info = env.step(action)
                    
                    agent.remember(state, action, reward, next_state, done)
                    state = next_state
                    total_reward += reward
                    
                    if 'params' in info:
                        episode_params.append(info['params'])
                
                agent.replay()
                episode_rewards.append(total_reward)
                
                # Track best parameters
                if total_reward > best_reward:
                    best_reward = total_reward
                    best_params = episode_params[-1] if episode_params else None
                
                if episode % 200 == 0:
                    recent_rewards = episode_rewards[-100:]
                    avg_reward = np.mean(recent_rewards)
                    print(f"Episode {episode}, Avg Reward: {avg_reward:.2f}, Epsilon: {agent.epsilon:.3f}")
        
        # Validate RL parameters
        if best_params:
            self.best_rl_params = {
                'omega': best_params[0],
                'alpha': best_params[1],
                'beta': best_params[2]
            }
            
            # Cross-validate RL model
            self._validate_rl_model()
        
        print(f"\nRL Training Complete!")
        print(f"Best Parameters: {self.best_rl_params}")
        
        return best_params, episode_rewards
    
    def _validate_rl_model(self):
        """Validate RL model performance"""
        if not self.best_rl_params:
            return
        
        # Split data for validation
        split_idx = int(len(self.returns) * 0.8)
        train_returns = self.returns[:split_idx]
        val_returns = self.returns[split_idx:]
        
        omega = self.best_rl_params['omega']
        alpha = self.best_rl_params['alpha']
        beta = self.best_rl_params['beta']
        
        # Generate forecasts using RL parameters
        val_forecasts = []
        val_actuals = []
        
        # Rolling window validation
        for i in range(len(val_returns)):
            if i == 0:
                recent_returns = train_returns[-30:]
            else:
                recent_returns = np.concatenate([train_returns[-30:], val_returns[:i]])
            
            # Use GARCH(1,1) formula for prediction
            recent_vol = np.std(recent_returns)
            last_return = recent_returns[-1]
            
            # σ²(t+1) = ω + α * r²(t) + β * σ²(t)
            pred_var = omega + alpha * (last_return ** 2) + beta * (recent_vol ** 2)
            pred_vol = np.sqrt(max(pred_var, 1e-8))
            
            actual_vol = abs(val_returns[i])
            
            val_forecasts.append(pred_vol)
            val_actuals.append(actual_vol)
        
        # Calculate validation metrics
        val_mse = mean_squared_error(val_actuals, val_forecasts)
        val_mae = mean_absolute_error(val_actuals, val_forecasts)
        
        self.validation_results['rl_enhanced'] = {
            'mse': val_mse,
            'mae': val_mae,
            'forecasts': val_forecasts,
            'actuals': val_actuals
        }
        
        print(f"\nRL Model Validation:")
        print(f"MSE: {val_mse:.8f}")
        print(f"MAE: {val_mae:.8f}")
    
    def calculate_ensemble_weights(self):
        """Calculate optimal ensemble weights based on validation performance"""
        if 'traditional' not in self.validation_results or 'rl_enhanced' not in self.validation_results:
            self.ensemble_weights = [0.5, 0.5]  # Equal weights as fallback
            return
        
        # Use inverse MSE as weights (lower MSE = higher weight)
        trad_mse = self.validation_results['traditional']['mse']
        rl_mse = self.validation_results['rl_enhanced']['mse']
        
        # Avoid division by zero
        trad_weight = 1.0 / (trad_mse + 1e-8)
        rl_weight = 1.0 / (rl_mse + 1e-8)
        
        # Normalize weights
        total_weight = trad_weight + rl_weight
        self.ensemble_weights = [trad_weight / total_weight, rl_weight / total_weight]
        
        print(f"\nEnsemble Weights (Traditional, RL): {self.ensemble_weights}")
    
    def predict_next_day(self):
        """Enhanced prediction with ensemble approach and confidence intervals"""
        print("\n" + "="*80)
        print("ENHANCED NEXT DAY PREDICTIONS WITH VALIDATION")
        print("="*80)
        
        if not self.data.empty:
            latest_price = self.data['price'].iloc[-1]
            latest_return = self.returns[-1]
            
            print(f"Latest Price: ${latest_price:.2f}")
            print(f"Latest Return: {latest_return:.6f}")
            
            # Model comparison
            self._compare_models()
            
            # Calculate ensemble weights
            self.calculate_ensemble_weights()
            
            predictions = {}
            
            # 1. Traditional GARCH Prediction
            if self.traditional_model is not None:
                pred_trad = self._predict_traditional()
                predictions['traditional'] = pred_trad
                
            # 2. RL-Enhanced GARCH Prediction  
            if self.best_rl_params is not None:
                pred_rl = self._predict_rl()
                predictions['rl_enhanced'] = pred_rl
                
            # 3. Ensemble Prediction
            if len(predictions) == 2:
                pred_ensemble = self._predict_ensemble(predictions)
                predictions['ensemble'] = pred_ensemble
                
            # Display all predictions
            self._display_predictions(predictions, latest_price)
            
            # Risk analysis
            self._analyze_risk(predictions, latest_price)
            
            # Trading recommendations
            self._generate_trading_signals(predictions, latest_price)
            
            print("="*80)
            
            return predictions
        
        return None
    
    def _compare_models(self):
        """Compare model performance"""
        print(f"\n📊 MODEL PERFORMANCE COMPARISON:")
        
        if 'traditional' in self.validation_results and 'rl_enhanced' in self.validation_results:
            trad_results = self.validation_results['traditional']
            rl_results = self.validation_results['rl_enhanced']
            
            print(f"Traditional GARCH:")
            print(f"  MSE: {trad_results['mse']:.8f}")
            print(f"  MAE: {trad_results['mae']:.8f}")
            
            print(f"RL-Enhanced GARCH:")
            print(f"  MSE: {rl_results['mse']:.8f}")
            print(f"  MAE: {rl_results['mae']:.8f}")
            
            # Performance improvement
            mse_improvement = ((trad_results['mse'] - rl_results['mse']) / trad_results['mse']) * 100
            mae_improvement = ((trad_results['mae'] - rl_results['mae']) / trad_results['mae']) * 100
            
            print(f"RL Improvement:")
            print(f"  MSE: {mse_improvement:.2f}%")
            print(f"  MAE: {mae_improvement:.2f}%")
    
    def _predict_traditional(self):
        """Traditional GARCH prediction"""
        try:
            forecast = self.traditional_model.forecast(horizon=1)
            
            next_day_variance = forecast.variance.values[-1, 0]
            next_day_volatility = np.sqrt(next_day_variance) / 100
            
            mean_forecast = forecast.mean.values[-1, 0] / 100
            
            return {
                'volatility': next_day_volatility,
                'return': mean_forecast,
                'confidence': 'medium'  # Based on validation performance
            }
        except Exception as e:
            print(f"Error in traditional prediction: {str(e)}")
            return None
    
    def _predict_rl(self):
        """RL-enhanced GARCH prediction"""
        try:
            omega = self.best_rl_params['omega']
            alpha = self.best_rl_params['alpha']
            beta = self.best_rl_params['beta']
            
            # Current volatility estimate
            recent_returns = self.returns[-30:]
            current_vol = np.std(recent_returns)
            last_return = self.returns[-1]
            
            # GARCH forecast
            next_var = omega + alpha * (last_return ** 2) + beta * (current_vol ** 2)
            next_vol = np.sqrt(max(next_var, 1e-8))
            
            # Mean return (can be enhanced with trend analysis)
            mean_return = np.mean(recent_returns)
            
            return {
                'volatility': next_vol,
                'return': mean_return,
                'confidence': 'high'  # Assuming RL optimization improves confidence
            }
        except Exception as e:
            print(f"Error in RL prediction: {str(e)}")
            return None
    
    def _predict_ensemble(self, predictions):
        """Ensemble prediction"""
        if not self.ensemble_weights:
            self.calculate_ensemble_weights()
        
        trad_pred = predictions['traditional']
        rl_pred = predictions['rl_enhanced']
        
        if trad_pred and rl_pred:
            ensemble_vol = (self.ensemble_weights[0] * trad_pred['volatility'] + 
                          self.ensemble_weights[1] * rl_pred['volatility'])
            
            ensemble_return = (self.ensemble_weights[0] * trad_pred['return'] + 
                             self.ensemble_weights[1] * rl_pred['return'])
            
            return {
                'volatility': ensemble_vol,
                'return': ensemble_return,
                'confidence': 'high'
            }
        
        return None
    
    def _display_predictions(self, predictions, latest_price):
        """Display all predictions with confidence intervals"""
        z_scores = {'95%': 1.96, '99%': 2.576}
        
        for model_name, pred in predictions.items():
            if pred is None:
                continue
                
            print(f"\n📈 {model_name.upper()} PREDICTION:")
            print(f"   Volatility: {pred['volatility']:.6f}")
            print(f"   Expected Return: {pred['return']:.6f}")
            print(f"   Confidence: {pred['confidence']}")
            
            # Price predictions
            predicted_price = latest_price * (1 + pred['return'])
            print(f"   Predicted Price: ${predicted_price:.2f}")
            
            # Confidence intervals
            for conf_level, z_score in z_scores.items():
                upper = latest_price * (1 + pred['return'] + z_score * pred['volatility'])
                lower = latest_price * (1 + pred['return'] - z_score * pred['volatility'])
                interval_width = upper - lower
                
                print(f"   {conf_level} CI: [${lower:.2f}, ${upper:.2f}] (width: ${interval_width:.2f})")
    
    def _analyze_risk(self, predictions, latest_price):
        """Risk analysis"""
        print(f"\n⚠️  RISK ANALYSIS:")
        
        for model_name, pred in predictions.items():
            if pred is None:
                continue
                
            # Value at Risk (VaR)
            var_95 = latest_price * 1.96 * pred['volatility']
            var_99 = latest_price * 2.576 * pred['volatility']
            
            print(f"{model_name.capitalize()} VaR:")
            print(f"   95% VaR: ${var_95:.2f}")
            print(f"   99% VaR: ${var_99:.2f}")
    
    def _generate_trading_signals(self, predictions, latest_price):
        """Generate trading signals with confidence assessment"""
        print(f"\n💡 TRADING SIGNALS:")
        
        if 'ensemble' in predictions:
            pred = predictions['ensemble']
            predicted_price = latest_price * (1 + pred['return'])
            
            # Signal strength based on return magnitude and confidence
            return_pct = pred['return'] * 100
            vol_pct = pred['volatility'] * 100
            
            # Risk-adjusted signal
            if abs(return_pct) > vol_pct:  # Signal stronger than volatility
                if return_pct > 0:
                    signal = "🟢 STRONG BUY"
                else:
                    signal = "🔴 STRONG SELL"
            elif abs(return_pct) > vol_pct * 0.5:
                if return_pct > 0:
                    signal = "🟡 MODERATE BUY"
                else:
                    signal = "🟡 MODERATE SELL"
            else:
                signal = "⚪ NEUTRAL/HOLD"
            
            print(f"   Primary Signal: {signal}")
            print(f"   Expected Return: {return_pct:.3f}%")
            print(f"   Risk (Volatility): {vol_pct:.3f}%")
            print(f"   Risk-Adjusted Return: {return_pct/vol_pct:.2f}")
            
            # Model agreement analysis
            if len(predictions) > 1:
                trad_return = predictions.get('traditional', {}).get('return', 0) * 100
                rl_return = predictions.get('rl_enhanced', {}).get('return', 0) * 100
                
                agreement = abs(trad_return - rl_return)
                if agreement < 0.1:  # Very close predictions
                    print(f"   Model Agreement: HIGH (difference: {agreement:.3f}%)")
                elif agreement < 0.5:
                    print(f"   Model Agreement: MODERATE (difference: {agreement:.3f}%)")
                else:
                    print(f"   Model Agreement: LOW (difference: {agreement:.3f}%)")
                    print(f"   ⚠️  Caution: Models disagree significantly")
        
        # Additional risk warnings
        print(f"\n⚠️  RISK WARNINGS:")
        print(f"   • Past performance does not guarantee future results")
        print(f"   • High volatility indicates increased uncertainty")
        print(f"   • Consider position sizing based on risk tolerance")
        print(f"   • Monitor model performance over time")

# Usage example
def main():
    """Main execution function"""
    print("🚀 Enhanced GARCH-RL Model with Validation")
    print("="*50)
    
    # Initialize model
    model = EnhancedGARCHRL('garch.xlsx')
    
    # Step 1: Load and validate data
    print("\n📊 Loading Data...")
    if not model.load_data():
        print("❌ Failed to load data. Please check your file path and format.")
        return
    
    # Step 2: Fit traditional GARCH model
    print("\n🔧 Fitting Traditional GARCH Model...")
    traditional_results, traditional_forecast = model.traditional_garch()
    
    if traditional_results is None:
        print("❌ Failed to fit traditional GARCH model.")
        return
    
    # Step 3: Train RL-enhanced GARCH model
    print("\n🤖 Training RL-Enhanced GARCH Model...")
    rl_params, episode_rewards = model.rl_enhanced_garch(episodes=1500)
    
    if rl_params is None:
        print("❌ Failed to train RL model.")
        return
    
    # Step 4: Generate predictions and analysis
    print("\n🔮 Generating Predictions...")
    predictions = model.predict_next_day()
    
    # Step 5: Visualization
    print("\n📈 Creating Visualizations...")
    model.create_visualizations(episode_rewards, predictions)
    
    # Step 6: Model diagnostics
    print("\n🔍 Running Model Diagnostics...")
    model.run_diagnostics()
    
    # Step 7: Export results
    print("\n💾 Exporting Results...")
    model.export_results(predictions)
    
    print("\n✅ Analysis Complete!")
    print("Check the generated plots and exported results for detailed insights.")

def create_visualizations(self, episode_rewards, predictions):
    """Create comprehensive visualizations"""
    plt.style.use('seaborn-v0_8')
    fig = plt.figure(figsize=(20, 16))
    
    # 1. Price and Returns Time Series
    ax1 = plt.subplot(3, 3, 1)
    self.data['price'].plot(ax=ax1, title='Price Time Series', color='blue')
    ax1.set_ylabel('Price')
    ax1.grid(True, alpha=0.3)
    
    ax2 = plt.subplot(3, 3, 2)
    self.data['returns'].plot(ax=ax2, title='Returns Time Series', color='red', alpha=0.7)
    ax2.set_ylabel('Returns')
    ax2.grid(True, alpha=0.3)
    
    # 2. RL Training Progress
    ax3 = plt.subplot(3, 3, 3)
    if episode_rewards:
        # Smooth the rewards for better visualization
        smoothed_rewards = pd.Series(episode_rewards).rolling(window=50).mean()
        ax3.plot(episode_rewards, alpha=0.3, color='gray', label='Raw Rewards')
        ax3.plot(smoothed_rewards, color='blue', linewidth=2, label='Smoothed Rewards')
        ax3.set_title('RL Training Progress')
        ax3.set_xlabel('Episode')
        ax3.set_ylabel('Reward')
        ax3.legend()
        ax3.grid(True, alpha=0.3)
    
    # 3. Volatility Clustering
    ax4 = plt.subplot(3, 3, 4)
    rolling_vol = pd.Series(self.returns).rolling(window=20).std()
    rolling_vol.plot(ax=ax4, title='Rolling Volatility (20-day)', color='green')
    ax4.set_ylabel('Volatility')
    ax4.grid(True, alpha=0.3)
    
    # 4. Model Comparison - Validation Performance
    ax5 = plt.subplot(3, 3, 5)
    if 'traditional' in self.validation_results and 'rl_enhanced' in self.validation_results:
        models = ['Traditional', 'RL-Enhanced']
        mse_values = [self.validation_results['traditional']['mse'], 
                     self.validation_results['rl_enhanced']['mse']]
        mae_values = [self.validation_results['traditional']['mae'], 
                     self.validation_results['rl_enhanced']['mae']]
        
        x = np.arange(len(models))
        width = 0.35
        
        ax5.bar(x - width/2, mse_values, width, label='MSE', alpha=0.7)
        ax5.bar(x + width/2, mae_values, width, label='MAE', alpha=0.7)
        ax5.set_title('Model Performance Comparison')
        ax5.set_ylabel('Error')
        ax5.set_xticks(x)
        ax5.set_xticklabels(models)
        ax5.legend()
        ax5.grid(True, alpha=0.3)
    
    # 5. Prediction Accuracy
    ax6 = plt.subplot(3, 3, 6)
    if 'traditional' in self.validation_results:
        forecasts = self.validation_results['traditional']['forecasts']
        actuals = self.validation_results['traditional']['actuals']
        
        ax6.scatter(actuals, forecasts, alpha=0.6, label='Traditional')
        
        if 'rl_enhanced' in self.validation_results:
            rl_forecasts = self.validation_results['rl_enhanced']['forecasts']
            ax6.scatter(actuals, rl_forecasts, alpha=0.6, label='RL-Enhanced')
        
        # Perfect prediction line
        max_val = max(max(actuals), max(forecasts))
        ax6.plot([0, max_val], [0, max_val], 'r--', alpha=0.8, label='Perfect Prediction')
        
        ax6.set_xlabel('Actual Volatility')
        ax6.set_ylabel('Predicted Volatility')
        ax6.set_title('Prediction Accuracy')
        ax6.legend()
        ax6.grid(True, alpha=0.3)
    
    # 6. GARCH Parameters Evolution (if available)
    ax7 = plt.subplot(3, 3, 7)
    if hasattr(self, 'param_evolution'):
        for param_name, param_values in self.param_evolution.items():
            ax7.plot(param_values, label=param_name, linewidth=2)
        ax7.set_title('GARCH Parameters Evolution')
        ax7.set_xlabel('Training Steps')
        ax7.set_ylabel('Parameter Value')
        ax7.legend()
        ax7.grid(True, alpha=0.3)
    else:
        ax7.text(0.5, 0.5, 'Parameter Evolution\nData Not Available', 
                ha='center', va='center', transform=ax7.transAxes)
        ax7.set_title('GARCH Parameters Evolution')
    
    # 7. Residual Analysis
    ax8 = plt.subplot(3, 3, 8)
    if self.traditional_model is not None:
        residuals = self.traditional_model.resid
        ax8.hist(residuals, bins=30, alpha=0.7, color='blue', density=True)
        ax8.set_title('Residual Distribution')
        ax8.set_xlabel('Residuals')
        ax8.set_ylabel('Density')
        ax8.grid(True, alpha=0.3)
        
        # Add normal distribution overlay
        x = np.linspace(residuals.min(), residuals.max(), 100)
        y = stats.norm.pdf(x, residuals.mean(), residuals.std())
        ax8.plot(x, y, 'r-', linewidth=2, label='Normal Distribution')
        ax8.legend()
    
    # 8. Ensemble Weights
    ax9 = plt.subplot(3, 3, 9)
    if self.ensemble_weights:
        labels = ['Traditional', 'RL-Enhanced']
        ax9.pie(self.ensemble_weights, labels=labels, autopct='%1.1f%%', 
                startangle=90, colors=['lightblue', 'lightgreen'])
        ax9.set_title('Ensemble Model Weights')
    else:
        ax9.text(0.5, 0.5, 'Ensemble Weights\nNot Available', 
                ha='center', va='center', transform=ax9.transAxes)
        ax9.set_title('Ensemble Model Weights')
    
    plt.tight_layout()
    plt.savefig('garch_rl_analysis.png', dpi=300, bbox_inches='tight')
    plt.show()

def run_diagnostics(self):
    """Run comprehensive model diagnostics"""
    print("\n🔍 MODEL DIAGNOSTICS")
    print("="*50)
    
    # 1. Data Quality Checks
    print("\n📊 Data Quality Assessment:")
    print(f"   Total observations: {len(self.returns)}")
    print(f"   Missing values: {np.isnan(self.returns).sum()}")
    print(f"   Infinite values: {np.isinf(self.returns).sum()}")
    
    # 2. Statistical Tests
    print("\n📈 Statistical Tests:")
    
    # Normality test
    shapiro_stat, shapiro_p = stats.shapiro(self.returns[:5000])  # Limit sample size
    print(f"   Shapiro-Wilk test p-value: {shapiro_p:.6f}")
    print(f"   Returns are {'NOT ' if shapiro_p > 0.05 else ''}normally distributed")
    
    # Stationarity test
    from statsmodels.tsa.stattools import adfuller
    adf_stat, adf_p, _, _, critical_values, _ = adfuller(self.returns)
    print(f"   ADF test p-value: {adf_p:.6f}")
    print(f"   Returns are {'NOT ' if adf_p > 0.05 else ''}stationary")
    
    # ARCH effect test
    from statsmodels.stats.diagnostic import het_arch
    arch_stat, arch_p, _, _ = het_arch(self.returns, maxlag=5)
    print(f"   ARCH test p-value: {arch_p:.6f}")
    print(f"   {'Strong' if arch_p < 0.01 else 'Weak' if arch_p < 0.05 else 'No'} ARCH effects detected")
    
    # 3. Model Validation
    print("\n🔧 Model Validation:")
    
    if self.traditional_model is not None:
        print("   Traditional GARCH Model:")
        params = self.traditional_model.params
        
        # Parameter significance
        pvalues = self.traditional_model.pvalues
        for param, pval in pvalues.items():
            significance = "***" if pval < 0.001 else "**" if pval < 0.01 else "*" if pval < 0.05 else ""
            print(f"     {param}: p-value = {pval:.6f} {significance}")
        
        # Model fit statistics
        print(f"     Log-likelihood: {self.traditional_model.llf:.2f}")
        print(f"     AIC: {self.traditional_model.aic:.2f}")
        print(f"     BIC: {self.traditional_model.bic:.2f}")
    
    if self.best_rl_params:
        print("   RL-Enhanced GARCH Model:")
        omega, alpha, beta = self.best_rl_params.values()
        print(f"     Omega: {omega:.6f}")
        print(f"     Alpha: {alpha:.6f}")
        print(f"     Beta: {beta:.6f}")
        print(f"     Persistence (α+β): {alpha + beta:.6f}")
        print(f"     Stationarity: {'✓' if alpha + beta < 1 else '✗'}")
    
    # 4. Prediction Quality
    print("\n🎯 Prediction Quality:")
    for model_name, results in self.validation_results.items():
        print(f"   {model_name.replace('_', ' ').title()}:")
        print(f"     MSE: {results['mse']:.8f}")
        print(f"     MAE: {results['mae']:.8f}")
        print(f"     RMSE: {np.sqrt(results['mse']):.8f}")
        
        # Additional metrics
        forecasts = np.array(results['forecasts'])
        actuals = np.array(results['actuals'])
        
        # Mean Absolute Percentage Error
        mape = np.mean(np.abs((actuals - forecasts) / actuals)) * 100
        print(f"     MAPE: {mape:.2f}%")
        
        # Correlation
        correlation = np.corrcoef(forecasts, actuals)[0, 1]
        print(f"     Correlation: {correlation:.4f}")

def export_results(self, predictions):
    """Export analysis results to files"""
    try:
        # Create results directory
        import os
        if not os.path.exists('garch_rl_results'):
            os.makedirs('garch_rl_results')
        
        # 1. Export predictions
        pred_df = pd.DataFrame()
        for model_name, pred in predictions.items():
            if pred:
                pred_df[f'{model_name}_volatility'] = [pred['volatility']]
                pred_df[f'{model_name}_return'] = [pred['return']]
                pred_df[f'{model_name}_confidence'] = [pred['confidence']]
        
        pred_df.to_csv('garch_rl_results/predictions.csv', index=False)
        
        # 2. Export validation results
        val_df = pd.DataFrame()
        for model_name, results in self.validation_results.items():
            val_df[f'{model_name}_forecasts'] = pd.Series(results['forecasts'])
            val_df[f'{model_name}_actuals'] = pd.Series(results['actuals'])
        
        val_df.to_csv('garch_rl_results/validation_results.csv', index=False)
        
        # 3. Export model parameters
        params_dict = {}
        
        if self.traditional_model is not None:
            params_dict['traditional_omega'] = self.traditional_model.params['omega']
            params_dict['traditional_alpha'] = self.traditional_model.params['alpha[1]']
            params_dict['traditional_beta'] = self.traditional_model.params['beta[1]']
        
        if self.best_rl_params:
            params_dict.update({f'rl_{k}': v for k, v in self.best_rl_params.items()})
        
        if self.ensemble_weights:
            params_dict['ensemble_weight_traditional'] = self.ensemble_weights[0]
            params_dict['ensemble_weight_rl'] = self.ensemble_weights[1]
        
        params_df = pd.DataFrame([params_dict])
        params_df.to_csv('garch_rl_results/model_parameters.csv', index=False)
        
        # 4. Export summary report
        with open('garch_rl_results/analysis_summary.txt', 'w') as f:
            f.write("GARCH-RL Analysis Summary\n")
            f.write("="*50 + "\n\n")
            
            f.write(f"Data Period: {self.data.index[0]} to {self.data.index[-1]}\n")
            f.write(f"Total Observations: {len(self.returns)}\n")
            f.write(f"Return Statistics:\n")
            f.write(f"  Mean: {np.mean(self.returns):.6f}\n")
            f.write(f"  Std Dev: {np.std(self.returns):.6f}\n")
            f.write(f"  Skewness: {stats.skew(self.returns):.4f}\n")
            f.write(f"  Kurtosis: {stats.kurtosis(self.returns):.4f}\n\n")
            
            f.write("Model Performance:\n")
            for model_name, results in self.validation_results.items():
                f.write(f"  {model_name.replace('_', ' ').title()}:\n")
                f.write(f"    MSE: {results['mse']:.8f}\n")
                f.write(f"    MAE: {results['mae']:.8f}\n")
                f.write(f"    RMSE: {np.sqrt(results['mse']):.8f}\n\n")
            
            if predictions:
                f.write("Next Day Predictions:\n")
                for model_name, pred in predictions.items():
                    if pred:
                        f.write(f"  {model_name.replace('_', ' ').title()}:\n")
                        f.write(f"    Volatility: {pred['volatility']:.6f}\n")
                        f.write(f"    Expected Return: {pred['return']:.6f}\n")
                        f.write(f"    Confidence: {pred['confidence']}\n\n")
        
        print("✅ Results exported successfully to 'garch_rl_results/' directory")
        
    except Exception as e:
        print(f"❌ Error exporting results: {str(e)}")

# Add methods to the EnhancedGARCHRL class
EnhancedGARCHRL.create_visualizations = create_visualizations
EnhancedGARCHRL.run_diagnostics = run_diagnostics
EnhancedGARCHRL.export_results = export_results

# Execute the main function
if __name__ == "__main__":
    main()