Repository: autogluon/autogluon-assistant (language: python)

Issue:
[Might Need Disscusion] Refine Time Limit
# Time Limit Implementation Strategies

issue #130 

## Core Principle
The core principle is that when a user specifies a time_limit, we should complete both training and prediction with the best available model, even if the time limit is reached. But as prediction time is hard to estimate, we have several options:

### Option 1: Excluding Prediction Time from Time Limit (Current)
- **Approach**: time_limit applies only to training (end_of_fit - start_of_init), excluding prediction time
- **Advantages**:
  - Simple to implement
  - Maximizes training time utilization
- **Disadvantages**:
  - Will always exceed specified time_limit by the prediction duration
  - Unpredictable total execution time from user perspective

### Option 2: Percentage-Based Allocation
- **Approach**: Reserve a percentage (e.g., 10%) of time_limit for prediction
- **Advantages**:
  - Scales proportionally with total time
  - Least likely to exceed time_limit
- **Disadvantages**:
  - May unnecessarily restrict training time for long time_limits
  - Example: With 12-hour time_limit, reserving 1.2 hours for prediction is excessive

### Option 3: Fixed Time Reservation
- **Approach**: Reserve a fixed duration (e.g., 5 minutes) for prediction
- **Advantages**:
  - Predictable prediction time allocation
  - Works well for medium-sized datasets
- **Disadvantages**:
  - Requires minimum time_limit threshold
  - Inefficient for small datasets
  - May be insufficient for large datasets

### Option 4: Hybrid Approach (Percentage with Cap)
- **Approach**: Reserve percentage of time_limit (e.g., 10%) with maximum cap (e.g., 20 minutes)
- **Advantages**:
  - Balances scaling and practicality
  - Prevents excessive prediction time reservation
- **Disadvantages**:
  - Introduces multiple hardcoded parameters
  - More complex to maintain and explain

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[Might Need Disscusion] Refine Time Limit
# Time Limit Implementation Strategies

issue #130 

## Core Principle
The core principle is that when a user specifies a time_limit, we should complete both training and prediction with the best available model, even if the time limit is reached. But as prediction time is hard to estimate, we have several options:

### Option 1: Excluding Prediction Time from Time Limit (Current)
- **Approach**: time_limit applies only to training (end_of_fit - start_of_init), excluding prediction time
- **Advantages**:
  - Simple to implement
  - Maximizes training time utilization
- **Disadvantages**:
  - Will always exceed specified time_limit by the prediction duration
  - Unpredictable total execution time from user perspective

### Option 2: Percentage-Based Allocation
- **Approach**: Reserve a percentage (e.g., 10%) of time_limit for prediction
- **Advantages**:
  - Scales proportionally with total time
  - Least likely to exceed time_limit
- **Disadvantages**:
  - May unnecessarily restrict training time for long time_limits
  - Example: With 12-hour time_limit, reserving 1.2 hours for prediction is excessive

### Option 3: Fixed Time Reservation
- **Approach**: Reserve a fixed duration (e.g., 5 minutes) for prediction
- **Advantages**:
  - Predictable prediction time allocation
  - Works well for medium-sized datasets
- **Disadvantages**:
  - Requires minimum time_limit threshold
  - Inefficient for small datasets
  - May be insufficient for large datasets

### Option 4: Hybrid Approach (Percentage with Cap)
- **Approach**: Reserve percentage of time_limit (e.g., 10%) with maximum cap (e.g., 20 minutes)
- **Advantages**:
  - Balances scaling and practicality
  - Prevents excessive prediction time reservation
- **Disadvantages**:
  - Introduces multiple hardcoded parameters
  - More complex to maintain and explain

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.



Discussion / hints:
/platform_tests
[Platform Tests Output][1]

[1]: https://github.com/autogluon/autogluon-assistant/actions/runs/11866096046
/platform_tests
[Platform Tests Output][1]

[1]: https://github.com/autogluon/autogluon-assistant/actions/runs/11866096046

Fix patch:
diff --git a/README.md b/README.md
index 0a2d569..c44133c 100644
--- a/README.md
+++ b/README.md
@@ -111,9 +111,9 @@ the `config_overrides` parameter with format `"key1=value1, key2.nested=value2"`
 Here are some example commands on using configuration overrides:
 
 ```bash
-aga run toy_data --config_overrides "feature_transformers.enabled_models=None, autogluon.predictor_fit_kwargs.time_limit=3600"
+aga run toy_data --config_overrides "feature_transformers.enabled_models=None, time_limit=3600"
 
 # OR
 
-aga run toy_data --config_overrides "feature_transformers.enabled_models=None" --config_overrides "autogluon.predictor_fit_kwargs.time_limit=3600"
+aga run toy_data --config_overrides "feature_transformers.enabled_models=None" --config_overrides "time_limit=3600"
 ```
diff --git a/src/autogluon/assistant/ui/pages/task.py b/src/autogluon/assistant/ui/pages/task.py
index 2662272..671b967 100644
--- a/src/autogluon/assistant/ui/pages/task.py
+++ b/src/autogluon/assistant/ui/pages/task.py
@@ -44,9 +44,7 @@ def update_config_overrides():
     """
     config_overrides = []
     if st.session_state.time_limit:
-        config_overrides.append(
-            f"autogluon.predictor_fit_kwargs.time_limit={TIME_LIMIT_MAPPING[st.session_state.time_limit]}"
-        )
+        config_overrides.append(f"time_limit={TIME_LIMIT_MAPPING[st.session_state.time_limit]}")
     if st.session_state.llm:
         config_overrides.append(f"llm.model={LLM_MAPPING[st.session_state.llm]}")
         config_overrides.append(f"llm.provider={PROVIDER_MAPPING[st.session_state.llm]}")


Test patch:
diff --git a/tests/unittests/integration/test_integration.py b/tests/unittests/integration/test_integration.py
index c379cda..23863e6 100644
--- a/tests/unittests/integration/test_integration.py
+++ b/tests/unittests/integration/test_integration.py
@@ -49,11 +49,11 @@ def titanic_data_path(tmp_path):
 def light_config():
     return OmegaConf.create(
         {
+            "time_limit": 300,  # 5 minutes timeout
             "llm": {"provider": "bedrock", "model": "anthropic.claude-3-5-haiku-20241022-v1:0"},
             "autogluon": {
                 "predictor_fit_kwargs": {
                     "presets": "medium_quality",  # lighter preset
-                    "time_limit": 300,  # 5 minutes timeout
                 }
             },
         }
@@ -66,7 +66,7 @@ def test_titanic_prediction(titanic_data_path, light_config):
         f"llm.provider={light_config.llm.provider}",
         f"llm.model={light_config.llm.model}",
         f"autogluon.predictor_fit_kwargs.presets={light_config.autogluon.predictor_fit_kwargs.presets}",
-        f"autogluon.predictor_fit_kwargs.time_limit={light_config.autogluon.predictor_fit_kwargs.time_limit}",
+        f"time_limit={light_config.time_limit}",
         "feature_transformers.enabled_models=null",
     ]