Reproducible FBA protocol benchmark: FBA vs pFBA vs loopless/FVA on E. coli
Domain: biology (systems-biology, method-benchmarking, constraint-based-modelling)
Topic: Reproducible FBA protocol benchmark: FBA vs pFBA vs loopless/FVA on E. coli
Venue: ARC-Bench Biology 2026
Headline metric: primary_metric (maximize)

Research briefing:
A method benchmark is well matched to mfa-agent because it tests robust use of
COBRApy APIs rather than biological novelty alone. A credible study loads a
validated E. coli GSMM, runs standard FBA, pFBA, loopless FBA when available,
and FVA at a fixed fraction of optimum, then compares growth, flux sparsity,
runtime, and reaction-level variability.

Hypotheses:
statement: FBA, pFBA, and loopless FBA preserve the same biomass optimum within solver tolerance under the same medium., statement: pFBA produces a lower total absolute flux norm and/or fewer active reactions than unconstrained FBA., statement: FVA reveals that only a subset of central-carbon reactions are tightly constrained near optimum while others remain variable.

Experiment design:
research_question: How do standard FBA, pFBA, loopless FBA, and FVA differ in flux parsimony and variability while preserving E. coli growth predictions?
conditions: name: standard_fba
description: Maximize biomass on aerobic glucose-minimal medium., name: pfba
description: Maximize biomass then minimize total absolute flux., name: loopless_fba
description: Run loopless solution when available and compare objective/fluxes., name: fva_95_percent_growth
description: Run FVA at 95% of optimum and classify rigid vs flexible reactions.
baselines: Standard FBA biomass optimum., COBRApy solver status and runtime.
metrics: name: growth_rate_1_per_h
direction: match
description: Biomass flux for each method., name: total_abs_flux
direction: minimize
description: Sum of absolute reaction fluxes., name: active_reaction_count
direction: minimize
description: Number of reactions with absolute flux above tolerance., name: fva_width
direction: characterize
description: Maximum minus minimum FVA interval per reaction.
datasets: name: E. coli GSMM
source: BIGG iJO1366 or iAF1260 via cobra.io.load_model
compute_requirements: gpu_required: no
estimated_wall_clock_sec: 1800

Requirements:
type: artifact
description: results.json contains method-level metrics, hypotheses, summary, and solver/version metadata., type: artifact
description: A method-comparison table exists with method, status, growth, total_abs_flux, active_reactions, and runtime., type: artifact
description: An FVA table exists with reaction ID, minimum, maximum, width, and rigid/flexible classification., type: artifact
description: At least one labelled figure compares methods or FVA variability., type: discussion
description: Each of hypotheses.h1, hypotheses.h2, hypotheses.h3 in results.json MUST have an explicit `supported` boolean and a `details` string of at least 40 characters describing the evidence used to reach that verdict.
, type: discussion
description: The protocol comparison is summarised with a method recommendation — which method is most reproducible and why, referencing flux-norm and FVA evidence. Acceptable inside summary, method_comparison, or a separate writeup field., type: discussion
description: The solver backend (cobra optlang interface name + version) and COBRApy version are reported in results.json. Required for reproducibility but not for scientific correctness.

Scoring rubric:
requirements: A reproducible FBA-method benchmark on E. coli that compares standard FBA, pFBA, loopless FBA when available, and FVA near optimum using consistent medium, metrics, runtime logging, and interpretable figures.
judging_note: This is a protocol-quality task. Reward robust implementation, structured outputs, and careful handling of unavailable loopless solvers over biological novelty.
sub_tasks: requirements: The code implements method comparison with consistent model state and metrics.
sub_tasks: requirements: Loads E. coli BIGG model, sets medium and biomass objective, and validates reference FBA.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Runs standard FBA, pFBA, and loopless FBA or records a justified loopless-unavailable status.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Runs FVA at a documented fraction of optimum and computes interval widths and rigid/flexible labels.
task_category: Code Development
finegrained_task_category: Method Implementation, requirements: Computes total absolute flux, active reaction count, objective value, solver status, and runtime for each method.
task_category: Code Development
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Execution produces complete benchmark artifacts and handles optional method failures explicitly.
sub_tasks: requirements: Standard FBA and pFBA solve optimally; loopless and FVA either solve or have explicit, non-crashing failure records.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Mass-balance check passes for the loaded model under the benchmark medium; FBA, pFBA, and loopless FBA all return non-negative biomass fluxes; no negative flux norms or ill-conditioned solver status appear without explicit logging. This is the biology-validity gate equivalent to physics-validity checks in the physics rubric.
task_category: Code Execution
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Writes results.json, method comparison table, FVA table, and method flux tables.
task_category: Code Execution
finegrained_task_category: Logging, Analysis & Presentation, requirements: Results compare objective preservation, parsimony, and reaction variability.
sub_tasks: requirements: Reports objective differences among methods and checks agreement within solver tolerance where methods succeed. Score 100% if FBA and pFBA biomass objectives agree within 1% relative error and any available loopless FBA also agrees within 1%; 67% if FBA and pFBA agree but loopless comparison is missing or failed with explicit explanation; 33% if only qualitative agreement is stated; 0% otherwise.
task_category: Result Analysis
finegrained_task_category: Evaluation, Metrics & Benchmarking, requirements: Quantifies pFBA parsimony using total flux norm and active reaction count compared with standard FBA. Score 100% if pFBA total absolute flux norm is numerically lower than FBA AND active reaction count is lower under pFBA; 67% if flux norm is lower but active reaction count is not compared; 33% if pFBA parsimony is stated qualitatively without numeric comparison; 0% otherwise.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: Summarizes FVA width distribution and names central-carbon reactions that are rigid or flexible near optimum. Score 100% if FVA interval widths are reported for all model reactions and ≥5 central-carbon reactions are classified as rigid (interval width < 1 mmol/gDW/h) with subsystem identification; 67% if rigid/flexible classification is reported without central-carbon specifics; 33% if FVA results are reported without classification; 0% otherwise.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: Produces labelled method-comparison and/or FVA-width figures with units and clear legends.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The README or results writeup discusses each hypothesis outcome, summarises the method-level comparison (parsimony, variability, runtime), and gives a one-paragraph protocol recommendation stating which method is most suitable for reproducible phenotype prediction and why.
task_category: Result Analysis
finegrained_task_category: Logging, Analysis & Presentation, requirements: The benchmark is reproducible with pinned model, medium, tolerance, solver, and code path.
sub_tasks: requirements: The loaded E. coli GSMM is either persisted (models/<model_id>.json) or pinned by BIGG model ID and source URL in the writeup; a downstream user can locate the exact model object and reproduce the benchmark medium.
task_category: Code Development
finegrained_task_category: Reproducibility, requirements: Saves model ID, objective, medium bounds, FVA fraction, active-flux tolerance, and method list.
task_category: Code Development
finegrained_task_category: Reproducibility, requirements: Records COBRApy, optlang, solver backend/version, and runtime environment.
task_category: Code Development
finegrained_task_category: Reproducibility