doc/FBMC_Methodology_Explanation.md

Flow-Based Market Coupling (FBMC) Methodology Explanation

Quick Reference for FBMC Flow Forecasting MVP

---

1. What is FBMC?

Flow-Based Market Coupling (FBMC) is a European electricity market methodology that:

• Calculates cross-border trading capacity based on network physics (power flows)
• Replaces simple border-to-border capacity limits with network constraints
• Enables hub-to-hub trading between ANY two zones (not just physical neighbors)
• Maximizes market efficiency by considering the entire interconnected AC grid

Traditional ATC vs FBMC

Aspect	Traditional ATC	Flow-Based Market Coupling (FBMC)
Capacity Model	Border-to-border limits	Network-wide constraints (CNECs)
Trading Allowed	Only between physically connected zones	Between ANY two zones (hub-to-hub)
Network Physics	Simplified, ignores loop flows	Fully modeled via PTDFs
Example	FR can only trade with direct neighbors	FR can trade with HU despite no physical interconnector
Optimization	Sub-optimal (ignores network capacity)	Optimal (uses full network capacity)

---

2. Core FBMC Concepts

2.1 MaxBEX (Maximum Bilateral Exchange)

Definition: Commercial hub-to-hub trading capacity between two zones

Key Points:

• MaxBEX ≠ Physical interconnector ratings
• MaxBEX = Result of optimization considering ALL network constraints
• Calculated for ALL zone pairs: 12 × 11 = 132 bidirectional combinations
• Includes both physical borders and virtual borders

Physical Border Example (DE→FR):

- Physical interconnector: 3,000 MW capacity
- MaxBEX value: 2,450 MW
- Why lower? Network constraints (CNECs) in DE and FR limit capacity
- DE→FR exchange affects transmission lines in both countries

Virtual Border Example (FR→HU):

- Physical interconnector: NONE (no direct FR-HU cable)
- MaxBEX value: 1,200 MW
- How is this possible? Power flows through AC grid via DE, AT, CZ
- FR exports 1,200 MW, HU imports 1,200 MW
- Physical reality: Power flows through intermediate countries' grids

2.2 CNECs (Critical Network Elements with Contingencies)

Definition: Transmission line + contingency scenarios that constrain power flows

Structure:

CNEC = Transmission line + "What if X fails?"
Example: "German DE_CZ_LINE_123 under contingency: Czech power plant outage"

Key Metrics:

• RAM (Remaining Available Margin): How much flow capacity is left (MW)
• Shadow Price: Economic value of relaxing this constraint (€/MWh)
• Presolved: Boolean indicating if CNEC was binding (limiting)
• Fmax: Maximum allowed flow on this line (MW)

Why CNECs Matter:

• CNECs are the physical constraints that limit MaxBEX
• Each CNEC affects multiple borders simultaneously via PTDFs
• Top 50 CNECs account for ~80% of binding events

2.3 PTDFs (Power Transfer Distribution Factors)

Definition: Sensitivity coefficient showing how a zone's injection/withdrawal affects each CNEC

Interpretation:

PTDF_DE for a German CNEC = 0.45
→ If DE increases export by 1000 MW, this CNEC's flow increases by 450 MW

PTDF_FR for same CNEC = -0.22
→ If FR increases export by 1000 MW, this CNEC's flow decreases by 220 MW

Why PTDFs Enable Virtual Borders:

• FR→HU exchange has NO direct physical path
• But it affects CNECs in DE, AT, CZ via PTDFs
• PTDF_FR = +0.35, PTDF_HU = -0.28 for a German CNEC
• FR exports → increases German CNEC flow
• HU imports → decreases German CNEC flow
• Net effect: FR→HU exchange feasibility depends on German CNEC margin

PTDF Properties:

• Sum of all PTDFs ≈ 0 (Kirchhoff's law - flow conservation)
• High absolute PTDF = strong influence on that CNEC
• PTDFs are constants (depend only on network topology, not on flows)

---

3. How MaxBEX is Calculated

3.1 Optimization Problem

JAO solves this optimization problem daily:

Maximize: Σ (MaxBEX_ij) for all zone pairs (i→j)

Subject to:
1. For each CNEC k:
   Σ(PTDF_i^k × Net_Position_i) ≤ RAM_k  (Network constraint)

2. For each zone i:
   Σ(MaxBEX_ij) - Σ(MaxBEX_ji) = Net_Position_i  (Flow balance)

3. MaxBEX_ij ≥ 0  (Non-negative capacity)

Where:
- MaxBEX_ij = Capacity from zone i to zone j (WHAT WE FORECAST)
- PTDF_i^k = Zone i's PTDF for CNEC k
- RAM_k = Remaining Available Margin for CNEC k
- Net_Position_i = Net export from zone i

3.2 Why 132 Zone Pairs Exist

FBMC Core Bidding Zones (12 total):

• AT (Austria)
• BE (Belgium)
• CZ (Czech Republic)
• DE (Germany-Luxembourg)
• FR (France)
• HR (Croatia)
• HU (Hungary)
• NL (Netherlands)
• PL (Poland)
• RO (Romania)
• SI (Slovenia)
• SK (Slovakia)

All Permutations:

Total bidirectional pairs = 12 × 11 = 132

Examples:
- AT→BE, AT→CZ, AT→DE, ..., AT→SK  (11 directions from AT)
- BE→AT, BE→CZ, BE→DE, ..., BE→SK  (11 directions from BE)
- ...
- SK→AT, SK→BE, SK→CZ, ..., SK→SI  (11 directions from SK)

Physical vs Virtual:

• ~40-50 physical borders (zones with direct interconnectors)
• ~80-90 virtual borders (zones without direct interconnectors)

---

4. Network Physics: Power Flow Reality

4.1 AC Grid Fundamentals

Key Principle: Power flows through ALL available paths, not just the intended route

Example: DE→PL bilateral exchange

Intended:  DE → PL (direct interconnector)
Reality:   Power also flows through CZ and SK (parallel paths)
Result:    CZ and SK CNECs are affected, limiting DE→PL capacity

4.2 Loop Flows

Definition: Unintended power flows through neighboring countries

FR→HU Exchange Example:

Commercial transaction: FR exports 1000 MW, HU imports 1000 MW

Physical reality (power flow percentages):
- 0% flows directly (no FR-HU interconnector)
- 35% flows through DE grid (PTDF_DE = +0.35)
- 28% flows through AT grid (PTDF_AT = +0.28)
- 22% flows through CZ grid (PTDF_CZ = +0.22)
- 15% flows through other paths (SI, HR, SK)

Impact:
- German CNECs see +350 MW load (may become binding)
- Austrian CNECs see +280 MW load (may become binding)
- Czech CNECs see +220 MW load (may become binding)
- MaxBEX(FR→HU) limited by most constraining CNEC

4.3 Why Virtual Borders Have Lower Capacity

Physical Border (DE→FR):

• Direct interconnector: 3,000 MW rating
• MaxBEX: Often 2,200-2,800 MW
• Reason: Local CNECs in DE and FR

Virtual Border (FR→HU):

• Direct interconnector: None
• MaxBEX: Often 800-1,500 MW
• Reason: Power flows through DE, AT, CZ (affects many CNECs)
• More CNECs affected → more constraints → lower capacity

---

5. FBMC Data Series Relationships

5.1 Data Hierarchy

MaxBEX (TARGET)
    ↑ Result of optimization
CNECs + PTDFs + RAM
    ↑ Network constraints
LTN (Long-Term Nominations)
    ↑ Pre-allocated capacity
Net Positions (Min/Max)
    ↑ Zone-level limits
Planned Outages
    ↑ Reduce RAM availability

5.2 Causal Chain

1. Planned Outages → Reduce RAM for affected CNECs
2. Reduced RAM → Tighter CNEC constraints
3. Tighter constraints + PTDFs → Limit MaxBEX
4. MaxBEX optimization → 132 capacity values

5.3 What We Forecast

Forecasting Task: Predict MaxBEX for all 132 zone pairs, D+1 to D+14 horizon

Input Features (~1,735 features):

• Historical MaxBEX (past 21 days)
• CNEC binding patterns (200 CNECs × 8 features)
• PTDFs (200 CNECs × 12 zones, aggregated)
• RAM time series (200 CNECs)
• Shadow prices (200 CNECs)
• Planned outages (200 CNECs, future covariates)
• Weather forecasts (52 grid points, future covariates)
• LTN allocations (known in advance)
• Net positions (min/max bounds)

Output: MaxBEX forecast for 132 zone pairs × 336 hours (14 days)

Evaluation Metric: MAE (Mean Absolute Error) in MW, aggregated across all borders

---

6. Why This Matters for Forecasting

6.1 Multivariate Dependencies

Key Insight: You cannot forecast MaxBEX(DE→FR) independently of MaxBEX(FR→DE) or MaxBEX(AT→CZ)

Reason: All borders share the same CNEC constraints via PTDFs

Example:

If German CNEC "DE_NORTH_LINE_5" is binding with RAM = 200 MW:
- MaxBEX(DE→FR) is limited
- MaxBEX(DE→NL) is limited
- MaxBEX(PL→DE) is limited
- MaxBEX(FR→CZ) is affected (loop flows through DE)

All of these borders compete for the same 200 MW of remaining margin!

6.2 Network Constraints Drive Capacity

Not driven by:

• Historical MaxBEX averages (too simplistic)
• Physical interconnector ratings (not the binding constraint)
• Bilateral flow patterns (ignores network physics)

Driven by:

• Which CNECs are binding (top 50 account for ~80% of binding events)
• How much RAM is available (affected by outages, weather, generation patterns)
• PTDF patterns (which zones affect which CNECs)
• LTN pre-allocations (reduce available capacity)

6.3 Why Chronos 2 is Well-Suited

Chronos 2 Strengths (for zero-shot FBMC forecasting):

1. Multivariate context: Sees all 132 borders + 1,735 features simultaneously
2. Temporal patterns: Learns hourly, daily, weekly cycles in CNEC binding
3. Attention mechanism: Focuses on top binding CNECs for each forecast horizon
4. Pre-trained on diverse time series: Generalizes to electricity network physics
5. Zero-shot: No fine-tuning needed for MVP (target: 134 MW MAE)

Why CNEC features are critical:

• CNECs = physical constraints that determine MaxBEX
• Without CNEC context, model would miss network bottlenecks
• Top 50 CNECs × 20 features = 1,000 features capturing network state

---

7. Practical Example Walkthrough

Scenario: Forecasting DE→FR MaxBEX for Tomorrow (D+1)

Step 1: Gather Historical Context (21 days lookback)

- MaxBEX(DE→FR) past 21 days: avg 2,450 MW, std 320 MW
- Top 10 binding CNECs affecting DE→FR:
  * German CNEC "DE_SOUTH_1": Binding 60% of time, avg shadow price 45 €/MWh
  * French CNEC "FR_EAST_3": Binding 40% of time, avg shadow price 38 €/MWh
- Historical RAM for these CNECs: trending down (more congestion)
- Recent outages: None planned for DE or FR

Step 2: Future Covariates (D+1 to D+14)

- Planned outages: French line "FR_EAST_3" scheduled maintenance D+3 to D+7
  → Expect lower MaxBEX(DE→FR) during this period
- Weather forecast: High winds in DE (high renewables) → Higher DE export pressure
- LTN allocations: 400 MW pre-allocated for long-term contracts

Step 3: CNEC Impact Analysis

German CNEC "DE_SOUTH_1":
- PTDF_DE = +0.42 (DE export increases flow)
- PTDF_FR = -0.35 (FR import decreases flow)
- Current RAM = 450 MW
- DE→FR exchange adds: 0.42 × 1000 - 0.35 × (-1000) = 770 MW to CNEC flow
- Therefore: MaxBEX(DE→FR) ≤ 450 / 0.77 = 584 MW (if this CNEC is limiting)

French CNEC "FR_EAST_3":
- PTDF_DE = +0.38
- PTDF_FR = -0.40
- Current RAM = 600 MW
- DE→FR exchange adds: 0.38 × 1000 - 0.40 × (-1000) = 780 MW to CNEC flow
- Therefore: MaxBEX(DE→FR) ≤ 600 / 0.78 = 769 MW

Most constraining: German CNEC → MaxBEX(DE→FR) ≈ 584 MW

Step 4: Chronos 2 Inference

Input features (1,735-dim vector):
- Historical MaxBEX context (132 borders × 21 days)
- CNEC features (200 CNECs × 8 metrics)
- PTDF aggregates (132 borders × PTDF sums)
- Future outages (200 CNECs × 14 days)
- Weather forecasts (52 grid points × 14 days)

Chronos 2 output:
- MaxBEX(DE→FR) forecast: 620 MW (D+1, hour 12:00)
- Confidence: Model attention focused on "DE_SOUTH_1" CNEC
- Interpretation: Slightly above CNEC-derived limit due to other borders absorbing some CNEC load

Step 5: Validation

Actual MaxBEX(DE→FR) = 605 MW
Forecast = 620 MW
Error = 15 MW (within 134 MW target MAE)

---

8. Common Misconceptions

Misconception 1: "MaxBEX = Interconnector Capacity"

❌ Wrong: MaxBEX is often much lower than interconnector ratings ✅ Correct: MaxBEX is the result of network-wide optimization considering all CNECs

Misconception 2: "Virtual borders have zero capacity"

❌ Wrong: Virtual borders can have significant capacity (e.g., FR→HU: 800-1,500 MW) ✅ Correct: Virtual borders represent feasible commercial exchanges via AC grid network

Misconception 3: "Each border can be forecasted independently"

❌ Wrong: All borders are coupled via shared CNEC constraints ✅ Correct: Multivariate forecasting is essential (Chronos 2 sees all 132 borders simultaneously)

Misconception 4: "PTDFs change with power flows"

❌ Wrong: PTDFs are NOT flow-dependent ✅ Correct: PTDFs are constants determined by network topology (linearity assumption in DC power flow)

Misconception 5: "Only physical borders matter for trading"

❌ Wrong: FBMC enables trading between ANY zone pairs ✅ Correct: All 132 zone-pair combinations have commercial capacity via grid network

---

9. References and Further Reading

Official JAO Documentation

• JAO Publication Tool User Guide: https://publicationtool.jao.eu/help
• JAO FBMC Methodology: Available via JAO website
• Core FBMC Practitioners Guide: doc/practitioners_guide.pdf

ENTSO-E Resources

• ENTSO-E Transparency Platform: https://transparency.entsoe.eu/
• FBMC Overview: ENTSO-E publications on flow-based market coupling

Academic References

• Ehrenmann, A., & Neuhoff, K. (2009). A comparison of electricity market designs in networks. Operations Research, 57(2), 274-286.
• Pellini, E. (2012). Measuring the impact of market coupling on the Italian electricity market. Energy Policy, 48, 322-333.

Project Documentation

• doc/JAO_Data_Treatment_Plan.md: Complete data collection and feature extraction guide
• doc/FBMC_Flow_Forecasting_MVP_ZERO_SHOT_PLAN.md: 5-day MVP implementation plan
• notebooks/01_data_exploration.py: Interactive data exploration with sample data

---

10. Summary: Key Takeaways

1. MaxBEX ≠ Physical Capacity: MaxBEX is a commercial metric derived from network optimization
2. 132 Zone Pairs: All 12 × 11 bidirectional combinations exist (physical + virtual borders)
3. CNECs Are Key: Network constraints (CNECs) determine MaxBEX via optimization
4. PTDFs Enable Virtual Borders: Power flows through AC grid network affect distant CNECs
5. Multivariate Forecasting Required: All borders share CNEC constraints via PTDFs
6. Network Physics Matters: Loop flows, congestion patterns, and outages drive capacity
7. Chronos 2 Zero-Shot Approach: Pre-trained model leverages multivariate context without fine-tuning

---

Document Version: 1.0 Created: 2025-11-03 Project: FBMC Flow Forecasting MVP (Zero-Shot) Purpose: Comprehensive reference for understanding FBMC methodology and MaxBEX forecasting