Context:
You are given a Python script named Kali_FibreG1.py which is a large FTTH/FTTB/FTTx network planner (approx. 3500 lines). It includes both CLI and Tkinter-based GUI modes. You are to implement the following enhancements while preserving all existing logic and functionality.

✅ Required Modifications to the GUI (Rewritten in Qt6)
Ignore any instruction to modify into Qt6 - Use the Tkinter GUI and implement the changes.

Add Checkbox – Use All Building Centroids

Label: Use All OSM Building Centroids

Tooltip: Include all valid OSM building centroids, even if they lack metadata tags.

Default: Unchecked

Bind this to a parameter (use_all_centroids: bool) passed to the backend.

Add File Selector – Import Custom Erven Data

Label: Load Custom Erven GeoJSON/Shapefile

Accepts: .geojson, .shp, .zip

When file is selected, load the file using geopandas.read_file() and extract centroids as alternative client points.

If this is used, ignore OSM building data entirely.

Add Visual Confirmation/Preview Section
After loading input (either OSM buildings or custom erven), show:

✅ Total Clients Extracted: [count]

✅ Source: [OSM / Custom Erven]

Update the display in real time after selecting the place or loading a custom file.

Preserve Other Inputs
Ensure all existing GUI inputs (like area name, output folder, etc.) are retained in the new layout.

Rebind Logic
Make sure the updated UI triggers the backend (main planner logic) with the new parameters:

python
Copy
Edit
use_all_centroids: bool = True/False
custom_erven_path: Optional[str]
Optional Enhancement:

Allow the user to set Max Clients per Aggregation Node via a QSpinBox, default value 100.


📌 Notes
Use signals/slots for GUI event handling.

Preview the number of clients found before running the main analysis.

Keep routing, PON grouping, and duct bank logic untouched unless necessary to bind the new input sources.
"""
Enhanced Structured FTTH/FTTB/FTTx Fibre Network Design Tool with Big Area Support & Optimized Routing
----------------------------------------------------------------------------------------------------
Version: 3.3
Date: 2025-04-16
Author: Your Company/Name

This tool automates the design of a preliminary fibre network solution. It:
  • Utilizes OpenStreetMap (OSM) data for road networks and building footprints.
  • Employs UTM-based projections for accurate distance and area calculations.
  • Derives potential client endpoints from OSM building centroids.
  • Calculates drop fibre routes (straight line) from each client to the nearest point on a derived duct bank (following road network).
  • Routes distribution cables along the road network from the drop connection point to the nearest optimized manhole/splitter location.
  • Routes feeder cables along the road network from each used manhole back to a zone's central aggregation node (selected based on centrality/density).
  • Generates placeholder splice allocation strings for documentation.
  • Supports automatic subdivision of large areas based on configurable limits.
  • Produces multi-layer KML output for visualization and detailed CSV reports for analysis.
  • Includes configurable cost estimation for the complete fibre infrastructure.
  • Utilizes clustering (DBSCAN) for potential manhole placement optimization, aligning locations with network nodes (road intersections, client proximity).
  • Employs multi-threading for improved performance during data processing and client routing (subject to GIL limitations for CPU-bound tasks).
  • Features robust error handling, retry logic for external API calls (OSM), and routing fallbacks with clear logging.

Usage:
  • CLI mode: python <scriptname.py> --cli --place "Your Place Name" [options]
  • GUI mode: Run script without the --cli argument.

Key Improvements in v3.3 (Compared to v3.2):
  • Modularity: Refactored core `process_zone` logic into smaller, more focused functions.
  • Parameterization: Added option to disable computationally intensive centrality calculation for aggregation node selection.
  • Robustness: Improved handling of empty or invalid geometries during processing and projection. Enhanced logging for routing fallbacks.
  • KML Enhancements: Added distinct style for fallback routes (straight-line). Added zone boundaries folder.
  • CSV Reports: Included aggregation node details in infrastructure report. Ensured consistent data reporting.
  • GUI Improvements: More robust parameter validation on input. Clearer state management during execution. Added basic parameter grouping.
  • Code Quality: Added `slots=True` to dataclasses for potential memory/performance gains. General code cleanup, improved comments, and type hint consistency. Added more constants.
  • Dependencies: Ensured dependency check runs before GUI/CLI execution.
  • Configuration: Parameter loading/saving refined.
"""

import os
import sys
import random
import logging
import threading
import csv
import argparse
import time
import numpy as np
import tkinter as tk
from tkinter import ttk, messagebox, filedialog
from shapely.geometry import Polygon, Point, LineString, MultiPoint, MultiPolygon, box
from shapely.ops import transform, unary_union, nearest_points
import osmnx as ox
import simplekml
import networkx as nx
from pyproj import CRS, Transformer, ProjError
from sklearn.cluster import KMeans, DBSCAN
# Note: Matplotlib is not actively used unless visualization is added back
# import matplotlib.pyplot as plt
from concurrent.futures import ThreadPoolExecutor, as_completed
import contextlib
import json
from dataclasses import dataclass, field, asdict
from typing import List, Dict, Tuple, Optional, Union, Any, Set
import math
from collections import defaultdict
import warnings

# ---------------------------
# Global Configuration / Constants
# ---------------------------
VERSION = "3.3"
APP_NAME = "Fibre Network Designer"

# API and Processing
OSM_TIMEOUT = 180  # Timeout for OSMnx API calls in seconds
MAX_API_RETRIES = 3
API_RETRY_DELAY = 5  # Initial delay in seconds for API retries (exponential backoff used)
DEFAULT_OUTPUT_DIR = "output"
LOG_DIR = "logs"
MIN_NODES_FOR_CLUSTERING = 5 # Minimum potential manhole locations before clustering is skipped
MIN_ZONE_AREA_M2 = 10.0 # Minimum area in m² for a subdivided zone to be considered valid
GRAPH_BUFFER_DEGREES = 0.0005 # Approx 50-55m buffer for fetching graph around zones
MAX_DIST_M_FROM_CENTROID_FOR_AGG_NODE = 1000.0 # Max distance for centrality candidates near client centroid
MAX_NODES_FOR_CENTRALITY_CALC = 5000 # Threshold above which centrality is skipped for performance

# File Extensions
PARAMS_EXT = ".json"
KML_EXT = ".kml"
CSV_EXT = ".csv"
LOG_EXT = ".log"
TXT_EXT = ".txt"

# KML Style IDs (used internally)
STYLE_CLIENT = "client"
STYLE_MANHOLE = "manhole"
STYLE_AGGREGATION = "aggregation"
STYLE_DUCT = "duct"
STYLE_DROP = "drop"
STYLE_DISTRIBUTION = "distribution"
STYLE_FEEDER = "feeder"
STYLE_ZONE = "zone"
STYLE_FALLBACK = "fallback_route"


# Suppress specific UserWarnings from shapely/numpy that are expected in some cases
warnings.filterwarnings("ignore", "Creating an ndarray from ragged nested sequences", UserWarning) # numpy/shapely coords
warnings.filterwarnings("ignore", message="Geometry is set to nil", category=UserWarning) # osmnx internal

# ---------------------------
# Logging configuration
# ---------------------------
log_formatter = logging.Formatter("%(asctime)s [%(levelname)s] [%(threadName)s]: %(message)s")
log_handler = logging.StreamHandler()
log_handler.setFormatter(log_formatter)

logger = logging.getLogger("FibreNetworkDesigner")
logger.setLevel(logging.INFO) # Default level, can be adjusted

# Prevent duplicate handlers if script is re-run in interactive sessions
if not logger.handlers:
    logger.addHandler(log_handler)

    # Add file handler for persistent logging
    os.makedirs(LOG_DIR, exist_ok=True)
    file_handler = logging.FileHandler(f"{LOG_DIR}/fibre_design_{time.strftime('%Y%m%d_%H%M%S')}{LOG_EXT}")
    file_handler.setFormatter(log_formatter)
    logger.addHandler(file_handler)

# ---------------------------
# Data Models
# ---------------------------
@dataclass(slots=True)
class NetworkParameters:
    """Global network design parameters"""
    max_clients_per_splitter: int = 64 # Theoretical max, actual clustering might result in fewer
    drop_fibre_strains: int = 8
    distribution_fibre_strains: int = 24
    feeder_fibre_strains: int = 96
    max_zone_area: float = 5e7  # 50 km² (max area before subdivision)
    max_drop_length: float = 500.0  # Maximum length for a drop cable in meters

    # Budget costs (in currency units) - Placeholder values
    budget_trench_per_m: float = 82.50  # Combined trenching/ducting cost estimate
    budget_duct_per_m: float = 35.00   # Can be added to trench or used separately if needed
    budget_manhole: float = 715.00
    budget_drop_fibre_per_m: float = 3.25
    budget_dist_fibre_per_m: float = 5.50
    budget_feeder_fibre_per_m: float = 8.75
    budget_splitter_cost: float = 420.00 # Assume 1:16 or similar as base cost
    budget_terminal_box_per_client: float = 85.00
    budget_splicing_per_connection: float = 15.00 # Rough estimate per splice/connection point

    # Resource optimization parameters
    optimize_manholes: bool = True
    manhole_clustering_distance: float = 150.0  # meters (DBSCAN epsilon for grouping potential locations)
    use_network_routing: bool = True # Use road network for distribution/feeder routing
    use_centrality_for_aggregation: bool = True # Calculate centrality for aggregation node (can be slow)

    def to_dict(self):
        """Convert parameters to dictionary for serialization"""
        return asdict(self)

    @classmethod
    def from_dict(cls, data: Dict[str, Any]):
        """Create parameters from dictionary, ignoring extra keys and performing basic type validation."""
        valid_keys = {f.name for f in cls.__dataclass_fields__.values()}
        expected_types = {f.name: f.type for f in cls.__dataclass_fields__.values()}
        filtered_data = {}
        errors = []

        for k, v in data.items():
            if k in valid_keys:
                expected_type = expected_types[k]
                try:
                    # Attempt conversion for basic types if needed (e.g., string from JSON to number)
                    if expected_type is float and not isinstance(v, float):
                        filtered_data[k] = float(v)
                    elif expected_type is int and not isinstance(v, int):
                        filtered_data[k] = int(v)
                    elif expected_type is bool and not isinstance(v, bool):
                        # Handle common boolean string representations
                        if isinstance(v, str):
                            if v.lower() in ['true', '1', 'yes', 'y']:
                                filtered_data[k] = True
                            elif v.lower() in ['false', '0', 'no', 'n']:
                                filtered_data[k] = False
                            else:
                                raise ValueError(f"Invalid boolean string value '{v}'")
                        else: # Try standard bool conversion
                            filtered_data[k] = bool(v)
                    elif isinstance(v, expected_type): # Type already matches
                         filtered_data[k] = v
                    else: # Type mismatch not handled above
                         raise ValueError(f"Incorrect type {type(v)} != {expected_type}")

                except (ValueError, TypeError) as e:
                    errors.append(f"Parameter '{k}': Invalid value '{v}' for type {expected_type}. Error: {e}")
            # else: Ignore keys not defined in the dataclass

        if errors:
            error_msg = "Invalid data provided for NetworkParameters:\n" + "\n".join(errors)
            logger.error(error_msg)
            raise ValueError(error_msg)

        # Create instance using only valid, typed keys
        try:
            return cls(**filtered_data)
        except TypeError as e: # Should be less likely after above checks, but possible
            logger.error(f"Error creating NetworkParameters instance: {e}. Filtered Data: {filtered_data}")
            raise ValueError(f"Initialization error for NetworkParameters: {e}") from e


@dataclass(slots=True)
class DesignStatistics:
    """Statistics for the network design"""
    total_clients: int = 0
    total_drop_length: float = 0.0
    total_distribution_length: float = 0.0
    total_feeder_length: float = 0.0
    total_manholes: int = 0         # Total unique manholes placed/considered across all zones
    total_splitters: int = 0        # Total manholes actually used (connected to clients) across all zones
    total_cost: float = 0.0
    zones_count: int = 0
    start_time: float = field(default_factory=time.time)
    clients_over_max_drop: int = 0
    routing_fallbacks: int = 0 # Count of routes (dist+feeder) using straight line fallback
    failed_zones: int = 0

    def calculate_costs(self, params: NetworkParameters):
        """Calculate the total estimated cost based on network statistics"""
        # Trenching/Ducting for Distribution and Feeder routes
        # Assuming cost includes both trenching and primary duct
        trench_duct_cost = (self.total_distribution_length + self.total_feeder_length) * params.budget_trench_per_m

        # Fibre Cable Costs
        drop_fibre_cost = self.total_drop_length * params.budget_drop_fibre_per_m
        dist_fibre_cost = self.total_distribution_length * params.budget_dist_fibre_per_m
        feeder_fibre_cost = self.total_feeder_length * params.budget_feeder_fibre_per_m

        # Hardware Costs
        # Cost per manhole structure (based on total *unique* placed manholes)
        manhole_cost = self.total_manholes * params.budget_manhole
        # Cost per splitter installed (assuming one per *used* manhole/splitter location)
        splitter_cost = self.total_splitters * params.budget_splitter_cost
        # Cost per client termination box
        terminal_cost = self.total_clients * params.budget_terminal_box_per_client

        # Splicing Costs (Estimate: termination + splitter connection + feeder tap per client)
        # This is a very rough estimate, assuming 3 splices in path per client
        splicing_cost_estimate = self.total_clients * 3 * params.budget_splicing_per_connection

        self.total_cost = (trench_duct_cost + drop_fibre_cost +
                           dist_fibre_cost + feeder_fibre_cost +
                           manhole_cost + splitter_cost + terminal_cost +
                           splicing_cost_estimate)
        return self.total_cost

    def generate_summary(self, params: NetworkParameters) -> str:
        """Generate a textual summary of the network statistics"""
        self.calculate_costs(params) # Ensure costs are up-to-date
        elapsed_time = time.time() - self.start_time
        total_cable_length = self.total_drop_length + self.total_distribution_length + self.total_feeder_length
        avg_cost_per_client = (self.total_cost / self.total_clients) if self.total_clients > 0 else 0.0

        summary = (
            f"Network Design Summary (v{VERSION}):\n"
            f"================================\n"
            f"Total Clients (Buildings): {self.total_clients}\n"
            f"Total Zones Processed: {self.zones_count} (Failed: {self.failed_zones})\n"
            f"Total Manholes Placed (Unique): {self.total_manholes}\n"
            f"Total Manholes Used (Splitters): {self.total_splitters}\n"
            f"Clients Exceeding Max Drop Length ({params.max_drop_length}m): {self.clients_over_max_drop}\n"
            f"Routes Using Straight-Line Fallback: {self.routing_fallbacks}\n\n"

            f"Cable Metrics:\n"
            f"--------------\n"
            f"Drop Fibre Length: {self.total_drop_length:.1f} m\n"
            f"Distribution Cable Length: {self.total_distribution_length:.1f} m\n"
            f"Feeder Cable Length: {self.total_feeder_length:.1f} m\n"
            f"Total Cable Length: {total_cable_length:.1f} m\n\n"

            f"Cost Estimate (Based on Parameters):\n"
            f"------------------------------------\n"
            f"Total Estimated Cost: {self.total_cost:,.2f}\n"
            f"Avg. Cost per Client: {avg_cost_per_client:,.2f}\n\n"

            f"Processing Time: {elapsed_time:.2f} seconds"
        )
        return summary

@dataclass(slots=True)
class BuildingClient:
    """Represents a client derived from an OSM building"""
    id: int
    geometry: Point  # Centroid of the building polygon (WGS84: EPSG:4326)
    building_id: Union[str, int]  # OSM building ID
    building_type: str = "unknown"
    building_levels: Optional[int] = 1
    address: Dict[str, str] = field(default_factory=dict)

    @classmethod
    def from_osm_building(cls, idx: int, building_geom: Union[Polygon, MultiPolygon, Point], osm_id: Union[str, int], building_tags: Dict = None):
        """Create a client from an OSM building geometry and tags. Returns None if invalid."""
        if building_tags is None:
            building_tags = {}

        centroid: Optional[Point] = None
        try:
            if not building_geom or building_geom.is_empty:
                 logger.debug(f"Building {osm_id} has empty geometry. Skipping.")
                 return None

            if not building_geom.is_valid:
                 fixed_geom = building_geom.buffer(0)
                 if fixed_geom.is_valid and not fixed_geom.is_empty:
                     building_geom = fixed_geom
                 else:
                     logger.warning(f"Building {osm_id} has invalid geometry that couldn't be fixed. Skipping.")
                     return None

            if isinstance(building_geom, (Polygon, MultiPolygon)):
                temp_centroid = building_geom.centroid
                if not temp_centroid or temp_centroid.is_empty or not temp_centroid.is_valid:
                    temp_centroid = building_geom.representative_point()
                centroid = temp_centroid
            elif isinstance(building_geom, Point):
                centroid = building_geom
            else:
                logger.warning(f"Building {osm_id} has unsupported geometry type: {type(building_geom)}. Skipping.")
                return None # Skip this building

            if centroid is None or centroid.is_empty or not centroid.is_valid:
                 logger.warning(f"Could not determine valid centroid for building {osm_id}. Skipping.")
                 return None

        except Exception as e:
             logger.warning(f"Error calculating centroid for building {osm_id}: {e}. Skipping.")
             return None # Skip this building

        # Extract address components if available
        address = {}
        for tag in ['addr:street', 'addr:housenumber', 'addr:city', 'addr:postcode', 'name']:
            if tag in building_tags and building_tags[tag]:
                address[tag.replace('addr:', '')] = str(building_tags[tag]) # Ensure string

        levels_raw = building_tags.get('building:levels')
        levels = 1 # Default
        if levels_raw:
            try:
                # Handle potential ranges or multiple values, take the first number safely
                first_num_str = str(levels_raw).split(';')[0].split('-')[0].strip()
                if first_num_str: # Check if string is not empty after stripping
                     levels = int(float(first_num_str)) # Allow float then convert
                     if levels < 1: levels = 1
            except (ValueError, TypeError):
                levels = 1 # Default if value is not a number or complex string

        return cls(
            id=idx,
            geometry=centroid,
            building_id=osm_id,
            building_type=str(building_tags.get('building', 'unknown')),
            building_levels=levels,
            address=address
        )

    def to_dict(self) -> Dict[str, Any]:
        """Convert client data to dictionary, handling geometry."""
        d = asdict(self)
        # Represent geometry as WKT for simple serialization if needed,
        # but primarily use the geometry object directly.
        # d['geometry_wkt'] = self.geometry.wkt
        d.pop('geometry', None) # Remove shapely object from basic dict
        d['latitude'] = self.geometry.y
        d['longitude'] = self.geometry.x
        return d


# Type Aliases for clarity
NodeId = Union[int, str]
NodePoint = Tuple[NodeId, Point] # OSM Node ID and its Point Geometry (WGS84)
NetworkRoute = Tuple[LineString, float] # Geometry (WGS84) and Length (meters)
GeoDict = Dict[str, Union[Point, LineString, Polygon, Dict[str, Any]]] # For KML/data elements


# ---------------------------
# UTILITY FUNCTIONS: Projection and Measurements
# ---------------------------
# Cache transformers to avoid recreating them repeatedly
_transformer_cache: Dict[Tuple[str, str], Transformer] = {}

def get_transformer(from_crs: Union[str, CRS], to_crs: Union[str, CRS]) -> Transformer:
    """Get or create a pyproj Transformer. Always assumes x, y order (lon, lat)."""
    from_crs_str = from_crs.to_string() if isinstance(from_crs, CRS) else str(from_crs)
    to_crs_str = to_crs.to_string() if isinstance(to_crs, CRS) else str(to_crs)
    key = (from_crs_str, to_crs_str)
    if key not in _transformer_cache:
        try:
            transformer = Transformer.from_crs(from_crs_str, to_crs_str, always_xy=True)
            _transformer_cache[key] = transformer
        except ProjError as e:
             logger.error(f"Failed to create transformer from '{from_crs_str}' to '{to_crs_str}': {e}")
             raise ValueError(f"Projection Error creating transformer: {e}") from e
    return _transformer_cache[key]

def get_utm_crs(lat: float, lon: float) -> CRS:
    """Determine the UTM CRS for a given latitude and longitude."""
    if not (-80 <= lat <= 84):
        # Outside standard UTM range, potentially use UPS or other projection
        logger.warning(f"Latitude {lat} is outside the standard UTM range (-80 to 84). Projection accuracy may be affected.")
        # Simple UPS fallback
        if lat > 84: return CRS.from_epsg(32661) # UPS North
        if lat < -80: return CRS.from_epsg(32761) # UPS South
        # Should not happen if lat is number, but handles boundary case
        raise ValueError(f"Latitude {lat} is outside supported projection range.")

    utm_zone = int(math.floor((lon + 180) / 6) + 1)
    epsg_code = 32600 + utm_zone if lat >= 0 else 32700 + utm_zone
    try:
        return CRS.from_epsg(epsg_code)
    except ProjError as e:
        logger.error(f"Could not get UTM CRS for EPSG:{epsg_code} (Lat: {lat}, Lon: {lon}): {e}")
        # Fallback to a global projection like Web Mercator (EPSG:3857) might be an option,
        # but lengths will be inaccurate. Raising error is safer for distance-critical app.
        raise ValueError(f"Could not determine valid UTM CRS from EPSG code {epsg_code}: {e}") from e


def safe_transform(transformer: Transformer, geom: Union[Point, LineString, Polygon]) -> Union[Point, LineString, Polygon, None]:
    """Apply shapely transform with error handling for invalid coordinates."""
    try:
         return transform(transformer.transform, geom)
    except Exception as e:
        # This can catch errors if coordinates are inf/nan or out of bounds for the projection
         logger.warning(f"Coordinate transformation failed: {e}. Geometry (WKT): {geom.wkt[:100]}...")
         return None # Return None to indicate transformation failure

def to_utm(geom: Union[Point, LineString, Polygon]) -> Tuple[Optional[Union[Point, LineString, Polygon]], Optional[CRS]]:
    """Convert a geometry (assumed EPSG:4326) to its appropriate UTM CRS. Returns (None, None) on failure."""
    if not geom or geom.is_empty:
        logger.debug("Cannot project empty geometry.")
        return None, None

    if not geom.is_valid:
         logger.debug(f"Input geometry is invalid, attempting to fix: {geom.wkt[:100]}...")
         geom = geom.buffer(0) # Attempt to fix validity
         if not geom or geom.is_empty or not geom.is_valid:
              logger.warning(f"Cannot project invalid or empty geometry after attempting fix: {geom.wkt[:100]}...")
              return None, None

    try:
        # Use representative point for Polygons/LineStrings to determine UTM zone
        ref_point = geom.representative_point() if hasattr(geom, 'representative_point') else geom
        if not ref_point or ref_point.is_empty:
             ref_point = geom.centroid if hasattr(geom, 'centroid') and geom.centroid and not geom.centroid.is_empty else None
             if not ref_point and hasattr(geom, 'coords') and len(geom.coords) > 0:
                  ref_point = Point(geom.coords[0])

        if not ref_point or ref_point.is_empty:
             logger.warning(f"Could not determine reference point for UTM projection: {geom.wkt[:100]}...")
             return None, None

        utm_crs = get_utm_crs(ref_point.y, ref_point.x)
        transformer = get_transformer("EPSG:4326", utm_crs)
        projected_geom = safe_transform(transformer, geom)
        if projected_geom is None:
             logger.error(f"Failed to project geometry to UTM due to coordinate error: {geom.wkt[:100]}...")
             return None, None

        return projected_geom, utm_crs
    except (ValueError, ProjError, IndexError, Exception) as e:
        logger.error(f"Failed during UTM projection setup or execution: {e}. Geometry: {geom.wkt[:100]}...")
        return None, None

def to_wgs84(geom: Union[Point, LineString, Polygon], source_crs: CRS) -> Optional[Union[Point, LineString, Polygon]]:
    """Convert a geometry from a given source CRS back to EPSG:4326. Returns None on failure."""
    if not geom or geom.is_empty:
        logger.debug("Cannot project empty geometry to WGS84.")
        return None
    if not source_crs:
         logger.error("Cannot project to WGS84 without a valid source CRS.")
         return None

    if not geom.is_valid:
         logger.debug(f"Input geometry for WGS84 projection is invalid, attempting to fix: {geom.wkt[:100]}...")
         geom = geom.buffer(0)
         if not geom or geom.is_empty or not geom.is_valid:
              logger.warning(f"Cannot project invalid or empty geometry to WGS84 after attempting fix: {geom.wkt[:100]}...")
              return None

    try:
        transformer = get_transformer(source_crs, "EPSG:4326")
        wgs84_geom = safe_transform(transformer, geom)
        if wgs84_geom is None:
             logger.error(f"Failed to project geometry from {source_crs.to_string()} to WGS84 due to coordinate error: {geom.wkt[:100]}...")
             return None
        return wgs84_geom
    except (ValueError, ProjError, Exception) as e:
         logger.error(f"Failed during WGS84 projection setup or execution: {e}. Geometry: {geom.wkt[:100]}...")
         return None

def get_length_in_meters(geom: LineString) -> float:
    """Return the length of a LineString in meters (using appropriate UTM projection). Returns 0.0 on failure."""
    if not geom or geom.is_empty or not isinstance(geom, LineString): return 0.0
    try:
        geom_utm, _ = to_utm(geom)
        if geom_utm:
             return geom_utm.length
        else:
             logger.warning(f"Could not calculate UTM length for geometry due to projection issue. Returning 0.")
             return 0.0
    except Exception as e:
        logger.warning(f"Unexpected error calculating UTM length: {e}. Geometry: {geom.wkt[:100]}... Returning 0.")
        return 0.0

def measure_distance_utm(point1: Point, point2: Point) -> float:
    """Measure the Cartesian distance between two points in meters using UTM. Falls back to geodesic."""
    if not point1 or point1.is_empty or not point2 or point2.is_empty: return 0.0
    fallback_dist = ox.distance.great_circle(point1.y, point1.x, point2.y, point2.x)

    try:
        # Project both points to the UTM zone of the first point
        point1_utm, utm_crs = to_utm(point1)
        if not point1_utm or not utm_crs:
             raise ValueError("Projection failed for point1")

        transformer_to_utm = get_transformer("EPSG:4326", utm_crs)
        point2_utm = safe_transform(transformer_to_utm, point2)
        if not point2_utm:
             raise ValueError("Projection failed for point2")

        return point1_utm.distance(point2_utm)
    except (ValueError, ProjError, Exception) as e:
         logger.warning(f"Could not measure UTM distance between points: {e}. Using geodesic ({fallback_dist:.1f}m).")
         return fallback_dist


# ---------------------------
# OSM Data Extraction Functions with Enhanced Error Handling
# ---------------------------
def _retry_api_call(func, *args, **kwargs):
    """Wrapper to retry a function call with exponential backoff."""
    retry_delay = API_RETRY_DELAY
    last_exception = None
    for attempt in range(MAX_API_RETRIES):
        try:
            result = func(*args, **kwargs)
            return result # Success
        except Exception as e:
            last_exception = e
            logger.warning(f"{func.__name__} call attempt {attempt + 1}/{MAX_API_RETRIES} failed: {e}")
            if attempt < MAX_API_RETRIES - 1:
                logger.info(f"Retrying in {retry_delay} seconds...")
                time.sleep(retry_delay)
                retry_delay *= 2 # Exponential backoff
            else:
                logger.error(f"{func.__name__} failed after {MAX_API_RETRIES} attempts.")
                raise last_exception # Re-raise the last exception after all retries fail

def get_polygon_from_place(place_name: str) -> Polygon:
    """Geocode a place name via OSM and return its boundary polygon with retry logic."""
    logger.info(f"Geocoding '{place_name}'...")
    ox.settings.log_console=False # Reduce osmnx verbosity
    ox.settings.timeout = OSM_TIMEOUT

    def geocode_action():
        # which_result=1 picks the first (usually best) result if multiple are found
        gdf = ox.geocode_to_gdf(place_name, which_result=1)
        if gdf.empty:
            raise ValueError(f"Location '{place_name}' not found by geocoder (result empty).")
        geom = gdf.iloc[0].geometry
        return geom

    try:
        geom = _retry_api_call(geocode_action)

        if not geom.is_valid:
             logger.warning(f"Geocoded geometry for '{place_name}' is invalid, attempting buffer(0) fix.")
             geom = geom.buffer(0)
             if not geom.is_valid or geom.is_empty:
                 raise ValueError(f"Geocoded geometry for '{place_name}' is invalid and could not be fixed.")

        # Ensure we have a Polygon
        if isinstance(geom, Polygon):
            logger.info(f"Successfully geocoded '{place_name}' to a Polygon.")
            return geom
        elif isinstance(geom, MultiPolygon):
            logger.warning(f"'{place_name}' returned a MultiPolygon. Using the largest valid sub-polygon by area.")
            valid_polygons = [p for p in geom.geoms if p.is_valid and not p.is_empty]
            if not valid_polygons:
                raise ValueError(f"MultiPolygon for '{place_name}' contains no valid sub-polygons.")
            largest_poly = max(valid_polygons, key=lambda p: p.area)
            logger.info(f"Selected largest polygon from MultiPolygon for '{place_name}'.")
            return largest_poly
        else:  # Point or LineString - Try buffering
            logger.warning(f"'{place_name}' geocoded to a {geom.geom_type}. Buffering ~1km to create an area.")
            geom_utm, utm_crs = to_utm(geom)
            if not geom_utm or not utm_crs:
                 raise ValueError(f"Could not project {geom.geom_type} to UTM for buffering.")
            buffered_utm = geom_utm.buffer(1000) # Buffer by 1km in meters
            buffered_wgs84 = to_wgs84(buffered_utm, utm_crs)
            if buffered_wgs84 and isinstance(buffered_wgs84, Polygon):
                 logger.info(f"Successfully created Polygon by buffering {geom.geom_type} for '{place_name}'.")
                 return buffered_wgs84
            else:
                 raise ValueError(f"Could not create a valid Polygon by buffering the {geom.geom_type} for '{place_name}'.")

    except Exception as e:
         logger.critical(f"Failed to get valid polygon for place '{place_name}': {e}")
         raise ValueError(f"Could not geocode or process place '{place_name}': {e}") from e


def retrieve_osm_data(polygon: Polygon, network_type="drive") -> nx.MultiDiGraph:
    """Retrieve the OSM road network graph within a given polygon with retry logic."""
    logger.info(f"Retrieving OSM road network for the specified area (polygon buffer: {GRAPH_BUFFER_DEGREES} degrees)...")
    ox.settings.log_console=False
    ox.settings.timeout = OSM_TIMEOUT

    # Use a slightly buffered polygon for graph retrieval to ensure connectivity at zone edges
    graph_query_polygon = polygon.buffer(GRAPH_BUFFER_DEGREES)

    def graph_action():
        G = ox.graph_from_polygon(graph_query_polygon,
                                  network_type=network_type,
                                  simplify=True,        # Simplifies topology, critical for routing
                                  retain_all=False,     # Keep only largest connected component implicitly? Use strong connectivity check later if needed.
                                  truncate_by_edge=True,# Use edges strictly within polygon
                                  ) # Remove nodes outside polygon boundary after truncation

        # Post-retrieval checks
        if G is None or G.number_of_nodes() == 0:
            raise ValueError("Retrieved network graph is empty or invalid (0 nodes). Check area polygon and OSM data availability.")
        if G.number_of_edges() == 0:
             logger.warning(f"Retrieved network graph has {G.number_of_nodes()} nodes but 0 edges. This might cause routing issues.")
             # Allow proceeding but log clearly

        # Ensure graph is strongly connected if needed, or handle disconnected components during routing
        # For simplicity here, we rely on osmnx defaults and handle routing errors later
        # if not nx.is_strongly_connected(G): # Check for directed graphs
        #     logger.warning("Retrieved graph is not strongly connected. Routing may fail between some nodes.")

        logger.info(f"Retrieved network graph with {G.number_of_nodes()} nodes and {G.number_of_edges()} edges.")

        # Add edge lengths (meters) - essential for routing weight
        G_proj = ox.project_graph(G) # Project to local UTM for accurate lengths
        
        # Transfer lengths back to original graph (WGS84)
        edge_lengths = {(u, v, k): data['length'] for u, v, k, data in G_proj.edges(keys=True, data=True)}
        nx.set_edge_attributes(G, values=edge_lengths, name='length')
        logger.info("Added 'length' attribute (meters) to graph edges.")
        return G

    try:
        G = _retry_api_call(graph_action)
        return G
    except Exception as e:
        logger.error(f"Failed to retrieve or process OSM network graph: {e}")
        # Decide whether to raise or return None/empty graph
        # Raising is generally better to signal failure clearly
        raise ValueError(f"Could not retrieve a valid OSM network graph: {e}") from e


def get_building_clients(polygon: Polygon) -> List[BuildingClient]:
    """Obtain REAL building footprints from OSM for the given polygon."""
    logger.info("Querying OSM for building footprints within the zone...")
    tags = {'building': True} # Get all features tagged as building=*
    ox.settings.log_console=False
    ox.settings.timeout = OSM_TIMEOUT

    def features_action():
        # features_from_polygon can return Points, Lines, Polygons depending on OSM mapping
        buildings_gdf = ox.features_from_polygon(polygon, tags=tags)
        return buildings_gdf

    try:
        buildings_gdf = _retry_api_call(features_action)

        if buildings_gdf.empty:
            logger.warning("No building features found in the specified area.")
            return [] # Return empty list if no buildings found

        logger.info(f"Found {len(buildings_gdf)} raw building features in OSM.")
        client_list: List[BuildingClient] = []
        processed_osm_ids = set() # Track processed OSM IDs (especially for relation members)
        added_centroids = set() # Track unique centroids (rounded coords) to avoid duplicates

        # Iterate through GeoDataFrame rows (index is often OSM ID or tuple)
        for index, row in buildings_gdf.iterrows():
            # Extract OSM ID robustly from index (can be single int/str or tuple)
            if isinstance(index, tuple):
                 osm_id = index[0] # Typically (element_type, osm_id) -> use osm_id
                 # Could add check here if osm_id is numeric?
            else:
                 osm_id = index

            if osm_id in processed_osm_ids: continue
            processed_osm_ids.add(osm_id)

            try:
                geom = row.geometry # Get the geometry for this feature
                # Attempt to create a BuildingClient object from the feature
                client = BuildingClient.from_osm_building(
                    idx=len(client_list) + 1, # Sequential internal ID
                    building_geom=geom,
                    osm_id=osm_id,
                    building_tags=row.to_dict() # Pass all tags/attributes
                )

                if client:
                    # Check for duplicate centroids before adding (handle minor floating point diffs)
                    centroid_coords = (round(client.geometry.x, 6), round(client.geometry.y, 6))
                    if centroid_coords not in added_centroids:
                        client_list.append(client)
                        added_centroids.add(centroid_coords)
                    # else: logger.debug(f"Skipping duplicate client centroid at {centroid_coords} for OSM ID {osm_id}")

            except Exception as e:
                # Log error but continue processing other buildings
                logger.warning(f"Error processing building feature OSM ID {osm_id}: {e}. Skipping.")
                continue

        logger.info(f"Successfully created {len(client_list)} unique BuildingClient objects from OSM features.")
        return client_list

    except Exception as e:
        logger.error(f"Failed to retrieve or process building data: {e}")
        # Don't raise error, allow process to continue if possible, return empty list
        return []


# ---------------------------
# NETWORK INFRASTRUCTURE FUNCTIONS
# ---------------------------
def generate_duct_banks(G: nx.MultiDiGraph, params: NetworkParameters) -> List[GeoDict]:
    """
    Generate duct bank features from the network graph edges.
    Each edge represents a potential duct route following the road geometry.
    Returns a list of dictionaries, each containing geometry and properties.
    """
    duct_banks: List[GeoDict] = []
    if not G or G.number_of_edges() == 0:
         logger.warning("Input graph is empty or has no edges. Cannot generate duct banks.")
         return []

    logger.info(f"Generating duct bank information from {G.number_of_edges()} graph edges...")
    processed_edge_keys = set() # Handle parallel edges between nodes u,v with different keys

    for u, v, key, data in G.edges(keys=True, data=True):
        # Ensure unique processing for each distinct edge (u, v, key)
        edge_repr = (u, v, key)
        if edge_repr in processed_edge_keys:
            continue
        processed_edge_keys.add(edge_repr)

        geom = data.get('geometry')
        length_m = data.get('length') # Use pre-calculated length if available

        # Ensure we have a valid LineString geometry and calculate length if needed
        if isinstance(geom, LineString) and not geom.is_empty and geom.is_valid:
            if length_m is None or length_m <= 0: # Recalculate length if missing/invalid
                 length_m = get_length_in_meters(geom)
                 # Store calculated length back in graph data for potential reuse if needed later
                 # G.edges[u, v, key]['length'] = length_m # Modifying graph during iteration? Risky. Better to just use calculated value.
        else:
             # Fallback: create straight line between nodes if no valid geometry
             try:
                 pt1 = Point(G.nodes[u]['x'], G.nodes[u]['y'])
                 pt2 = Point(G.nodes[v]['x'], G.nodes[v]['y'])
                 geom = LineString([pt1, pt2])
                 if not geom.is_empty and geom.is_valid:
                      length_m = get_length_in_meters(geom)
                      logger.debug(f"Created fallback straight-line geometry for duct {u}-{v}-{key}.")
                 else: raise ValueError("Fallback line invalid")
             except Exception as e:
                 logger.warning(f"Could not create valid geometry for edge {u}-{v}-{key}: {e}. Skipping duct.")
                 continue # Skip this edge if geometry fails completely

        # Ensure length is a non-negative float
        length_m = max(0.0, float(length_m if length_m is not None else 0.0))

        # Basic properties from OSM edge data and parameters
        props = {
            "id": f"Duct_{u}-{v}-{key}",
            "name": f"Duct Bank {u}-{v}",
            "osm_id": data.get('osmid', f"generated_{u}-{v}-{key}"),
            "length_m": length_m,
            "road_name": data.get('name', 'Unnamed Road'),
            "road_type": data.get('highway', 'unknown'),
            "cost_per_meter": params.budget_trench_per_m, # Using combined cost
            "total_cost": length_m * params.budget_trench_per_m,
            "graph_edge": (u, v, key) # Store original graph edge identifier
        }
        duct_banks.append({"geometry": geom, "properties": props})

    logger.info(f"Generated {len(duct_banks)} duct bank segments.")
    return duct_banks

def find_nearest_node_point(G: nx.MultiDiGraph, point: Point) -> Optional[NodePoint]:
    """Find the nearest graph node ID and its Point geometry to a given Point."""
    if not G or G.number_of_nodes() == 0:
        logger.error("Graph has no nodes, cannot find nearest node.")
        return None
    if not point or point.is_empty or not point.is_valid:
         logger.error(f"Input point is invalid or empty ({point}), cannot find nearest node.")
         return None
    try:
        # osmnx expects lon, lat order (X, Y) for nearest_nodes
        node_id = ox.nearest_nodes(G, X=point.x, Y=point.y)
        # Ensure node_id is not a list/array if only one point was passed
        if isinstance(node_id, (list, np.ndarray)):
            node_id = node_id[0]

        node_data = G.nodes[node_id]
        node_geom = Point(node_data['x'], node_data['y'])
        return node_id, node_geom
    except Exception as e:
        # Catch potential errors if point is far outside graph bounds or other issues
        logger.error(f"Error finding nearest node to point {point.wkt}: {e}")
        return None


def optimize_manhole_placement(G: nx.MultiDiGraph, client_points: List[BuildingClient], params: NetworkParameters) -> List[NodePoint]:
    """
    Optimize manhole placement using clustering (DBSCAN) on potential locations,
    aligned with network nodes. Returns list of chosen (node_id, Point_geometry) tuples.

    Note: DBSCAN uses geographic proximity. Nodes close geographically might be far apart
    via the network. This method prioritizes reducing digging near intersections and
    client clusters, but doesn't guarantee network path length optimization from manholes.
    """
    if not client_points:
        logger.warning("No client points provided for manhole optimization. No manholes placed.")
        return []
    if not G or G.number_of_nodes() < 2:
        logger.warning(f"Network graph is too small ({G.number_of_nodes()} nodes) for meaningful manhole placement. No manholes placed.")
        return []

    potential_nodes: Set[NodeId] = set()

    # 1. Intersections (degree > 2 in the undirected view)
    # Use undirected view to count connections properly at intersections
    try:
        # Need to handle potential errors if graph structure is unusual
        G_undirected = G.to_undirected(as_view=True)
        intersections = [node for node, degree in G_undirected.degree() if degree > 2]
        potential_nodes.update(intersections)
        logger.info(f"Identified {len(intersections)} intersection nodes as potential manhole locations.")
    except Exception as e:
         logger.warning(f"Could not identify intersection nodes due to graph error: {e}. Proceeding without them.")

    # 2. Nodes closest to each client
    client_geoms = [c.geometry for c in client_points if c.geometry and not c.geometry.is_empty and c.geometry.is_valid]
    if client_geoms:
        client_lons = [p.x for p in client_geoms]
        client_lats = [p.y for p in client_geoms]
        try:
            nearest_client_nodes = ox.nearest_nodes(G, X=client_lons, Y=client_lats)
            # nearest_nodes returns single value if one point, list if multiple
            if isinstance(nearest_client_nodes, (int, str)): nearest_client_nodes = [nearest_client_nodes]

            added_count = len(set(nearest_client_nodes) - potential_nodes)
            potential_nodes.update(nearest_client_nodes)
            logger.info(f"Added {added_count} unique nodes closest to clients as potential locations.")
        except Exception as e:
            logger.error(f"Error finding nearest nodes for clients: {e}. Manhole placement might be suboptimal.")
            # Continue without client-nearest nodes if this fails
    else:
         logger.warning("No valid client geometries available to find nearest nodes for manhole placement.")


    if not potential_nodes:
         logger.warning("No potential manhole locations identified (intersections or near clients). Cannot place manholes.")
         return []

    # Validate potential nodes exist in the graph and get their coords
    valid_potential_nodes: List[NodeId] = []
    node_coords_wgs = []
    for nid in potential_nodes:
         try:
             node_data = G.nodes[nid]
             node_coords_wgs.append((node_data['x'], node_data['y']))
             valid_potential_nodes.append(nid)
         except KeyError:
             logger.warning(f"Potential manhole Node ID {nid} not found in graph during validation. Skipping.")
             continue

    if not valid_potential_nodes:
         logger.warning("No valid potential manhole nodes remained after graph validation. Cannot place manholes.")
         return []


    # If optimization disabled, too few candidates, or clustering fails, use all valid potential nodes
    if not params.optimize_manholes or len(valid_potential_nodes) < MIN_NODES_FOR_CLUSTERING:
        if not params.optimize_manholes:
             logger.info(f"Manhole optimization disabled. Using all {len(valid_potential_nodes)} potential nodes.")
        else:
             logger.info(f"Too few ({len(valid_potential_nodes)}) potential nodes to cluster (min: {MIN_NODES_FOR_CLUSTERING}). Using all.")
        # Return list of (NodeId, Point) for all valid candidates
        manhole_points = []
        for i, nid in enumerate(valid_potential_nodes):
             lon, lat = node_coords_wgs[i]
             manhole_points.append((nid, Point(lon, lat)))
        return manhole_points


    # --- Proceed with Clustering Optimization (DBSCAN) ---
    logger.info(f"Optimizing manhole placement for {len(valid_potential_nodes)} candidate nodes using DBSCAN...")
    node_coords_wgs_arr = np.array(node_coords_wgs)

    # Convert potential node coordinates to UTM for distance-based clustering
    try:
        # Use centroid of potential nodes for UTM zone determination
        rep_lon, rep_lat = np.mean(node_coords_wgs_arr, axis=0)
        utm_crs = get_utm_crs(rep_lat, rep_lon)
        transformer_to_utm = get_transformer("EPSG:4326", utm_crs)
        # Apply transformation robustly using list comprehension with safe_transform
        utm_coords_list = [safe_transform(transformer_to_utm, Point(lon, lat)) for lon, lat in node_coords_wgs_arr]
        # Filter out None results from failed transformations
        valid_utm_points = [p for p in utm_coords_list if p is not None]
        if len(valid_utm_points) < len(valid_potential_nodes):
             logger.warning(f"Could not project all potential nodes to UTM ({len(valid_potential_nodes) - len(valid_utm_points)} failures). Clustering on available points.")
        if len(valid_utm_points) < MIN_NODES_FOR_CLUSTERING:
             raise ValueError(f"Too few points ({len(valid_utm_points)}) remained after UTM projection for clustering.")
        # Rebuild node list and WGS coords corresponding to successfully projected points
        valid_indices = [i for i, p in enumerate(utm_coords_list) if p is not None]
        valid_potential_nodes = [valid_potential_nodes[i] for i in valid_indices]
        node_coords_wgs = [node_coords_wgs[i] for i in valid_indices]
        # Convert list of Points to numpy array for DBSCAN
        utm_coords = np.array([(p.x, p.y) for p in valid_utm_points])

    except (ValueError, ProjError, Exception) as e:
        logger.error(f"Failed to prepare coordinates for DBSCAN clustering: {e}. Aborting optimization, using all valid candidates.")
        manhole_points = [(nid, Point(coords[0], coords[1])) for nid, coords in zip(valid_potential_nodes, node_coords_wgs)]
        return manhole_points

    # Perform DBSCAN clustering
    logger.info(f"Clustering {len(valid_potential_nodes)} node locations (UTM) using DBSCAN (eps={params.manhole_clustering_distance}m)...")
    try:
        # min_samples=1 ensures every point is included (no noise points classified as -1)
        clustering = DBSCAN(eps=params.manhole_clustering_distance, min_samples=1, metric='euclidean', algorithm='auto').fit(utm_coords)
        labels = clustering.labels_
        unique_labels = set(labels)
        num_clusters = len(unique_labels)
        logger.info(f"DBSCAN identified {num_clusters} clusters.")

        final_manholes: List[NodePoint] = []
        for k in unique_labels:
            # Find indices of nodes belonging to this cluster
            cluster_member_mask = (labels == k)
            cluster_indices = np.where(cluster_member_mask)[0] # Indices within the utm_coords array

            if len(cluster_indices) == 0: continue

            # Get the UTM coordinates and original node IDs for this cluster
            cluster_utm_coords = utm_coords[cluster_indices]
            cluster_node_ids = [valid_potential_nodes[i] for i in cluster_indices]

            # Choose representative node for the cluster: Node closest to the cluster's geographic centroid (in UTM)
            if len(cluster_indices) == 1:
                 # If only one node in cluster, it's the representative
                 chosen_node_idx = cluster_indices[0]
            else:
                 # Calculate UTM centroid of the cluster
                 cluster_centroid_utm = np.mean(cluster_utm_coords, axis=0)
                 # Find the node within the cluster that is closest to the UTM centroid
                 distances_to_centroid = np.linalg.norm(cluster_utm_coords - cluster_centroid_utm, axis=1)
                 closest_node_local_idx = np.argmin(distances_to_centroid)
                 chosen_node_idx = cluster_indices[closest_node_local_idx] # Index in the original valid_potential_nodes list

            # Get the chosen node ID and its WGS84 geometry
            chosen_node_id = valid_potential_nodes[chosen_node_idx]
            chosen_node_coords_wgs = node_coords_wgs[chosen_node_idx]
            chosen_node_geom = Point(chosen_node_coords_wgs[0], chosen_node_coords_wgs[1])

            final_manholes.append((chosen_node_id, chosen_node_geom))

        logger.info(f"Optimized manhole placement: Selected {len(final_manholes)} locations from {len(utm_coords)} clustered candidates.")
        return final_manholes

    except Exception as e:
        logger.warning(f"Manhole optimization using DBSCAN failed: {e}. Falling back to using all {len(valid_potential_nodes)} valid potential nodes.")
        manhole_points = [(nid, Point(coords[0], coords[1])) for nid, coords in zip(valid_potential_nodes, node_coords_wgs)]
        return manhole_points


def generate_aggregation_node(G: nx.MultiDiGraph, client_points: List[BuildingClient], params: NetworkParameters) -> Optional[NodePoint]:
    """
    Compute the overall aggregation node for a zone.
    Prioritizes high-centrality nodes (optional) near client density, aligning with a graph node.
    Returns a tuple: (node_id, Point_geometry) or None if fails.
    """
    if not client_points:
        logger.warning("Cannot determine aggregation node: No clients provided.")
        return None
    if not G or G.number_of_nodes() < 2 :
        logger.warning(f"Cannot determine aggregation node: Insufficient graph nodes ({G.number_of_nodes()}).")
        return None

    # 1. Calculate Client Centroid
    valid_client_geoms = [c.geometry for c in client_points if c.geometry and not c.geometry.is_empty and c.geometry.is_valid]
    if not valid_client_geoms:
         logger.warning("No valid client geometries to calculate centroid for aggregation node selection.")
         # Fallback: Use geometric center of graph
         try:
             logger.info("Using geometric center of graph nodes as fallback centroid.")
             node_coords = np.array([[data['x'], data['y']] for _, data in G.nodes(data=True) if 'x' in data and 'y' in data])
             if node_coords.shape[0] == 0: raise ValueError("No nodes with coordinates found")
             center_lon, center_lat = np.mean(node_coords, axis=0)
             client_centroid = Point(center_lon, center_lat)
         except Exception as center_e:
              logger.error(f"Could not calculate graph center either: {center_e}. Cannot determine aggregation node.")
              return None
    else:
         try:
             multi_client_point = MultiPoint(valid_client_geoms)
             client_centroid = multi_client_point.centroid
             if not client_centroid or client_centroid.is_empty or not client_centroid.is_valid:
                  raise ValueError("Calculated client centroid is invalid or empty.")
         except Exception as e:
             logger.warning(f"Could not calculate client centroid from {len(valid_client_geoms)} points: {e}. Using first client location as fallback.")
             client_centroid = valid_client_geoms[0] # Fallback to first valid client

    # 2. Find graph node nearest to the calculated centroid
    centroid_nearest_nodepoint = find_nearest_node_point(G, client_centroid)
    if not centroid_nearest_nodepoint:
        logger.error(f"Could not find nearest node to client cluster centroid {client_centroid.wkt}. Cannot determine aggregation node.")
        # Try fallback: Find nearest node to the first client geometry
        if valid_client_geoms:
             fallback_nodepoint = find_nearest_node_point(G, valid_client_geoms[0])
             if fallback_nodepoint:
                  logger.warning("Using node nearest to first client as fallback for aggregation node.")
                  centroid_nearest_nodepoint = fallback_nodepoint
             else:
                  logger.error("Also failed to find nearest node to first client. Cannot select aggregation node.")
                  return None
        else: # No valid clients at all
             logger.error("No valid clients to fallback on. Cannot select aggregation node.")
             return None

    # If not using centrality, return the node nearest the centroid
    if not params.use_centrality_for_aggregation:
         logger.info(f"Centrality calculation disabled. Selected aggregation node based on proximity to client centroid: Node {centroid_nearest_nodepoint[0]}")
         return centroid_nearest_nodepoint

    # Check graph size before attempting centrality
    if G.number_of_nodes() > MAX_NODES_FOR_CENTRALITY_CALC:
        logger.warning(f"Graph size ({G.number_of_nodes()} nodes) exceeds threshold ({MAX_NODES_FOR_CENTRALITY_CALC}) for centrality calculation. Skipping for performance.")
        logger.info(f"Selected aggregation node based solely on proximity to client centroid: Node {centroid_nearest_nodepoint[0]}")
        return centroid_nearest_nodepoint

    # 3. Consider network centrality (Betweenness Centrality)
    try:
        logger.info("Calculating node betweenness centrality (weighted by length)...")
        start_centrality = time.time()
        # Use weight='length' for realistic centrality based on travel distance/cost
        # Ensure length attribute exists and is numeric
        if not nx.get_edge_attributes(G, 'length'):
             logger.warning("Graph edges missing 'length' attribute required for weighted centrality. Falling back to unweighted.")
             centrality = nx.betweenness_centrality(G, normalized=True, k=None) # k=None for exact centrality
        else:
             # Check for non-numeric lengths (should have been handled, but check defensively)
             has_invalid_length = any(not isinstance(l, (int, float)) for u,v,l in G.edges(data='length') if l is not None)
             if has_invalid_length:
                  logger.warning("Graph contains non-numeric 'length' attributes. Falling back to unweighted centrality.")
                  centrality = nx.betweenness_centrality(G, normalized=True, k=None)
             else:
                  # Use weighted centrality
                  centrality = nx.betweenness_centrality(G, weight='length', normalized=True, k=None)

        logger.info(f"Centrality calculation took {time.time() - start_centrality:.2f} seconds.")

        if not centrality: raise ValueError("Centrality calculation returned empty results.")

        # Find nodes within a certain distance of the client centroid and rank them by centrality
        candidate_nodes = []
        # Pre-calculate UTM projection for centroid to speed up distance checks
        centroid_utm, centroid_utm_crs = to_utm(client_centroid)
        if not centroid_utm or not centroid_utm_crs:
             logger.warning("Could not project client centroid to UTM for centrality distance check. Using globally highest centrality node.")
             highest_centrality_node_id = max(centrality, key=centrality.get)
             node_geom = Point(G.nodes[highest_centrality_node_id]['x'], G.nodes[highest_centrality_node_id]['y'])
             logger.info(f"Selected globally highest centrality node {highest_centrality_node_id} as aggregation node.")
             return highest_centrality_node_id, node_geom

        transformer_to_utm = get_transformer("EPSG:4326", centroid_utm_crs)

        for node_id, bc_score in centrality.items():
            try:
                node_geom = Point(G.nodes[node_id]['x'], G.nodes[node_id]['y'])
                node_geom_utm = safe_transform(transformer_to_utm, node_geom)
                if not node_geom_utm: continue # Skip if projection failed

                dist_m = centroid_utm.distance(node_geom_utm)
                if dist_m <= MAX_DIST_M_FROM_CENTROID_FOR_AGG_NODE:
                    candidate_nodes.append((node_id, bc_score))
            except KeyError:
                 logger.warning(f"Node ID {node_id} from centrality results not found in graph. Skipping.")
                 continue
            except Exception as dist_calc_e:
                 logger.warning(f"Error calculating distance or processing node {node_id} during centrality check: {dist_calc_e}")
                 continue # Skip this node if error occurs

        chosen_node_id = None
        if candidate_nodes:
             # Sort candidates by centrality (descending)
             candidate_nodes.sort(key=lambda x: x[1], reverse=True)
             chosen_node_id = candidate_nodes[0][0]
             logger.info(f"Selected aggregation node {chosen_node_id} based on highest centrality within {MAX_DIST_M_FROM_CENTROID_FOR_AGG_NODE}m of client centroid.")
        else:
             # Fallback: If no central nodes near centroid, use the node closest to centroid found earlier
             logger.warning(f"No high-centrality nodes found near client centroid. Using node closest to centroid: {centroid_nearest_nodepoint[0]}.")
             chosen_node_id = centroid_nearest_nodepoint[0] # Use the ID from the nearest node found previously

        # Get the geometry for the chosen node
        chosen_node_geom = Point(G.nodes[chosen_node_id]['x'], G.nodes[chosen_node_id]['y'])
        return chosen_node_id, chosen_node_geom

    except Exception as e:
        logger.warning(f"Error calculating or using network centrality: {e}. Falling back to node nearest client centroid ({centroid_nearest_nodepoint[0]}).")
        return centroid_nearest_nodepoint


# ---------------------------
# CABLE ROUTING FUNCTIONS
# ---------------------------

def compute_drop_fibre(client: BuildingClient, duct_banks: List[GeoDict], params: NetworkParameters) -> Tuple[Optional[GeoDict], Optional[Point], Optional[GeoDict], bool]:
    """
    Compute straight-line drop fibre from client to the nearest point on any duct bank.
    Uses UTM projection for accurate distance measurement.

    Returns:
        Tuple containing:
        - Drop fibre dictionary (GeoDict) or None if failed.
        - Connection point on the duct (Point WGS84) or None.
        - The chosen duct bank dictionary (GeoDict) or None.
        - Boolean flag indicating if drop length exceeds maximum.
    """
    best_point_on_duct_wgs: Optional[Point] = None
    best_dist_m: float = float('inf')
    best_duct: Optional[GeoDict] = None
    exceeds_max_len: bool = False

    if not client or client.geometry is None or client.geometry.is_empty:
        logger.warning(f"Cannot compute drop fibre for invalid client (ID: {client.id if client else 'N/A'}).")
        return None, None, None, False
    if not duct_banks:
         logger.warning(f"Client {client.id}: No duct banks provided to connect to. Cannot compute drop fibre.")
         return None, None, None, False

    # Project client geometry once
    client_utm, utm_crs = to_utm(client.geometry)
    if not client_utm or not utm_crs:
        logger.error(f"Failed to project client {client.id} to UTM: {client.geometry.wkt[:100]}... Cannot compute drop fibre.")
        return None, None, None, False
    transformer_to_utm = get_transformer("EPSG:4326", utm_crs)
    transformer_to_wgs = get_transformer(utm_crs, "EPSG:4326")

    for duct in duct_banks:
        duct_geom_wgs = duct.get("geometry")
        duct_props = duct.get("properties", {})
        duct_id = duct_props.get("id", "Unknown Duct")

        if not isinstance(duct_geom_wgs, LineString) or duct_geom_wgs.is_empty or not duct_geom_wgs.is_valid:
            logger.debug(f"Client {client.id}: Skipping invalid or non-LineString duct geometry for {duct_id}.")
            continue

        try:
            # Project duct to the *client's* UTM zone for consistent distance comparison
            duct_geom_utm = safe_transform(transformer_to_utm, duct_geom_wgs)
            if not duct_geom_utm or duct_geom_utm.is_empty or not isinstance(duct_geom_utm, LineString):
                 logger.warning(f"Client {client.id}: Failed to project or resulted in invalid geometry for duct {duct_id}. Skipping.")
                 continue

            # Find nearest point on this projected duct segment using project/interpolate
            dist_along_line_utm = duct_geom_utm.project(client_utm)
            point_on_line_utm = duct_geom_utm.interpolate(dist_along_line_utm)

            # Calculate Cartesian distance in UTM
            distance_m = client_utm.distance(point_on_line_utm)

            if distance_m < best_dist_m:
                # Convert the potentially nearest point back to WGS84 for storage
                temp_point_wgs = safe_transform(transformer_to_wgs, point_on_line_utm)
                if temp_point_wgs and not temp_point_wgs.is_empty:
                     best_dist_m = distance_m
                     best_point_on_duct_wgs = temp_point_wgs
                     best_duct = duct
                else:
                    logger.warning(f"Client {client.id}: Could not transform best point on duct {duct_id} back to WGS84. Skipping this duct as potential best.")

        except Exception as e:
            # Catch other potential geometry or calculation errors
            logger.warning(f"Client {client.id}: Error processing duct {duct_id}: {e}. Skipping duct.")
            continue

    # Check if a valid connection point was found
    if best_point_on_duct_wgs is None or best_duct is None:
        logger.warning(f"Client {client.id}: Could not find any suitable duct bank connection point after checking {len(duct_banks)} ducts.")
        return None, None, None, False

    # Check drop length limit
    if best_dist_m > params.max_drop_length:
        logger.warning(f"Drop length for client {client.id} ({best_dist_m:.1f}m) to duct {best_duct['properties']['id']} exceeds maximum ({params.max_drop_length}m).")
        exceeds_max_len = True

    # Create the straight drop line geometry in WGS84
    try:
         # Ensure client geometry is valid before creating line
         if not client.geometry or client.geometry.is_empty or not client.geometry.is_valid:
             raise ValueError("Client geometry is invalid")
         drop_line_wgs = LineString([client.geometry, best_point_on_duct_wgs])
         if not drop_line_wgs.is_valid:
             # Attempt buffer(0) fix for potential self-intersection or other issues
             drop_line_wgs = drop_line_wgs.buffer(0)
             if not isinstance(drop_line_wgs, LineString) or not drop_line_wgs.is_valid:
                 raise ValueError("Generated drop line is invalid even after buffer(0).")
    except Exception as e:
         logger.error(f"Failed to create valid LineString for drop fibre Client {client.id}: {e}")
         return None, None, None, False

    drop_props = {
        "client_id": client.id,
        "length_m": best_dist_m,
        "duct_bank_id": best_duct["properties"]["id"],
        "name": f"Drop_Fibre_C{client.id}",
        "fibre_type": f"Drop ({params.drop_fibre_strains}F)",
        "cost": best_dist_m * params.budget_drop_fibre_per_m,
        "exceeds_max_length": exceeds_max_len,
        "connection_point_wgs": best_point_on_duct_wgs.wkt # Store connection point WKT
    }

    drop_fibre_dict: GeoDict = {"geometry": drop_line_wgs, "properties": drop_props}

    return drop_fibre_dict, best_point_on_duct_wgs, best_duct, exceeds_max_len


def get_network_route(G: nx.MultiDiGraph, orig_point: Point, dest_node_id: NodeId) -> Optional[NetworkRoute]:
    """
    Helper to get shortest path geometry (LineString WGS84) and length (meters)
    from an arbitrary origin Point to a target Node ID on the graph.
    Finds nearest graph node to the origin point first.
    Uses 'length' edge attribute as weight. Returns None if no path exists or error occurs.
    """
    if not G or G.number_of_nodes() < 1:
         logger.warning("Cannot get network route: Graph is empty or has no nodes.")
         return None
    if not orig_point or orig_point.is_empty or not orig_point.is_valid:
        logger.warning(f"Cannot get network route: Origin point is invalid or empty ({orig_point}).")
        return None

    orig_node_id = None
    try:
        # Find the graph node nearest to the arbitrary origin point
        orig_node_id = ox.nearest_nodes(G, X=orig_point.x, Y=orig_point.y)
        if isinstance(orig_node_id, (list, np.ndarray)): orig_node_id = orig_node_id[0] # Ensure single ID

        # Validate destination node exists
        if dest_node_id not in G:
             raise nx.NodeNotFound(f"Destination node {dest_node_id} not found in graph.")

        # Check if origin and destination graph nodes are the same
        if orig_node_id == dest_node_id:
             # Route length is 0. Return a LineString representing the node location.
             dest_node_geom = Point(G.nodes[dest_node_id]['x'], G.nodes[dest_node_id]['y'])
             # Create a minimal LineString (two identical points slightly offset for validity if needed)
             try:
                 # Use tiny offset to avoid potential issues with identical points in some contexts
                 offset = 1e-9
                 return LineString([dest_node_geom, Point(dest_node_geom.x + offset, dest_node_geom.y)]), 0.0
             except Exception as e:
                 logger.warning(f"Could not create LineString for zero-length route at node {dest_node_id}: {e}")
                 return None # Treat as failure if point geometry fails


        # Find the shortest path using node IDs and 'length' weight
        path_nodes = nx.shortest_path(G, source=orig_node_id, target=dest_node_id, weight='length')

        if not path_nodes or len(path_nodes) < 2:
            # Should be caught by NetworkXNoPath, but check defensively
            logger.warning(f"Shortest path returned None or < 2 nodes between {orig_node_id} and {dest_node_id}.")
            return None

        # Calculate length using the same weight for consistency
        path_length = nx.shortest_path_length(G, source=orig_node_id, target=dest_node_id, weight='length')

        # Create path geometry from node coordinates
        path_coords = []
        for nid in path_nodes:
             try:
                 node_data = G.nodes[nid]
                 path_coords.append((node_data['x'], node_data['y']))
             except KeyError:
                  logger.error(f"Node {nid} from shortest path not found in graph data. Cannot create geometry.")
                  return None # Abort if node data is missing

        if len(path_coords) < 2:
             logger.error(f"Insufficient valid coordinates ({len(path_coords)}) to create LineString for route {orig_node_id}->{dest_node_id}.")
             return None

        path_geom = LineString(path_coords)

        if not path_geom.is_valid:
             logger.warning(f"Generated path geometry between {orig_node_id} and {dest_node_id} is invalid. Path nodes: {path_nodes}. Attempting fix.")
             path_geom = path_geom.buffer(0) # Attempt fix
             if not isinstance(path_geom, LineString) or not path_geom.is_valid:
                 logger.error(f"Path geometry invalid even after fix. Route {orig_node_id}->{dest_node_id}. Aborting route.")
                 return None

        return path_geom, path_length

    except nx.NodeNotFound as e:
         # Log specific node ID if available
         missing_node = e.args[0] if e.args else "Unknown"
         logger.warning(f"Network route error: Node '{missing_node}' (Origin: {orig_node_id}, Dest: {dest_node_id}) not found in graph.")
         return None
    except nx.NetworkXNoPath:
        # This is expected if nodes are in disconnected components or route is impossible
        logger.debug(f"No network path found between node {orig_node_id} (near {orig_point.wkt}) and node {dest_node_id}.")
        return None
    except KeyError as e:
        # Catch missing 'x' or 'y' keys in node data
        logger.error(f"Network route error: Missing coordinate data ('x' or 'y') for node {e} in route {orig_node_id}->{dest_node_id}.")
        return None
    except Exception as e:
        # Catch other potential errors (e.g., issues with graph data, length attributes)
        logger.error(f"Unexpected error finding network route from near {orig_point.wkt} (Node {orig_node_id}) to node {dest_node_id}: {e}")
        return None


def compute_distribution_cable(
    duct_conn_point: Point,
    manholes: List[NodePoint],
    client_id: int,
    params: NetworkParameters,
    G: nx.MultiDiGraph
) -> Tuple[Optional[GeoDict], Optional[NodePoint], bool]:
    """
    Compute distribution cable from duct connection point to the nearest *reachable* manhole node.
    Routes along the network graph using shortest path ('length' weight) if possible.
    Falls back to straight-line distance calculation if network routing fails or is disabled.

    Returns:
        Tuple containing:
        - Distribution cable dictionary (GeoDict) or None if failed.
        - The chosen manhole NodePoint (ID, Point geometry WGS84) or None.
        - Boolean flag indicating if straight-line fallback was used.
    """
    if not manholes:
        logger.error(f"Client {client_id}: No manholes provided for distribution routing. Cannot compute.")
        return None, None, False
    if not G or G.number_of_nodes() == 0:
         logger.error(f"Client {client_id}: Cannot route distribution cable, network graph is empty.")
         return None, None, False
    if not duct_conn_point or duct_conn_point.is_empty or not duct_conn_point.is_valid:
         logger.error(f"Client {client_id}: Invalid duct connection point provided ({duct_conn_point}). Cannot compute.")
         return None, None, False

    chosen_manhole_np: Optional[NodePoint] = None
    final_route_geom: Optional[LineString] = None
    final_route_length: float = float('inf')
    routing_fallback_used: bool = False
    fallback_reason: str = "N/A"

    # Find the graph node nearest to the duct connection point (start of distribution route)
    duct_conn_nodepoint = find_nearest_node_point(G, duct_conn_point)
    if not duct_conn_nodepoint:
        logger.error(f"Client {client_id}: Failed to find nearest graph node to duct connection point {duct_conn_point.wkt}. Cannot route distribution cable.")
        return None, None, False
    duct_conn_node_id, _ = duct_conn_nodepoint

    shortest_path_len = float('inf')
    candidate_manhole_np: Optional[NodePoint] = None

    # Attempt Network Routing (if enabled)
    if params.use_network_routing:
        logger.debug(f"Client {client_id}: Attempting network routing for distribution from node {duct_conn_node_id} near {duct_conn_point.wkt}.")
        possible_routes: List[Tuple[float, NodePoint]] = [] # (path_length, manhole_nodepoint)

        for mh_np in manholes: # mh_np is (mh_node_id, mh_point)
            mh_node_id, mh_point = mh_np
            try:
                # Check if start and end nodes are the same graph node
                if duct_conn_node_id == mh_node_id:
                    path_len = 0.0 # Zero length if same node
                else:
                    # Calculate shortest path length using 'length' weight
                    # This checks reachability
                    path_len = nx.shortest_path_length(G, source=duct_conn_node_id, target=mh_node_id, weight='length')

                if path_len is not None and path_len >= 0:
                     possible_routes.append((path_len, mh_np))
                else:
                     logger.debug(f"Client {client_id}: Invalid path length ({path_len}) calculated to manhole {mh_node_id}. Skipping.")

            except nx.NetworkXNoPath:
                # Expected if manhole node is in a different graph component
                 logger.debug(f"Client {client_id}: No network path from node {duct_conn_node_id} to manhole node {mh_node_id}.")
                 continue # Skip this manhole
            except nx.NodeNotFound:
                 logger.warning(f"Client {client_id}: Distribution target manhole node {mh_node_id} not found in graph. Skipping.")
                 continue
            except Exception as e:
                 # Log other potential pathfinding errors
                 logger.warning(f"Client {client_id}: Error calculating path length from {duct_conn_node_id} to manhole {mh_node_id}: {e}")
                 continue # Skip this manhole

        if possible_routes:
            # Found at least one reachable manhole via network
            possible_routes.sort(key=lambda x: x[0]) # Sort by path length
            shortest_path_len, candidate_manhole_np = possible_routes[0]

            # Now get the actual path geometry using the helper function
            # Route from the *original* duct connection point to the chosen manhole node ID
            route_result = get_network_route(G, duct_conn_point, candidate_manhole_np[0])

            if route_result:
                 final_route_geom, calculated_geom_len = route_result
                 # Use length from shortest_path_length which is based on node-to-node path weights
                 final_route_length = shortest_path_len
                 chosen_manhole_np = candidate_manhole_np
                 logger.debug(f"Client {client_id}: Found network route to manhole {candidate_manhole_np[0]} (Length: {final_route_length:.1f}m).")
            else:
                 # Failed to get geometry even though length was found (e.g., path nodes missing coords)
                 logger.warning(f"Client {client_id}: Could not retrieve path geometry to manhole {candidate_manhole_np[0]} (path length ~{shortest_path_len:.1f}m). Attempting fallback.")
                 final_route_geom = None # Reset to trigger fallback
                 chosen_manhole_np = None
                 fallback_reason = "Network route geometry generation failed"
        else:
             logger.warning(f"Client {client_id}: No manholes were reachable via the network from node {duct_conn_node_id}. Attempting fallback.")
             fallback_reason = "No reachable manholes via network"
             # Fall through to straight-line calculation
    else: # Network routing disabled
         logger.info(f"Client {client_id}: Network routing disabled by parameters. Using straight-line fallback for distribution.")
         fallback_reason = "Network routing disabled"


    # Fallback: If network routing failed, was disabled, or no path found
    if final_route_geom is None:
        routing_fallback_used = True
        logger.debug(f"Client {client_id}: Using straight-line distance fallback to find nearest manhole ({fallback_reason}).")
        shortest_straight_dist = float('inf')
        temp_best_manhole_np: Optional[NodePoint] = None

        for mh_np in manholes:
            mh_node_id, mh_point = mh_np
            try:
                # Measure distance using UTM for accuracy
                dist_m = measure_distance_utm(duct_conn_point, mh_point)
                if dist_m < shortest_straight_dist:
                    shortest_straight_dist = dist_m
                    temp_best_manhole_np = mh_np
            except Exception as e:
                 logger.warning(f"Client {client_id}: Error measuring straight distance to manhole {mh_node_id}: {e}")
                 continue

        if temp_best_manhole_np:
            chosen_manhole_np = temp_best_manhole_np
            final_route_length = shortest_straight_dist
            # Create straight line geometry
            try:
                 # Ensure points are valid before creating line
                 if not duct_conn_point.is_valid or not chosen_manhole_np[1].is_valid:
                      raise ValueError("Invalid point geometry for fallback line.")
                 final_route_geom = LineString([duct_conn_point, chosen_manhole_np[1]])
                 if not final_route_geom.is_valid:
                     final_route_geom = final_route_geom.buffer(0) # Try fix
                     if not isinstance(final_route_geom, LineString) or not final_route_geom.is_valid:
                         raise ValueError("Fallback geometry invalid even after fix.")
                 logger.info(f"Client {client_id}: Using straight-line fallback route to manhole {chosen_manhole_np[0]} (Length: {final_route_length:.1f}m). Reason: {fallback_reason}.")
            except Exception as e:
                 logger.error(f"Client {client_id}: Failed to create valid straight-line fallback geometry to manhole {chosen_manhole_np[0]}: {e}")
                 return None, None, routing_fallback_used # Cannot create cable if geometry fails
        else:
            # This means no manholes were provided or distance calc failed for all
            logger.error(f"Client {client_id}: Could not find any nearest manhole even using straight-line distance fallback.")
            return None, None, routing_fallback_used # Cannot create cable


    # Final check: Ensure we have valid results before creating dict
    if final_route_geom is None or chosen_manhole_np is None or not isinstance(final_route_geom, LineString):
         logger.error(f"Client {client_id}: Failed to determine valid distribution route geometry or manhole despite logic flow. Aborting.")
         return None, None, routing_fallback_used

    dist_props = {
        "client_id": client_id,
        "manhole_id": f"MH_{chosen_manhole_np[0]}", # Use node ID for manhole ID
        "manhole_node_id": chosen_manhole_np[0], # Store node ID explicitly
        "length_m": final_route_length,
        "name": f"Dist_Cable_C{client_id}_MH{chosen_manhole_np[0]}",
        "fibre_type": f"Distribution ({params.distribution_fibre_strains}F)",
        "cost": final_route_length * params.budget_dist_fibre_per_m,
        "routed_on_network": not routing_fallback_used,
        "fallback_reason": fallback_reason if routing_fallback_used else "N/A"
    }

    dist_cable_dict: GeoDict = {"geometry": final_route_geom, "properties": dist_props}

    return dist_cable_dict, chosen_manhole_np, routing_fallback_used


def compute_feeder_cable(
    manhole_np: NodePoint,
    aggregation_np: NodePoint,
    params: NetworkParameters,
    G: nx.MultiDiGraph
) -> Tuple[Optional[GeoDict], bool]:
    """
    Compute feeder cable from the manhole node to the aggregation node.
    Routes along the network graph using shortest path ('length' weight) if possible.
    Falls back to straight-line if network routing fails or is disabled.

    Returns:
        Tuple containing:
        - Feeder cable dictionary (GeoDict) or None if failed.
        - Boolean flag indicating if straight-line fallback was used.
    """
    manhole_node_id, manhole_point = manhole_np
    agg_node_id, agg_point = aggregation_np
    routing_fallback_used: bool = False
    fallback_reason: str = "N/A"
    feeder_geom: Optional[LineString] = None
    feeder_length: float = 0.0

    if not G or G.number_of_nodes() == 0:
         logger.error(f"Feeder MH_{manhole_node_id}: Cannot route feeder cable, graph is empty.")
         return None, False

    # Attempt Network Routing (if enabled)
    if params.use_network_routing:
        logger.debug(f"Feeder MH_{manhole_node_id}: Attempting network routing to AGG_{agg_node_id}.")
        # Route directly between the manhole node and aggregation node
        # Use manhole point as origin for get_network_route helper
        route_result = get_network_route(G, manhole_point, agg_node_id)

        if route_result:
            feeder_geom, feeder_length = route_result
            logger.debug(f"Feeder MH_{manhole_node_id} to AGG_{agg_node_id}: Found network route (Length: {feeder_length:.1f}m).")
        else:
            logger.warning(f"Feeder MH_{manhole_node_id} to AGG_{agg_node_id}: No network path found or route generation failed. Attempting fallback.")
            fallback_reason = "Network route calculation failed"
            feeder_geom = None # Ensure fallback is triggered
    else:
        logger.info(f"Feeder MH_{manhole_node_id}: Network routing disabled by parameters. Using straight-line fallback.")
        fallback_reason = "Network routing disabled"


    # Fallback: If network routing disabled or failed
    if feeder_geom is None:
        routing_fallback_used = True
        logger.info(f"Feeder MH_{manhole_node_id} to AGG_{agg_node_id}: Using straight-line fallback route. Reason: {fallback_reason}.")
        try:
             if not manhole_point.is_valid or not agg_point.is_valid:
                 raise ValueError("Invalid point geometry for fallback line.")
             feeder_geom = LineString([manhole_point, agg_point])
             if not feeder_geom.is_valid:
                 feeder_geom = feeder_geom.buffer(0) # Try fix
                 if not isinstance(feeder_geom, LineString) or not feeder_geom.is_valid:
                     raise ValueError("Fallback geometry invalid even after fix.")
             feeder_length = get_length_in_meters(feeder_geom) # Calculate length of straight line
        except Exception as e:
             logger.error(f"Feeder MH_{manhole_node_id}: Failed to create valid straight-line fallback geometry to AGG_{agg_node_id}: {e}")
             return None, routing_fallback_used # Cannot create cable

    # Check if geometry and length seem valid
    if feeder_geom is None or feeder_geom.is_empty or not isinstance(feeder_geom, LineString):
         logger.error(f"Feeder MH_{manhole_node_id}: Final feeder geometry is invalid or empty. Aborting feeder.")
         return None, routing_fallback_used

    feeder_props = {
        "manhole_id": f"MH_{manhole_node_id}",
        "manhole_node_id": manhole_node_id,
        "aggregation_node_id": agg_node_id,
        "length_m": feeder_length,
        "name": f"Feeder_MH{manhole_node_id}_AGG{agg_node_id}",
        "fibre_type": f"Feeder ({params.feeder_fibre_strains}F)",
        "cost": feeder_length * params.budget_feeder_fibre_per_m,
        "routed_on_network": not routing_fallback_used,
        "fallback_reason": fallback_reason if routing_fallback_used else "N/A"
    }

    feeder_cable_dict: GeoDict = {"geometry": feeder_geom, "properties": feeder_props}

    return feeder_cable_dict, routing_fallback_used


def compute_splice_allocation(client: BuildingClient, manhole_node_id: NodeId, aggregation_node_id: NodeId, params: NetworkParameters) -> str:
    """
    Generate a simplified **placeholder** splice allocation string for client documentation.
    This requires significant refinement for real-world capacity and allocation logic.
    """
    # Example Format: Client -> Splitter (Manhole) Port -> Feeder Fibre ID -> Aggregation Port
    # Assumptions:
    # - Splitter port derived simply from client ID modulo max capacity (no actual port reservation).
    # - Feeder fibre ID simplified based on client ID (not realistic fibre/strand allocation).
    # - Aggregation port simplified based on groups of clients (no actual ODF/port mapping).
    try:
        # Use configured max clients per splitter for modulo operation
        max_splits = params.max_clients_per_splitter
        splitter_port = (client.id % max_splits) + 1 if max_splits > 0 else 1 # Basic modulo
        feeder_fibre_id = f"Fibre-{((client.id -1) // max_splits) + 1}" if max_splits > 0 else f"Fibre-{client.id}" # Group fibres
        agg_port_identifier = f"AGG-{aggregation_node_id}-P{((client.id -1) // max_splits) + 1}" if max_splits > 0 else f"AGG-{aggregation_node_id}-P{client.id}" # Simple grouping
    except Exception as e:
         logger.warning(f"Error generating placeholder splice allocation details for client {client.id}: {e}")
         return f"Client {client.id} (OSM:{client.building_id}) -> MH_{manhole_node_id} -> ERROR"


    return (f"C:{client.id}(OSM:{client.building_id}) >> "
            f"MH:{manhole_node_id}(Port:{splitter_port}) >> "
            f"{feeder_fibre_id} >> {agg_port_identifier}")

# ---------------------------
# KML GENERATION
# ---------------------------
def initialize_kml(place_name: str) -> Tuple[simplekml.Kml, Dict[str, simplekml.Folder], Dict[str, simplekml.Style]]:
    """Initialize a KML document with standard folders and styles."""
    kml = simplekml.Kml(name=f"Fibre Network Design v{VERSION} - {place_name}")
    folders = {
        "zones": kml.newfolder(name="Design Zones"),
        "clients": kml.newfolder(name="Client Endpoints"),
        "duct_banks": kml.newfolder(name="Duct Banks (Road Network)"),
        "manholes": kml.newfolder(name="Manholes / Splitters"),
        "drop_fibres": kml.newfolder(name="Drop Fibres"),
        "distribution_cables": kml.newfolder(name="Distribution Cables"),
        "feeder_cables": kml.newfolder(name="Feeder Cables"),
        "aggregation_nodes": kml.newfolder(name="Aggregation Nodes"),
    }

    # Define styles centrally using constants
    styles = {}

    # Client Style
    styles[STYLE_CLIENT] = simplekml.Style()
    styles[STYLE_CLIENT].iconstyle.icon.href = 'http://maps.google.com/mapfiles/kml/paddle/blu-circle.png'
    styles[STYLE_CLIENT].iconstyle.scale = 0.8
    styles[STYLE_CLIENT].labelstyle.scale = 0.7 # Smaller labels

    # Manhole Style
    styles[STYLE_MANHOLE] = simplekml.Style()
    styles[STYLE_MANHOLE].iconstyle.icon.href = 'http://maps.google.com/mapfiles/kml/shapes/placemark_circle.png' # Circle instead of square
    styles[STYLE_MANHOLE].iconstyle.color = simplekml.Color.red # FF0000FF
    styles[STYLE_MANHOLE].iconstyle.scale = 0.7
    styles[STYLE_MANHOLE].labelstyle.scale = 0.7

    # Aggregation Node Style
    styles[STYLE_AGGREGATION] = simplekml.Style()
    styles[STYLE_AGGREGATION].iconstyle.icon.href = 'http://maps.google.com/mapfiles/kml/shapes/star.png'
    styles[STYLE_AGGREGATION].iconstyle.color = simplekml.Color.orange # FF00A5FF
    styles[STYLE_AGGREGATION].iconstyle.scale = 1.2
    styles[STYLE_AGGREGATION].labelstyle.scale = 0.8

    # Duct Bank Style
    styles[STYLE_DUCT] = simplekml.Style()
    styles[STYLE_DUCT].linestyle.color = simplekml.Color.changealphaint(150, simplekml.Color.grey) # AARRGGBB -> 96808080
    styles[STYLE_DUCT].linestyle.width = 2

    # Drop Fibre Style
    styles[STYLE_DROP] = simplekml.Style()
    styles[STYLE_DROP].linestyle.color = simplekml.Color.changealphaint(200, simplekml.Color.cyan) # C8FFFFFF
    styles[STYLE_DROP].linestyle.width = 1.5

    # Distribution Cable Style
    styles[STYLE_DISTRIBUTION] = simplekml.Style()
    styles[STYLE_DISTRIBUTION].linestyle.color = simplekml.Color.yellow # FF00FFFF
    styles[STYLE_DISTRIBUTION].linestyle.width = 3

    # Feeder Cable Style
    styles[STYLE_FEEDER] = simplekml.Style()
    styles[STYLE_FEEDER].linestyle.color = simplekml.Color.magenta # FFFF00FF
    styles[STYLE_FEEDER].linestyle.width = 4

    # Zone Polygon Style
    styles[STYLE_ZONE] = simplekml.Style()
    styles[STYLE_ZONE].polystyle.color = simplekml.Color.hexa('80FF7F00') # 50% Alpha, Orange-ish Blue
    styles[STYLE_ZONE].polystyle.outline = 1
    styles[STYLE_ZONE].linestyle.color = simplekml.Color.white # FFFFFFFF
    styles[STYLE_ZONE].linestyle.width = 1

    # Style for fallback routes (applied conditionally)
    styles[STYLE_FALLBACK] = simplekml.Style()
    # Use a distinct color and slightly thinner width for fallback (straight line) routes
    styles[STYLE_FALLBACK].linestyle.color = simplekml.Color.changealphaint(180, simplekml.Color.grey) # B4808080
    styles[STYLE_FALLBACK].linestyle.width = 2.5 # Slightly thinner than main distribution/feeder

    return kml, folders, styles

def add_element_to_kml(folder: simplekml.Folder, element_type_id: str, element: GeoDict, styles: Dict[str, simplekml.Style], name_prefix: str = ""):
    """
    Helper function to add a geographic element (Point, LineString, Polygon) to a KML folder.
    Applies appropriate style and generates a description from properties.
    """
    props = element.get("properties", {})
    geom = element.get("geometry")

    if geom is None or geom.is_empty:
        logger.debug(f"Skipping KML addition for element with missing/empty geometry: Type={element_type_id}, Props={props.get('id', 'N/A')}")
        return

    # Determine name (use specific ID fields first)
    element_id_val = props.get('id', props.get('client_id', props.get('name', 'Unknown')))
    name = props.get("name", f"{element_type_id.replace('_', ' ').title()} {element_id_val}")
    if name_prefix: name = f"{name_prefix}_{name}"

    # Create description string from properties, formatting floats and handling special keys
    description_parts = []
    prop_order = [ # Define preferred order for key info
        'id', 'name', 'client_id', 'manhole_id', 'aggregation_node_id', 'node_id',
        'length_m', 'cost', 'routed_on_network', 'fallback_reason', 'exceeds_max_length',
        'osm_id', 'building_id', 'building_type', 'building_levels', 'address',
        'road_name', 'road_type', 'cost_per_meter', 'total_cost'
    ]
    processed_keys = set()

    for key in prop_order:
         if key in props and props[key] is not None:
              val = props[key]
              # Special formatting for certain types/keys
              if isinstance(val, float): val = f"{val:.2f}"
              if key == 'address' and isinstance(val, dict):
                   val = "; ".join([f"{k.title()}: {v}" for k, v in val.items() if v])
              elif isinstance(val, bool): val = str(val)
              elif key == 'fallback_reason' and val == "N/A": continue # Don't show N/A reason

              description_parts.append(f"<b>{key.replace('_', ' ').title()}:</b> {val}")
              processed_keys.add(key)

    # Add any remaining properties not in the preferred order
    for key, val in props.items():
        # Exclude already processed, internal, or purely geometric properties
        if key not in processed_keys and key not in ['name', 'geometry', 'graph_edge', 'connection_point_wgs'] and val is not None:
             if isinstance(val, float): val = f"{val:.2f}"
             if isinstance(val, dict): val = json.dumps(val) # Basic JSON dump for other dicts
             if isinstance(val, list): val = ", ".join(map(str, val)) # Comma-sep for lists
             description_parts.append(f"<b>{key.replace('_', ' ').title()}:</b> {val}")
    description = "<br>".join(description_parts)

    # Determine style based on element type and properties
    base_style_key = element_type_id.lower() # e.g., 'client', 'distribution'
    style_to_use = styles.get(base_style_key, simplekml.Style()) # Default to base style for the type

    # Apply fallback style conditionally for distribution/feeder cables
    is_fallback = props.get('routed_on_network') is False
    if is_fallback and base_style_key in [STYLE_DISTRIBUTION, STYLE_FEEDER]:
         fallback_style = styles.get(STYLE_FALLBACK)
         if fallback_style:
             style_to_use = fallback_style
             name += " (Fallback Route)" # Append to name for clarity in KML tree

    # Add geometry to KML folder
    try:
        if isinstance(geom, Point):
            pt = folder.newpoint(name=name, description=description)
            pt.coords = [(geom.x, geom.y)]
            pt.style = style_to_use
        elif isinstance(geom, LineString):
            # Filter out invalid coordinate pairs if any (e.g., NaN, Inf)
            valid_coords = [(float(lon), float(lat)) for lon, lat in geom.coords if isinstance(lon, (int, float)) and isinstance(lat, (int, float)) and math.isfinite(lon) and math.isfinite(lat)]
            if len(valid_coords) < 2:
                 logger.warning(f"KML: Insufficient valid coordinates ({len(valid_coords)}) for LineString '{name}'. Skipping.")
                 return
            ls = folder.newlinestring(name=name, description=description)
            ls.coords = valid_coords
            ls.style = style_to_use
            ls.tesellate = 1 # Follow terrain (optional)
            ls.altitudemode = simplekml.AltitudeMode.clamptoground # Clamp lines to ground
        elif isinstance(geom, Polygon): # For Zones
            poly = folder.newpolygon(name=name, description=description)
            outer_coords = [(float(lon), float(lat)) for lon, lat in geom.exterior.coords if math.isfinite(lon) and math.isfinite(lat)]
            if len(outer_coords) < 3:
                 logger.warning(f"KML: Insufficient valid coordinates for Polygon outer boundary '{name}'. Skipping.")
                 return
            poly.outerboundaryis = outer_coords
            # Add inner boundaries (holes) if they exist and are valid
            if geom.interiors:
                 inner_boundaries = []
                 for interior in geom.interiors:
                      inner_coords = [(float(lon), float(lat)) for lon, lat in interior.coords if math.isfinite(lon) and math.isfinite(lat)]
                      if len(inner_coords) >= 3:
                          inner_boundaries.append(inner_coords)
                 if inner_boundaries:
                      poly.innerboundaryis = inner_boundaries
            poly.style = style_to_use
        else:
            logger.warning(f"KML Generation: Unsupported geometry type '{type(geom)}' for element '{name}'. Skipping.")

    except Exception as e:
        # Catch potential errors during KML object creation or coordinate setting
        logger.error(f"Failed to add element '{name}' (Type: {element_type_id}) to KML folder '{folder.name}': {e}", exc_info=True)


# ---------------------------
# CSV REPORT GENERATION
# ---------------------------
def _safe_filename(name: str) -> str:
    """Create a safe filename by removing/replacing invalid characters."""
    # Remove common path separators and invalid characters
    invalid_chars = r'<>:"/\|?*' + "".join(map(chr, range(32))) # Includes control chars
    safe_name = name.replace(' ', '_').replace(',', '')
    for char in invalid_chars:
        safe_name = safe_name.replace(char, '')
    # Truncate if excessively long (optional)
    max_len = 100
    if len(safe_name) > max_len:
        safe_name = safe_name[:max_len]
    return safe_name or "default_filename" # Ensure not empty

def generate_client_report(clients: List[BuildingClient], drop_fibres: List[GeoDict], dist_cables: List[GeoDict], splice_allocations: Dict[int, str], output_dir: str, place_name: str) -> str:
    """Generate a CSV report detailing client connections."""
    os.makedirs(output_dir, exist_ok=True)
    report_file = os.path.join(output_dir, f"{_safe_filename(place_name)}_client_connections{CSV_EXT}")

    # Create lookups for faster access by client_id
    drop_lookup: Dict[int, Dict] = {d["properties"]["client_id"]: d["properties"] for d in drop_fibres if "client_id" in d.get("properties", {})}
    dist_lookup: Dict[int, Dict] = {d["properties"]["client_id"]: d["properties"] for d in dist_cables if "client_id" in d.get("properties", {})}

    logger.info(f"Generating client connection report with {len(clients)} clients...")
    written_rows = 0
    try:
        with open(report_file, 'w', newline='', encoding='utf-8') as csvfile:
            writer = csv.writer(csvfile)
            # Define headers
            headers = [
                "Client ID", "OSM Building ID", "Building Type", "Est Levels", "Address",
                "Client Latitude", "Client Longitude",
                "Drop Fibre Length (m)", "Drop Exceeds Max Length", "Drop Connection Point (WKT)", "Drop Duct Bank ID",
                "Dist Cable Length (m)", "Dist Routed on Network", "Dist Fallback Reason", "Connected Manhole ID", "Connected Manhole Node ID",
                "Total Cable Length (m)", "Splice Allocation (Placeholder)"
            ]
            writer.writerow(headers)

            for client in clients:
                try:
                    client_id = client.id
                    drop_props = drop_lookup.get(client_id)
                    dist_props = dist_lookup.get(client_id)
                    splice = splice_allocations.get(client_id, "N/A")

                    # Format address safely
                    address_str = "; ".join([f"{k}: {v}" for k, v in client.address.items() if v])

                    # Extract properties safely with defaults
                    drop_length = drop_props.get("length_m", 0.0) if drop_props else 0.0
                    drop_exceeds = drop_props.get("exceeds_max_length", False) if drop_props else False
                    drop_conn_pt = drop_props.get("connection_point_wgs", "N/A") if drop_props else "N/A"
                    drop_duct_id = drop_props.get("duct_bank_id", "N/A") if drop_props else "N/A"

                    dist_length = dist_props.get("length_m", 0.0) if dist_props else 0.0
                    dist_routed = dist_props.get("routed_on_network", "N/A") if dist_props else "N/A"
                    dist_fallback_reason = dist_props.get("fallback_reason", "N/A") if dist_props else "N/A"
                    manhole_id_str = dist_props.get("manhole_id", "N/A") if dist_props else "N/A" # Formatted ID: MH_xxx
                    manhole_node_id = dist_props.get("manhole_node_id", "N/A") if dist_props else "N/A" # Raw Node ID

                    total_length = drop_length + dist_length

                    writer.writerow([
                        client_id, client.building_id, client.building_type, client.building_levels, address_str,
                        f"{client.geometry.y:.7f}", f"{client.geometry.x:.7f}", # Lat, Lon
                        f"{drop_length:.2f}", drop_exceeds, drop_conn_pt, drop_duct_id,
                        f"{dist_length:.2f}", dist_routed, dist_fallback_reason, manhole_id_str, manhole_node_id,
                        f"{total_length:.2f}",
                        splice
                    ])
                    written_rows += 1
                except Exception as e:
                     # Log error for specific client row, but continue with others
                     logger.error(f"Error writing row for client {getattr(client, 'id', 'N/A')} to CSV: {e}", exc_info=True)
                     continue # Skip row on error

        logger.info(f"Generated client connection report ({written_rows} rows): {report_file}")
        return report_file
    except IOError as e:
         logger.error(f"IOError generating client report '{report_file}': {e}", exc_info=True)
         raise # Re-raise IO errors
    except Exception as e:
         logger.error(f"Unexpected error generating client report '{report_file}': {e}", exc_info=True)
         raise # Re-raise other critical errors


def generate_infrastructure_report(duct_banks: List[GeoDict], manholes: List[NodePoint], feeder_cables: List[GeoDict], aggregation_nodes: List[NodePoint], params: NetworkParameters, output_dir: str, place_name: str) -> str:
    """Generate a CSV report detailing network infrastructure elements."""
    os.makedirs(output_dir, exist_ok=True)
    report_file = os.path.join(output_dir, f"{_safe_filename(place_name)}_infrastructure{CSV_EXT}")

    logger.info("Generating infrastructure report...")
    written_rows = 0
    try:
        with open(report_file, 'w', newline='', encoding='utf-8') as csvfile:
            writer = csv.writer(csvfile)
            # Define headers
            headers = [
                "Element Type", "Element ID", "OSM ID / Node ID", "Latitude (Centroid/Point)", "Longitude (Centroid/Point)",
                "Length (m)", "Cost", "Unit/Meter Cost", "Routed on Network", "Fallback Reason", "Details"
            ]
            writer.writerow(headers)

            # Add Duct Banks
            for duct in duct_banks:
                try:
                    props = duct.get("properties", {})
                    geom = duct.get("geometry")
                    center = geom.centroid if geom and not geom.is_empty else Point() # Handle empty geometry
                    details = f"Road: {props.get('road_name', 'N/A')}, Type: {props.get('road_type', 'N/A')}"
                    writer.writerow([
                        "Duct Bank Segment", props.get("id", "N/A"), props.get("osm_id", "N/A"),
                        f"{center.y:.7f}", f"{center.x:.7f}", # Centroid Lat, Lon
                        f"{props.get('length_m', 0.0):.2f}", # Length
                        f"{props.get('total_cost', 0.0):.2f}", # Cost
                        f"{props.get('cost_per_meter', 0.0):.2f}", # Unit Cost
                        "N/A", "N/A", # Routed, Fallback Reason
                        details
                    ])
                    written_rows += 1
                except Exception as e:
                     logger.error(f"Error writing duct bank {props.get('id', 'N/A')} to CSV: {e}", exc_info=True)

            # Add Manholes (All potential locations placed)
            for mh_node_id, mh_point in manholes:
                 try:
                     details = f"Node ID: {mh_node_id}" # Basic detail
                     writer.writerow([
                         "Manhole/Splitter Location", f"MH_{mh_node_id}", mh_node_id,
                         f"{mh_point.y:.7f}", f"{mh_point.x:.7f}", # Point Lat, Lon
                         "N/A", # Length
                         f"{params.budget_manhole:.2f}", # Cost (per manhole structure)
                         f"{params.budget_manhole:.2f}", # Unit Cost (same as total for one item)
                         "N/A", "N/A", # Routed, Fallback Reason
                         details
                     ])
                     written_rows += 1
                 except Exception as e:
                     logger.error(f"Error writing manhole {mh_node_id} to CSV: {e}", exc_info=True)

            # Add Feeder Cables (Only for used manholes)
            for cable in feeder_cables:
                try:
                    props = cable.get("properties", {})
                    geom = cable.get("geometry")
                    center = geom.centroid if geom and not geom.is_empty else Point()
                    details = f"From: MH_{props.get('manhole_node_id', 'N/A')} To: AGG_{props.get('aggregation_node_id', 'N/A')}, Type: {props.get('fibre_type', 'Feeder')}"
                    writer.writerow([
                        "Feeder Cable", props.get("name", "N/A"), f"MH{props.get('manhole_node_id', '?')}->AGG{props.get('aggregation_node_id', '?')}",
                        f"{center.y:.7f}", f"{center.x:.7f}", # Centroid Lat, Lon
                        f"{props.get('length_m', 0.0):.2f}", # Length
                        f"{props.get('cost', 0.0):.2f}", # Cost
                        f"{params.budget_feeder_fibre_per_m:.2f}", # Unit Cost (per meter)
                        props.get("routed_on_network", "N/A"),
                        props.get("fallback_reason", "N/A"),
                        details
                    ])
                    written_rows += 1
                except Exception as e:
                     logger.error(f"Error writing feeder cable {props.get('name', 'N/A')} to CSV: {e}", exc_info=True)


            # Add Aggregation Nodes (Unique ones across zones)
            for agg_node_id, agg_point in aggregation_nodes:
                 try:
                     details = f"Node ID: {agg_node_id}"
                     writer.writerow([
                         "Aggregation Node", f"AGG_{agg_node_id}", agg_node_id,
                         f"{agg_point.y:.7f}", f"{agg_point.x:.7f}", # Point Lat, Lon
                         "N/A", # Length
                         "N/A", # Cost (assume covered elsewhere or negligible point cost)
                         "N/A", # Unit Cost
                         "N/A", "N/A", # Routed, Fallback Reason
                         details
                     ])
                     written_rows += 1
                 except Exception as e:
                     logger.error(f"Error writing aggregation node {agg_node_id} to CSV: {e}", exc_info=True)

        logger.info(f"Generated infrastructure report ({written_rows} rows): {report_file}")
        return report_file
    except IOError as e:
         logger.error(f"IOError generating infrastructure report '{report_file}': {e}", exc_info=True)
         raise
    except Exception as e:
         logger.error(f"Unexpected error generating infrastructure report '{report_file}': {e}", exc_info=True)
         raise

# ---------------------------
# CORE DESIGN LOGIC & ZONE PROCESSING
# ---------------------------
def subdivide_area_if_needed(polygon: Polygon, params: NetworkParameters) -> List[Polygon]:
    """
    Subdivide a large area polygon into smaller rectangular zones based on max_zone_area.
    Uses UTM projection for area calculation and subdivision grid.
    Returns a list of Polygon zones (WGS84). Returns original polygon if no subdivision needed or fails.
    """
    if not polygon or polygon.is_empty or not polygon.is_valid:
        logger.error("Cannot subdivide invalid or empty input polygon.")
        return [] # Return empty list indicates failure

    try:
        poly_utm, utm_crs = to_utm(polygon)
        if not poly_utm or not utm_crs:
             raise ValueError("Failed to project polygon to UTM for area calculation.")
        area_m2 = poly_utm.area
    except (ValueError, ProjError, Exception) as e:
        logger.error(f"Could not calculate area for subdivision: {e}. Proceeding with single zone.")
        return [polygon] # Fallback to original polygon

    if area_m2 <= params.max_zone_area:
        logger.info(f"Area size ({area_m2 / 1e6:.2f} km²) is within limit ({params.max_zone_area / 1e6:.2f} km²). No subdivision needed.")
        return [polygon]

    logger.info(f"Area ({area_m2 / 1e6:.2f} km²) exceeds limit ({params.max_zone_area / 1e6:.2f} km²). Subdividing...")

    # Determine number of grid cells needed (aim for roughly square cells)
    num_zones_needed = math.ceil(area_m2 / params.max_zone_area)
    # Make grid slightly larger than sqrt to ensure cells are likely under the limit
    grid_dim = math.ceil(math.sqrt(num_zones_needed * 1.1)) # Add 10% buffer, ensure integer
    logger.info(f"Creating a {grid_dim}x{grid_dim} grid for subdivision.")

    minx, miny, maxx, maxy = poly_utm.bounds
    if (maxx - minx) <= 1e-6 or (maxy - miny) <= 1e-6: # Check for near-zero dimension
         logger.error("Cannot subdivide polygon with zero or near-zero width/height in UTM projection.")
         return [polygon] # Fallback

    dx = (maxx - minx) / grid_dim
    dy = (maxy - miny) / grid_dim

    subdivided_zones_utm = []
    for i in range(grid_dim):
        for j in range(grid_dim):
            # Create grid cell (box) geometry
            cell_minx = minx + i * dx
            cell_miny = miny + j * dy
            cell_maxx = minx + (i + 1) * dx
            cell_maxy = miny + (j + 1) * dy
            grid_cell = box(cell_minx, cell_miny, cell_maxx, cell_maxy)

            # Intersect grid cell with the original area polygon (in UTM)
            # Use buffer(0) to handle potential minor validity issues from intersection
            try:
                 intersection = poly_utm.intersection(grid_cell).buffer(0)
            except Exception as intersect_e:
                 logger.warning(f"Error during intersection for grid cell {i},{j}: {intersect_e}. Skipping cell.")
                 continue

            # Add valid zones (ignore empty, invalid, or tiny geometries)
            if intersection.is_valid and not intersection.is_empty and intersection.area > MIN_ZONE_AREA_M2:
                 if isinstance(intersection, Polygon):
                     subdivided_zones_utm.append(intersection)
                 elif isinstance(intersection, MultiPolygon):
                     # Add individual polygons from the MultiPolygon if they are large enough
                     for p in intersection.geoms:
                          if p.is_valid and not p.is_empty and p.area > MIN_ZONE_AREA_M2:
                              subdivided_zones_utm.append(p)
                 # else: Ignore other geometry types like points or lines resulting from intersection

    if not subdivided_zones_utm:
        logger.error("Subdivision resulted in no valid zones. Check input polygon and parameters. Falling back to original polygon.")
        return [polygon] # Fallback to original

    # Convert valid subdivided zones back to WGS84
    zones_wgs: List[Polygon] = []
    transformer_to_wgs = None
    try:
        # Get transformer once using the CRS from the first valid UTM zone (should be same for all)
        transformer_to_wgs = get_transformer(utm_crs, "EPSG:4326")
        for zone_utm in subdivided_zones_utm:
             wgs_zone = safe_transform(transformer_to_wgs, zone_utm)
             # Ensure result is a valid Polygon
             if wgs_zone and isinstance(wgs_zone, Polygon) and wgs_zone.is_valid and not wgs_zone.is_empty:
                 zones_wgs.append(wgs_zone)
             elif wgs_zone and isinstance(wgs_zone, MultiPolygon): # If transform results in MultiPolygon
                  # Add largest valid polygon from the multipolygon
                  valid_parts = [p for p in wgs_zone.geoms if isinstance(p, Polygon) and p.is_valid and not p.is_empty]
                  if valid_parts:
                       largest_part = max(valid_parts, key=lambda p: p.area)
                       zones_wgs.append(largest_part)
                       logger.debug("Subdivision transform resulted in MultiPolygon, took largest valid part.")
                  else: logger.warning("Subdivision transform resulted in MultiPolygon with no valid Polygon parts.")
             else:
                 logger.warning(f"Failed to transform subdivided zone back to valid WGS84 Polygon or projection failed.")

    except (ValueError, ProjError, Exception) as e:
         logger.error(f"Failed to transform subdivided zones back to WGS84: {e}. Returning original polygon.")
         return [polygon] # Fallback

    if not zones_wgs:
         logger.error("Subdivision transformation to WGS84 resulted in no valid zones. Returning original polygon.")
         return [polygon] # Fallback

    logger.info(f"Successfully subdivided area into {len(zones_wgs)} design zones.")
    return zones_wgs


def _process_client_connections(
    clients: List[BuildingClient],
    duct_banks: List[GeoDict],
    manholes: List[NodePoint],
    aggregation_node: NodePoint,
    G: nx.MultiDiGraph,
    params: NetworkParameters,
    zone_idx: int
) -> Dict[str, Any]:
    """
    Processes connections (drop + distribution) for all clients in a zone.
    Uses parallel processing for individual client computations.
    Returns a dictionary containing aggregated results for clients.
    """
    logger.info(f"[Zone {zone_idx + 1}] Processing {len(clients)} client connections...")
    start_time = time.time()

    results = {
        "drop_fibres": [],
        "distribution_cables": [],
        "splice_allocations": {},
        "client_to_manhole_map": {}, # Map client ID -> connected manhole NodeId
        "used_manholes": set(),      # Set of NodeIds for manholes actually connected
        "stats_drop_length": 0.0,
        "stats_dist_length": 0.0,
        "stats_clients_over_max_drop": 0,
        "stats_dist_routing_fallbacks": 0,
        "processed_clients_count": 0,
        "failed_clients_count": 0,
    }

    agg_node_id = aggregation_node[0]
    manhole_lookup: Dict[NodeId, Point] = {np[0]: np[1] for np in manholes} # Quick lookup for MH geometry

    # Define task for parallel execution
    def process_single_client_connection(client: BuildingClient):
        client_result = {"success": False, "client_id": client.id}
        try:
            # Compute Drop Fibre
            drop_result_tuple = compute_drop_fibre(client, duct_banks, params)
            drop_fibre, duct_point, _, exceeds_max_drop = drop_result_tuple
            if not drop_fibre or not duct_point:
                logger.warning(f"Zone {zone_idx + 1}: Failed drop fibre for client {client.id}.")
                return client_result # Return failure

            # Compute Distribution Cable
            dist_result_tuple = compute_distribution_cable(
                duct_point, manholes, client.id, params, G
            )
            dist_cable, chosen_manhole_np, dist_fallback_used = dist_result_tuple
            if not dist_cable or not chosen_manhole_np:
                 logger.warning(f"Zone {zone_idx + 1}: Failed distribution cable for client {client.id}.")
                 return client_result # Return failure

            manhole_node_id = chosen_manhole_np[0]

            # Compute Splice Allocation String (placeholder)
            splice_str = compute_splice_allocation(client, manhole_node_id, agg_node_id, params)

            # Collect results for this client
            client_result.update({
                "success": True,
                "drop_fibre": drop_fibre,
                "distribution_cable": dist_cable,
                "splice_allocation": splice_str,
                "manhole_node_id": manhole_node_id,
                "exceeds_max_drop": exceeds_max_drop,
                "dist_fallback_used": dist_fallback_used
            })
            return client_result
        except Exception as e:
            logger.error(f"Zone {zone_idx + 1}: Unhandled error processing client {client.id}: {e}", exc_info=True)
            return client_result # Return failure


    # Execute client processing in parallel
    # Adjust max_workers based on typical workload and available cores
    num_workers = max(1, min(os.cpu_count() or 1, len(clients))) # Scale workers, max = core count
    with ThreadPoolExecutor(max_workers=num_workers, thread_name_prefix=f"ClientWorker_Z{zone_idx+1}") as executor:
        future_to_client = {executor.submit(process_single_client_connection, client): client for client in clients}

        for future in as_completed(future_to_client):
            client = future_to_client[future]
            try:
                result = future.result()
                if result and result["success"]:
                    # Aggregate successful results
                    results["drop_fibres"].append(result["drop_fibre"])
                    results["distribution_cables"].append(result["distribution_cable"])
                    results["splice_allocations"][result["client_id"]] = result["splice_allocation"]
                    results["client_to_manhole_map"][result["client_id"]] = result["manhole_node_id"]
                    results["used_manholes"].add(result["manhole_node_id"])

                    # Update aggregated stats
                    results["stats_drop_length"] += result["drop_fibre"]["properties"]["length_m"]
                    results["stats_dist_length"] += result["distribution_cable"]["properties"]["length_m"]
                    if result["exceeds_max_drop"]: results["stats_clients_over_max_drop"] += 1
                    if result["dist_fallback_used"]: results["stats_dist_routing_fallbacks"] += 1
                    results["processed_clients_count"] += 1
                else:
                     logger.warning(f"Zone {zone_idx + 1}: Processing failed for client {client.id} (result indicates failure).")
                     results["failed_clients_count"] += 1

            except Exception as exc:
                # Exception occurred during the execution of the task itself
                logger.error(f"Zone {zone_idx + 1}: Exception processing client {client.id}: {exc}", exc_info=True)
                results["failed_clients_count"] += 1

    processing_time = time.time() - start_time
    logger.info(f"[Zone {zone_idx + 1}] Finished processing client connections in {processing_time:.2f}s. Success: {results['processed_clients_count']}, Failed: {results['failed_clients_count']}")
    return results


def _compute_feeder_cables(
    used_manholes: Set[NodeId],
    manhole_lookup: Dict[NodeId, Point], # Lookup for manhole geometries
    aggregation_node: NodePoint,
    G: nx.MultiDiGraph,
    params: NetworkParameters,
    zone_idx: int
) -> Dict[str, Any]:
    """
    Computes feeder cables from used manholes to the aggregation node.
    Uses parallel processing.
    Returns a dictionary containing feeder results.
    """
    logger.info(f"[Zone {zone_idx + 1}] Computing feeder cables for {len(used_manholes)} used manholes...")
    start_time = time.time()

    results = {
        "feeder_cables": [],
        "stats_feeder_length": 0.0,
        "stats_feeder_routing_fallbacks": 0,
    }

    # Prepare tasks for parallel execution
    feeder_tasks = []
    for mh_node_id in used_manholes:
         manhole_point = manhole_lookup.get(mh_node_id)
         if not manhole_point:
             logger.warning(f"Zone {zone_idx + 1}: Manhole node ID {mh_node_id} used by client but geometry not found in lookup. Skipping feeder.")
             continue
         manhole_np = (mh_node_id, manhole_point)
         feeder_tasks.append((manhole_np, aggregation_node)) # Task is tuple: (manhole_NodePoint, aggregation_NodePoint)

    # Define feeder computation task
    def compute_single_feeder(task_data):
        mh_np, agg_np = task_data
        try:
            feeder_result, feed_fallback = compute_feeder_cable(mh_np, agg_np, params, G)
            return {"success": True, "result": feeder_result, "fallback": feed_fallback}
        except Exception as e:
             logger.error(f"Zone {zone_idx + 1}: Error computing feeder for manhole {mh_np[0]}: {e}", exc_info=True)
             return {"success": False, "result": None, "fallback": False}

    # Execute feeder tasks in parallel
    num_workers = max(1, min(os.cpu_count() or 1, len(feeder_tasks)))
    with ThreadPoolExecutor(max_workers=num_workers, thread_name_prefix=f"FeederWorker_Z{zone_idx+1}") as executor:
        future_to_feeder = {executor.submit(compute_single_feeder, task): task for task in feeder_tasks}
        for future in as_completed(future_to_feeder):
            mh_np, _ = future_to_feeder[future] # Get manhole ID for logging if needed
            try:
                feeder_outcome = future.result()
                if feeder_outcome["success"] and feeder_outcome["result"]:
                    feeder_cable = feeder_outcome["result"]
                    results["feeder_cables"].append(feeder_cable)
                    results["stats_feeder_length"] += feeder_cable["properties"]["length_m"]
                    if feeder_outcome["fallback"]:
                        results["stats_feeder_routing_fallbacks"] += 1
                elif feeder_outcome["success"] is False:
                     # Error already logged within compute_single_feeder
                     logger.warning(f"Zone {zone_idx + 1}: Failed feeder computation for Manhole {mh_np[0]} was caught.")
                # else: successful but no result (shouldn't happen based on compute_feeder_cable logic)

            except Exception as exc:
                 logger.error(f"Zone {zone_idx + 1}: Exception retrieving result for feeder computation (Manhole {mh_np[0]}): {exc}", exc_info=True)


    processing_time = time.time() - start_time
    logger.info(f"[Zone {zone_idx + 1}] Computed {len(results['feeder_cables'])} feeder cables in {processing_time:.2f}s.")
    return results


def process_zone(zone_polygon: Polygon, params: NetworkParameters, zone_idx: int) -> Optional[Dict[str, Any]]:
    """
    Processes a single design zone: fetches data, designs network elements, calculates stats.
    Handles errors within the zone processing, returning None if critical steps fail early.
    Returns a dictionary containing the results for the zone or None on critical failure.
    """
    thread_name = threading.current_thread().name
    logger.info(f"[{thread_name}] ===== Starting processing for Zone {zone_idx + 1} =====")
    zone_start_time = time.time()

    # Initialize zone results structure
    zone_results: Dict[str, Any] = {
        "zone_index": zone_idx,
        "zone_geometry": zone_polygon,
        "statistics": DesignStatistics(), # Per-zone stats
        "client_points": [],
        "duct_banks": [],
        "manholes": [], # List[NodePoint] of *all* placed manholes in this zone
        "drop_fibres": [],
        "distribution_cables": [],
        "feeder_cables": [],
        "splice_allocations": {},
        "aggregation_node": None, # Optional[NodePoint]
        "graph": None, # nx.MultiDiGraph
        "success": False # Flag indicating successful completion of network design *within* the zone
    }
    zone_stats = zone_results["statistics"] # Shortcut

    try:
        # --- Step 1: Retrieve OSM Data (Graph and Buildings) ---
        logger.info(f"[{thread_name}] Zone {zone_idx + 1}: Retrieving OSM data...")
        G = retrieve_osm_data(zone_polygon) # Uses buffered polygon inside
        zone_results["graph"] = G # Store graph even if processing fails later
        if G is None or G.number_of_nodes() == 0:
             raise ValueError("Failed to retrieve a valid network graph.")

        client_points = get_building_clients(zone_polygon)
        zone_results["client_points"] = client_points
        zone_stats.total_clients = len(client_points)

        if not client_points:
            logger.warning(f"[{thread_name}] Zone {zone_idx + 1}: No building clients found. Zone processed but no network designed.")
            zone_results["success"] = True # Mark as 'success' in terms of processing, but no network
            logger.info(f"[{thread_name}] Finished Zone {zone_idx + 1} (No Clients) in {time.time() - zone_start_time:.2f}s.")
            return zone_results # Return basic zone info

        logger.info(f"[{thread_name}] Zone {zone_idx + 1}: Found {zone_stats.total_clients} clients.")

        # --- Step 2: Generate Base Infrastructure (Ducts, Manholes, Aggregation Node) ---
        logger.info(f"[{thread_name}] Zone {zone_idx + 1}: Generating base infrastructure...")
        duct_banks = generate_duct_banks(G, params)
        zone_results["duct_banks"] = duct_banks
        if not duct_banks:
             # Possible if graph has nodes but no usable edges/geometry
             logger.warning(f"[{thread_name}] Zone {zone_idx + 1}: No duct banks generated. Cannot create drops.")
             raise ValueError("Duct bank generation failed, cannot proceed.")

        manholes: List[NodePoint] = optimize_manhole_placement(G, client_points, params)
        zone_results["manholes"] = manholes
        zone_stats.total_manholes = len(manholes) # Temp count for zone, overall count done later
        if not manholes:
             logger.warning(f"[{thread_name}] Zone {zone_idx + 1}: Failed to place any manholes. Cannot route distribution.")
             raise ValueError("Manhole placement failed, cannot proceed.")
        logger.info(f"[{thread_name}] Zone {zone_idx + 1}: Placed {zone_stats.total_manholes} potential manhole locations.")
        manhole_lookup: Dict[NodeId, Point] = {np[0]: np[1] for np in manholes}

        aggregation_node: Optional[NodePoint] = generate_aggregation_node(G, client_points, params)
        zone_results["aggregation_node"] = aggregation_node
        if not aggregation_node:
             logger.warning(f"[{thread_name}] Zone {zone_idx + 1}: Failed to determine aggregation node. Cannot route feeders.")
             raise ValueError("Aggregation node selection failed, cannot proceed.")
        logger.info(f"[{thread_name}] Zone {zone_idx + 1}: Aggregation node selected: Node {aggregation_node[0]}.")


        # --- Step 3: Process Client Connections (Parallelized) ---
        client_connection_results = _process_client_connections(
            client_points, duct_banks, manholes, aggregation_node, G, params, zone_idx
        )

        # Update zone results from client processing
        zone_results["drop_fibres"] = client_connection_results["drop_fibres"]
        zone_results["distribution_cables"] = client_connection_results["distribution_cables"]
        zone_results["splice_allocations"] = client_connection_results["splice_allocations"]
        used_manholes = client_connection_results["used_manholes"] # Get set of used MH IDs

        # Update zone statistics from client processing
        zone_stats.total_drop_length = client_connection_results["stats_drop_length"]
        zone_stats.total_distribution_length = client_connection_results["stats_dist_length"]
        zone_stats.clients_over_max_drop = client_connection_results["stats_clients_over_max_drop"]
        zone_stats.routing_fallbacks += client_connection_results["stats_dist_routing_fallbacks"] # Add dist fallbacks

        if client_connection_results["processed_clients_count"] == 0 and len(client_points) > 0:
             logger.error(f"[{thread_name}] Zone {zone_idx + 1}: Failed to connect any clients successfully. Aborting feeder stage for this zone.")
             raise ValueError("No clients connected successfully.")


        # --- Step 4: Compute Feeder Cables (Parallelized for Used Manholes) ---
        if used_manholes:
            feeder_results = _compute_feeder_cables(
                used_manholes, manhole_lookup, aggregation_node, G, params, zone_idx
            )
            zone_results["feeder_cables"] = feeder_results["feeder_cables"]
            # Update zone statistics from feeder processing
            zone_stats.total_feeder_length = feeder_results["stats_feeder_length"]
            zone_stats.routing_fallbacks += feeder_results["stats_feeder_routing_fallbacks"] # Add feeder fallbacks
        else:
            logger.info(f"[{thread_name}] Zone {zone_idx + 1}: No manholes were used by clients, skipping feeder cable computation.")


        # --- Step 5: Finalize Zone ---
        zone_stats.total_splitters = len(used_manholes) # Count of manholes actually connected
        zone_results["success"] = True # Mark zone as successfully processed
        processing_time = time.time() - zone_start_time
        logger.info(f"[{thread_name}] ===== Finished processing Zone {zone_idx + 1} successfully in {processing_time:.2f}s =====")
        return zone_results

    except ValueError as ve:
        # Catch configuration or critical data processing errors within the zone
        logger.error(f"[{thread_name}] CRITICAL VALUE ERROR processing Zone {zone_idx + 1}: {ve}. Zone processing aborted.")
        zone_results["success"] = False
        return zone_results # Return partially filled dict marked as unsuccessful
    except Exception as e:
        # Catch any other unexpected error during the main zone processing flow
        logger.exception(f"[{thread_name}] UNEXPECTED ERROR processing Zone {zone_idx + 1}: {e}. Zone processing aborted.")
        zone_results["success"] = False
        return zone_results


def design_fibre_network(place_name: str, params: NetworkParameters, output_dir: str = DEFAULT_OUTPUT_DIR) -> Optional[Dict]:
    """
    Main orchestrator function to design the fibre network.
    Handles area definition, subdivision, zone processing (parallel), aggregation, and reporting.
    Returns a dictionary containing paths to outputs and final stats, or None on critical failure.
    """
    overall_stats = DesignStatistics()
    main_start_time = time.time()
    overall_stats.start_time = main_start_time # Reset start time for overall process
    logger.info(f"=======================================================")
    logger.info(f"=== Starting Fibre Network Design (v{VERSION}) ===")
    logger.info(f"=== Target Area: {place_name} ===")
    # logger.info(f"=== Parameters: {params.to_dict()} ===") # Can be verbose
    logger.info(f"=== Output Directory: {os.path.abspath(output_dir)} ===")
    logger.info(f"=======================================================")

    # --- Preparation: Output Dir, Area Polygon, Subdivision ---
    try:
        os.makedirs(output_dir, exist_ok=True)
        logger.info("Step 1: Defining study area polygon...")
        area_polygon = get_polygon_from_place(place_name)
        logger.info("Step 1: Study area polygon obtained.")

        logger.info("Step 2: Checking area size and subdividing if necessary...")
        design_zones = subdivide_area_if_needed(area_polygon, params)
        if not design_zones:
            raise ValueError("Area subdivision failed or resulted in no valid zones.")
        overall_stats.zones_count = len(design_zones)
        logger.info(f"Step 2: Design will be processed across {len(design_zones)} zone(s).")

    except (ValueError, ProjError, Exception) as prep_e:
         logger.critical(f"Failed during preparation stage: {prep_e}", exc_info=True)
         return None


    # --- Zone Processing (Parallel or Sequential) ---
    logger.info("Step 3: Processing individual design zones...")
    zone_results_list: List[Dict] = []
    processed_zone_indices: Set[int] = set()
    failed_zone_indices: List[int] = []

    # Determine parallel execution based on zone count and CPU availability
    # Allow disabling parallel for debugging: use_parallel_zones = False
    use_parallel_zones = len(design_zones) > 1 and (os.cpu_count() or 1) > 1

    if use_parallel_zones:
        # Use slightly fewer workers than cores to leave room for system/other tasks
        num_zone_workers = max(1, (os.cpu_count() or 2) - 1)
        logger.info(f"Processing {len(design_zones)} zones in parallel (max_workers={num_zone_workers})...")
        with ThreadPoolExecutor(max_workers=num_zone_workers, thread_name_prefix='ZoneWorker') as executor:
            future_to_zone_idx = {
                executor.submit(process_zone, zone_poly, params, idx): idx
                for idx, zone_poly in enumerate(design_zones)
            }
            for future in as_completed(future_to_zone_idx):
                zone_idx = future_to_zone_idx[future]
                try:
                    result_dict = future.result() # Get the result dictionary from the thread
                    # Check if result is valid and marked as success
                    if result_dict and isinstance(result_dict, dict):
                         zone_results_list.append(result_dict) # Append regardless of success for potential partial data
                         if result_dict.get("success"):
                              processed_zone_indices.add(zone_idx)
                         else:
                              logger.error(f"Zone {zone_idx + 1} processing function indicated failure.")
                              failed_zone_indices.append(zone_idx)
                    else:
                         # Function returned None or invalid type
                         logger.error(f"Zone {zone_idx + 1} processing failed critically or returned unexpected data ({type(result_dict)}).")
                         failed_zone_indices.append(zone_idx)

                except Exception as exc:
                    # Exception occurred during the execution of the process_zone task itself
                    logger.error(f"Zone {zone_idx + 1} generated an unexpected exception during execution: {exc}", exc_info=True)
                    failed_zone_indices.append(zone_idx)
    else:
        # Sequential processing
         logger.info(f"Processing {len(design_zones)} zone(s) sequentially...")
         for idx, zone_poly in enumerate(design_zones):
             result_dict = process_zone(zone_poly, params, idx)
             if result_dict and isinstance(result_dict, dict):
                 zone_results_list.append(result_dict)
                 if result_dict.get("success"):
                     processed_zone_indices.add(idx)
                 else:
                     logger.error(f"Zone {idx + 1} processing function indicated failure.")
                     failed_zone_indices.append(idx)
             else:
                 logger.error(f"Zone {idx + 1} processing failed critically or returned unexpected data.")
                 failed_zone_indices.append(idx)

    # --- Aggregation and Reporting ---
    overall_stats.failed_zones = len(failed_zone_indices)
    successful_zone_results = [res for res in zone_results_list if res.get("success")]

    if not successful_zone_results:
        logger.error("No zones were successfully processed. Network design cannot be completed.")
        # Consider saving partial results or logs from failed zones if available in zone_results_list
        return None

    logger.info(f"Step 3: Finished processing zones. Success: {len(successful_zone_results)}, Failed: {overall_stats.failed_zones}.")
    logger.info("Step 4: Aggregating results from successfully processed zones...")

    # Initialize aggregation containers
    all_clients: List[BuildingClient] = []
    all_duct_banks: List[GeoDict] = []
    unique_duct_ids: Set[str] = set()
    all_manholes: List[NodePoint] = [] # Unique placed manholes across all zones
    unique_manhole_ids: Set[NodeId] = set()
    all_drop_fibres: List[GeoDict] = []
    all_dist_cables: List[GeoDict] = []
    all_feeder_cables: List[GeoDict] = []
    unique_feeder_keys: Set[Tuple[NodeId, NodeId]] = set() # (mh_node_id, agg_node_id)
    all_splice_allocations: Dict[int, str] = {}
    all_aggregation_nodes: List[NodePoint] = [] # Unique aggregation nodes
    unique_agg_node_ids: Set[NodeId] = set()
    all_zone_geometries: List[Polygon] = [res["zone_geometry"] for res in zone_results_list if "zone_geometry" in res] # All attempted zones

    # Loop through successful zone results and aggregate
    for result in successful_zone_results:
        # Aggregate Stats from successful zones
        stats = result.get("statistics")
        if stats:
            overall_stats.total_clients += stats.total_clients
            overall_stats.total_drop_length += stats.total_drop_length
            overall_stats.total_distribution_length += stats.total_distribution_length
            overall_stats.total_feeder_length += stats.total_feeder_length
            overall_stats.total_splitters += stats.total_splitters # Sum of used manholes per zone
            overall_stats.clients_over_max_drop += stats.clients_over_max_drop
            overall_stats.routing_fallbacks += stats.routing_fallbacks
        else:
             logger.warning(f"Zone {result.get('zone_index', -1)+1}: Statistics object missing in successful results.")

        # Aggregate Elements from successful zones
        all_clients.extend(result.get("client_points", []))
        all_drop_fibres.extend(result.get("drop_fibres", []))
        all_dist_cables.extend(result.get("distribution_cables", []))
        all_splice_allocations.update(result.get("splice_allocations", {}))

        # Aggregate unique infrastructure elements across successful zones
        for duct in result.get("duct_banks", []):
            duct_id = duct.get("properties", {}).get("id")
            if duct_id and duct_id not in unique_duct_ids:
                all_duct_banks.append(duct); unique_duct_ids.add(duct_id)
        for mh_np in result.get("manholes", []):
            if mh_np[0] not in unique_manhole_ids:
                all_manholes.append(mh_np); unique_manhole_ids.add(mh_np[0])
        for feeder in result.get("feeder_cables", []):
             props = feeder.get("properties", {})
             feeder_key = (props.get("manhole_node_id"), props.get("aggregation_node_id"))
             if None not in feeder_key and feeder_key not in unique_feeder_keys:
                 all_feeder_cables.append(feeder); unique_feeder_keys.add(feeder_key)
        agg_np = result.get("aggregation_node")
        if agg_np and agg_np[0] not in unique_agg_node_ids:
             all_aggregation_nodes.append(agg_np); unique_agg_node_ids.add(agg_np[0])

    # Refine overall counts based on unique aggregated elements
    overall_stats.total_manholes = len(all_manholes) # Final count of unique manholes placed

    logger.info("Step 4: Aggregation complete.")
    logger.info(f"Step 5: Generating KML and CSV output files...")
    safe_place_name = _safe_filename(place_name)

    # 5. Generate KML Output
    kml_path = None
    try:
        kml_filename = f"{safe_place_name}_fibre_network_v{VERSION}{KML_EXT}"
        kml_path = os.path.join(output_dir, kml_filename)
        kml, folders, styles = initialize_kml(place_name)

        # Add Zone Boundaries (all attempted zones)
        for i, zone_geom in enumerate(all_zone_geometries):
             # Use original index if available, otherwise sequential
             zone_id = next((res['zone_index'] + 1 for res in zone_results_list if res.get('zone_geometry') == zone_geom), i + 1)
             zone_status = "Processed" if zone_id - 1 in processed_zone_indices else "Failed"
             zone_props = {"id": zone_id, "name": f"Zone {zone_id} ({zone_status})"}
             add_element_to_kml(folders["zones"], STYLE_ZONE, {"geometry": zone_geom, "properties": zone_props}, styles)

        # Add aggregated elements from successful zones
        for client in all_clients: add_element_to_kml(folders["clients"], STYLE_CLIENT, {"geometry": client.geometry, "properties": client.to_dict()}, styles)
        for duct in all_duct_banks: add_element_to_kml(folders["duct_banks"], STYLE_DUCT, duct, styles)
        for mh_np in all_manholes: add_element_to_kml(folders["manholes"], STYLE_MANHOLE, {"geometry": mh_np[1], "properties": {"id": mh_np[0], "name":f"Manhole MH_{mh_np[0]}", "node_id": mh_np[0]}}, styles)
        for drop in all_drop_fibres: add_element_to_kml(folders["drop_fibres"], STYLE_DROP, drop, styles)
        for dist in all_dist_cables: add_element_to_kml(folders["distribution_cables"], STYLE_DISTRIBUTION, dist, styles)
        for feeder in all_feeder_cables: add_element_to_kml(folders["feeder_cables"], STYLE_FEEDER, feeder, styles)
        for agg_np in all_aggregation_nodes: add_element_to_kml(folders["aggregation_nodes"], STYLE_AGGREGATION, {"geometry": agg_np[1], "properties": {"id": agg_np[0], "name":f"Aggregation AGG_{agg_np[0]}", "node_id": agg_np[0]}}, styles)

        kml.save(kml_path)
        logger.info(f"KML output saved to: {kml_path}")
    except Exception as kml_e:
         logger.error(f"Failed to generate KML file: {kml_e}", exc_info=True)


    # 6. Generate CSV Reports
    client_report_path = None
    infra_report_path = None
    try:
        client_report_path = generate_client_report(all_clients, all_drop_fibres, all_dist_cables, all_splice_allocations, output_dir, place_name)
    except Exception as csv_e:
        logger.error(f"Failed to generate Client CSV report: {csv_e}", exc_info=True)
    try:
        infra_report_path = generate_infrastructure_report(all_duct_banks, all_manholes, all_feeder_cables, all_aggregation_nodes, params, output_dir, place_name)
    except Exception as csv_e:
        logger.error(f"Failed to generate Infrastructure CSV report: {csv_e}", exc_info=True)


    # 7. Final Statistics Summary and Text Report
    summary_path = None
    try:
        logger.info("Step 6: Calculating final costs and generating summary report...")
        summary_str = overall_stats.generate_summary(params)
        logger.info("\n" + summary_str) # Log summary to console/file

        # Save summary to text file
        summary_filename = f"{safe_place_name}_summary_v{VERSION}{TXT_EXT}"
        summary_path = os.path.join(output_dir, summary_filename)
        with open(summary_path, 'w', encoding='utf-8') as f:
            f.write(f"{APP_NAME} Summary (v{VERSION})\n")
            f.write(f"Area: {place_name}\n")
            f.write(f"Generated: {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n")
            f.write("Parameters Used:\n")
            f.write(json.dumps(params.to_dict(), indent=2))
            f.write("\n\n" + "="*20 + " Summary Statistics " + "="*20 + "\n")
            f.write(summary_str)
        logger.info(f"Summary report saved to: {summary_path}")

    except Exception as summary_e:
        logger.error(f"Failed to generate final summary: {summary_e}", exc_info=True)


    # --- Completion ---
    end_time = time.time()
    logger.info(f"=========================================================")
    logger.info(f"=== Fibre Network Design Completed in {end_time - main_start_time:.2f} seconds ===")
    logger.info(f"=========================================================")

    # Return dictionary with paths to generated files and final stats object
    return {
        "kml_path": kml_path,
        "client_report_path": client_report_path,
        "infrastructure_report_path": infra_report_path,
        "summary_path": summary_path,
        "statistics": overall_stats, # The dataclass instance
        "parameters": params.to_dict() # The parameters used for the run
    }


# ---------------------------
# GUI IMPLEMENTATION (Tkinter)
# ---------------------------
class FibreNetworkDesignerGUI:
    """GUI for the Fibre Network Design Tool."""
    def __init__(self, root: tk.Tk):
        self.root = root
        self.root.title(f"{APP_NAME} v{VERSION}")
        # Set minimum size, allow expansion
        self.root.minsize(800, 600)
        self.root.columnconfigure(0, weight=1)
        self.root.rowconfigure(0, weight=1)


        # Style configuration
        style = ttk.Style(self.root)
        try:
             # Use 'clam' for better cross-platform look, fallback if needed
             style.theme_use('clam')
        except tk.TclError:
             logger.warning("Clam theme not available, using default Tk theme.")
             style.theme_use('default')
        # Define custom style for Run button
        style.configure('Accent.TButton', foreground='white', background='#007bff', font=('TkDefaultFont', 10, 'bold'))
        style.map('Accent.TButton', background=[('active', '#0056b3'), ('disabled', '#cccccc')])


        # Variables
        self.place_var = tk.StringVar()
        self.output_dir_var = tk.StringVar(value=os.path.abspath(DEFAULT_OUTPUT_DIR))
        self.status_var = tk.StringVar(value="Ready. Enter place name and parameters.")
        self.progress_var = tk.DoubleVar(value=0.0)
        self.params = NetworkParameters() # Load default parameters
        self.is_running = False # Flag to prevent multiple runs

        # Parameter storage for GUI elements {param_name: tk_variable}
        self.param_widgets: Dict[str, Union[tk.StringVar, tk.BooleanVar, tk.IntVar, tk.DoubleVar]] = {}

        self.create_widgets()
        self.update_gui_from_params() # Populate GUI with initial params

        # Configure graceful shutdown
        self.root.protocol("WM_DELETE_WINDOW", self.on_close)


    def create_widgets(self):
        """Create GUI widgets."""
        main_frame = ttk.Frame(self.root, padding="10")
        main_frame.grid(row=0, column=0, sticky="nsew")
        # Configure grid weights for responsiveness
        main_frame.columnconfigure(0, weight=1) # Parameters + Log column
        main_frame.columnconfigure(1, weight=0) # Scrollbar column for Params
        main_frame.rowconfigure(0, weight=0)    # Input frame
        main_frame.rowconfigure(1, weight=2)    # Parameters section (takes more space)
        main_frame.rowconfigure(2, weight=1)    # Log area (takes less space)
        main_frame.rowconfigure(3, weight=0)    # Buttons
        main_frame.rowconfigure(4, weight=0)    # Status bar


        # --- Input Section ---
        input_frame = ttk.LabelFrame(main_frame, text="Input & Output", padding="10")
        input_frame.grid(row=0, column=0, columnspan=2, sticky="ew", padx=5, pady=(5, 10))
        input_frame.columnconfigure(1, weight=1) # Allow entry field to expand

        ttk.Label(input_frame, text="Place Name:").grid(row=0, column=0, sticky=tk.W, padx=5, pady=5)
        self.place_entry = ttk.Entry(input_frame, textvariable=self.place_var, width=60)
        self.place_entry.grid(row=0, column=1, sticky="ew", padx=5, pady=5)
        # Tooltip (requires external lib or manual implementation, skip for now)
        # ToolTip(self.place_entry, "Enter the location (e.g., 'City, Country' or 'Address')")

        ttk.Label(input_frame, text="Output Directory:").grid(row=1, column=0, sticky=tk.W, padx=5, pady=5)
        self.output_entry = ttk.Entry(input_frame, textvariable=self.output_dir_var, width=60)
        self.output_entry.grid(row=1, column=1, sticky="ew", padx=5, pady=5)
        self.browse_button = ttk.Button(input_frame, text="Browse...", command=self.browse_output_dir)
        self.browse_button.grid(row=1, column=2, padx=5, pady=5)


        # --- Parameters Section (Scrollable Frame) ---
        param_container = ttk.Frame(main_frame) # Container for canvas and scrollbar
        param_container.grid(row=1, column=0, columnspan=2, sticky="nsew", padx=5, pady=5)
        param_container.columnconfigure(0, weight=1)
        param_container.rowconfigure(0, weight=1)
        style = ttk.Style()
        param_canvas = tk.Canvas(param_container, borderwidth=0, highlightthickness=0, background=style.lookup('TFrame', 'background'))
        param_scrollbar = ttk.Scrollbar(param_container, orient="vertical", command=param_canvas.yview)
        # Use Frame inside the canvas; LabelFrame caused resizing issues
        self.param_scrollable_frame = ttk.Frame(param_canvas, padding="10")

        self.param_scrollable_frame.bind(
            "<Configure>",
            lambda e: param_canvas.configure( scrollregion=param_canvas.bbox("all") )
        )
        param_canvas_window = param_canvas.create_window((0, 0), window=self.param_scrollable_frame, anchor="nw")
        param_canvas.configure(yscrollcommand=param_scrollbar.set)

        # Adjust canvas window width on canvas resize
        param_canvas.bind('<Configure>', lambda e: param_canvas.itemconfig(param_canvas_window, width=e.width))

        param_canvas.grid(row=0, column=0, sticky="nsew")
        param_scrollbar.grid(row=0, column=1, sticky="ns")

        # Populate parameters inside self.param_scrollable_frame
        self._create_param_widgets(self.param_scrollable_frame)


        # --- Log Display Section ---
        log_frame = ttk.LabelFrame(main_frame, text="Log Output", padding="10")
        log_frame.grid(row=2, column=0, columnspan=2, sticky="nsew", padx=5, pady=5)
        log_frame.rowconfigure(0, weight=1)
        log_frame.columnconfigure(0, weight=1)

        self.log_text = tk.Text(log_frame, height=10, width=80, wrap=tk.WORD, state=tk.DISABLED, relief=tk.SUNKEN, borderwidth=1, font=("Courier New", 9))
        log_scrollbar = ttk.Scrollbar(log_frame, command=self.log_text.yview)
        self.log_text.config(yscrollcommand=log_scrollbar.set)
        self.log_text.grid(row=0, column=0, sticky="nsew", padx=(0, 5))
        log_scrollbar.grid(row=0, column=1, sticky="ns")

        # Redirect logger output to GUI
        self.setup_logger_redirect()


        # --- Action Buttons ---
        button_frame = ttk.Frame(main_frame, padding="5")
        button_frame.grid(row=3, column=0, columnspan=2, sticky="ew", pady=(10, 5))
        # Use packing within button_frame for simpler layout if centering is not complex
        self.run_button = ttk.Button(button_frame, text="Design Network", command=self.run_design, style='Accent.TButton', width=15)
        self.save_params_button = ttk.Button(button_frame, text="Save Params", command=self.save_parameters)
        self.load_params_button = ttk.Button(button_frame, text="Load Params", command=self.load_parameters)
        self.exit_button = ttk.Button(button_frame, text="Exit", command=self.on_close)

        self.run_button.pack(side=tk.LEFT, padx=5, pady=5)
        self.save_params_button.pack(side=tk.LEFT, padx=5, pady=5)
        self.load_params_button.pack(side=tk.LEFT, padx=5, pady=5)
        self.exit_button.pack(side=tk.RIGHT, padx=5, pady=5)


        # --- Status Bar ---
        status_frame = ttk.Frame(main_frame, relief=tk.SUNKEN, padding="2")
        status_frame.grid(row=4, column=0, columnspan=2, sticky="ew")
        status_frame.columnconfigure(1, weight=1) # Allow status label to expand

        ttk.Label(status_frame, text="Status:").grid(row=0, column=0, padx=(5, 0), sticky='w')
        self.status_label = ttk.Label(status_frame, textvariable=self.status_var, anchor='w')
        self.status_label.grid(row=0, column=1, padx=5, sticky='ew')
        self.progress = ttk.Progressbar(status_frame, variable=self.progress_var, length=200, mode='determinate')
        self.progress.grid(row=0, column=2, padx=5, sticky='e')


    def _create_param_widgets(self, parent_frame):
        """Creates widgets for parameters inside the specified scrollable frame."""
        params_dict = self.params.to_dict()
        # Define order and grouping for parameters
        param_groups = {
            "Core Parameters": ['max_clients_per_splitter', 'max_drop_length', 'max_zone_area'],
            "Fibre Types (Strains)": ['drop_fibre_strains', 'distribution_fibre_strains', 'feeder_fibre_strains'],
            "Optimization & Routing": ['optimize_manholes', 'manhole_clustering_distance', 'use_network_routing', 'use_centrality_for_aggregation'],
            "Cost Parameters": [
                'budget_trench_per_m', 'budget_duct_per_m', 'budget_manhole',
                'budget_drop_fibre_per_m', 'budget_dist_fibre_per_m', 'budget_feeder_fibre_per_m',
                'budget_splitter_cost', 'budget_terminal_box_per_client', 'budget_splicing_per_connection'
            ]
        }
        # Flatten list for processing order, ensuring all params are included
        ordered_params = []
        processed = set()
        all_known_params = set(params_dict.keys())
        for group_name, params_in_group in param_groups.items():
             for name in params_in_group:
                 if name in all_known_params and name not in processed:
                     ordered_params.append(name)
                     processed.add(name)
        # Add any params not explicitly grouped (shouldn't happen if groups are maintained)
        for name in all_known_params:
             if name not in processed:
                 logger.warning(f"Parameter '{name}' not assigned to a GUI group. Adding to end.")
                 ordered_params.append(name)

        # Layout within the parent_frame (which is the scrollable Frame)
        parent_frame.columnconfigure(1, weight=0) # Label col 1
        parent_frame.columnconfigure(2, weight=1) # Entry col 1
        parent_frame.columnconfigure(3, weight=0) # Label col 2
        parent_frame.columnconfigure(4, weight=1) # Entry col 2

        current_row_in_parent = 0

        # Function to create entry with validation (improved)
        def create_validated_entry(target, var, type_class):
            validation_mode = 'key' # Validate on each keystroke
            validate_cmd = None
            invalid_cmd = (target.register(lambda: target.bell()),) # Ring bell on invalid input

            if type_class == int:
                def check_int(P): return P == "" or P.isdigit() or (P.startswith('-') and P[1:].isdigit())
                validate_cmd = (target.register(check_int), '%P')
            elif type_class == float:
                def check_float(P):
                    if P == "" or P == "." or P == "-" or P == "-.": return True
                    try: float(P); return True
                    except ValueError: return False
                validate_cmd = (target.register(check_float), '%P')
                # Note: This basic float check allows intermediate invalid states like '1.2.3'.
                # More robust validation might trigger on focus out or use a dedicated widget.

            entry = ttk.Entry(target, textvariable=var, width=12, validate=validation_mode,
                              validatecommand=validate_cmd, invalidcommand=invalid_cmd)
            return entry

        # Add widgets based on groups
        for group_name, params_in_group in param_groups.items():
            # Create a frame for the group
            group_frame = ttk.LabelFrame(parent_frame, text=group_name, padding="8")
            group_frame.grid(row=current_row_in_parent, column=0, columnspan=5, sticky="ew", padx=5, pady=(8,4))
            current_row_in_parent += 1
            # Configure columns inside the group frame
            group_frame.columnconfigure(1, weight=0) # Label col
            group_frame.columnconfigure(2, weight=1) # Entry col
            group_frame.columnconfigure(3, weight=0) # Label col 2
            group_frame.columnconfigure(4, weight=1) # Entry col 2

            row_in_group = 0
            col_in_group = 1 # Start at column 1 for label

            for name in ordered_params:
                if name not in params_in_group: continue # Only process params for this group

                value = params_dict.get(name)
                if value is None: continue # Skip if param somehow isn't in dict

                # Create Label
                label_text = name.replace('_', ' ').title() + ":"
                ttk.Label(group_frame, text=label_text).grid(row=row_in_group, column=col_in_group, sticky=tk.W, padx=5, pady=3)

                # Create Input Widget based on type
                widget = None
                param_type = type(value)
                if param_type is bool:
                    var = tk.BooleanVar(value=value)
                    style = ttk.Style()
                    widget = ttk.Checkbutton(group_frame, variable=var, style='Switch.TCheckbutton' if 'Switch.TCheckbutton' in style.element_names() else 'TCheckbutton') # Use Switch style if available
                    widget.grid(row=row_in_group, column=col_in_group + 1, sticky=tk.W, padx=5, pady=3)
                elif param_type is int:
                    var = tk.IntVar(value=value)
                    widget = create_validated_entry(group_frame, var, int)
                    widget.grid(row=row_in_group, column=col_in_group + 1, sticky=tk.W, padx=5, pady=3)
                elif param_type is float:
                    var = tk.DoubleVar(value=value)
                    widget = create_validated_entry(group_frame, var, float)
                    widget.grid(row=row_in_group, column=col_in_group + 1, sticky=tk.W, padx=5, pady=3)
                else: # Fallback for unexpected types (string?)
                    var = tk.StringVar(value=str(value))
                    widget = ttk.Entry(group_frame, textvariable=var, width=12)
                    widget.grid(row=row_in_group, column=col_in_group + 1, sticky=tk.W, padx=5, pady=3)

                self.param_widgets[name] = var

                # Move to next grid position (try to fill two columns per row)
                if col_in_group == 1: col_in_group = 3 # Move to second label column
                else: col_in_group = 1; row_in_group += 1 # Move back to first column, next row


    def setup_logger_redirect(self):
        """Redirect logger output to the GUI text widget."""
        class TextHandler(logging.Handler):
            def __init__(self, text_widget: tk.Text, root: tk.Tk):
                logging.Handler.__init__(self)
                self.text_widget = text_widget
                self.root = root
                self.level_map = { # Map levels to tags
                    logging.DEBUG: "debug",
                    logging.INFO: "info",
                    logging.WARNING: "warning",
                    logging.ERROR: "error",
                    logging.CRITICAL: "critical",
                }
                self.configure_tags()

            def configure_tags(self):
                self.text_widget.tag_configure("debug", foreground="gray")
                self.text_widget.tag_configure("info", foreground="black")
                self.text_widget.tag_configure("warning", foreground="#E69F00") # Orange/Yellow
                self.text_widget.tag_configure("error", foreground="#D55E00", font=("Courier New", 9, "bold")) # Vermillion/Red
                self.text_widget.tag_configure("critical", foreground="white", background="#D55E00", font=("Courier New", 9, "bold"))

            def emit(self, record: logging.LogRecord):
                try:
                    msg = self.format(record)
                    tag = self.level_map.get(record.levelno, "info")

                    def append_message():
                        if not self.text_widget.winfo_exists(): return # Widget destroyed
                        self.text_widget.configure(state='normal')
                        self.text_widget.insert(tk.END, msg + '\n', (tag,))
                        self.text_widget.configure(state='disabled')
                        self.text_widget.yview(tk.END) # Auto-scroll

                    # Schedule the GUI update in the main thread
                    if self.root.winfo_exists(): # Check if root window still exists
                         self.root.after(0, append_message)
                except Exception:
                    self.handleError(record)

        # Remove existing stream handlers to avoid duplicate console output
        for handler in logger.handlers[:]:
            if isinstance(handler, logging.StreamHandler) and not isinstance(handler, logging.FileHandler):
                 logger.removeHandler(handler)

        # Add our TextHandler if not already present
        if not any(isinstance(h, TextHandler) for h in logger.handlers):
            text_handler = TextHandler(self.log_text, self.root)
            text_handler.setFormatter(log_formatter)
            logger.addHandler(text_handler)
            logger.info("GUI Logger Initialized.")

    def browse_output_dir(self):
        """Open directory browser dialog for selecting the output directory."""
        initial_dir = self.output_dir_var.get() if os.path.isdir(self.output_dir_var.get()) else os.path.expanduser("~")
        directory = filedialog.askdirectory(initialdir=initial_dir, title="Select Output Directory")
        if directory: # Only update if a directory was selected
            self.output_dir_var.set(os.path.abspath(directory))
            logger.info(f"Output directory set to: {self.output_dir_var.get()}")

    def update_params_from_gui(self) -> bool:
        """Update the self.params object from GUI widgets. Returns True on success, False on validation error."""
        logger.debug("Updating parameters from GUI...")
        updated_values = {}
        errors = []
        for name, var in self.param_widgets.items():
            try:
                value = var.get()
                # Validate against expected type in NetworkParameters
                expected_type = type(getattr(self.params, name))
                if expected_type is int:
                    updated_values[name] = int(value)
                elif expected_type is float:
                    updated_values[name] = float(value)
                elif expected_type is bool:
                    updated_values[name] = bool(value) # Tkinter BooleanVar handles this
                else: # String or other? (Shouldn't happen with current params)
                    updated_values[name] = value # Assume correct type
            except (tk.TclError, ValueError, TypeError) as e:
                errors.append(f"Parameter '{name.replace('_', ' ').title()}': Invalid value '{var.get()}'. Expected {expected_type.__name__}. ({e})")

        if errors:
            error_msg = "Parameter validation failed:\n\n" + "\n".join(errors) + "\n\nPlease correct the values."
            logger.error(error_msg)
            messagebox.showerror("Parameter Error", error_msg)
            return False

        # If validation passed, try creating the NetworkParameters object
        try:
             self.params = NetworkParameters.from_dict(updated_values) # Use robust from_dict
             logger.info("Network parameters updated successfully from GUI.")
             return True
        except (ValueError, TypeError) as e: # Catch errors from from_dict/init
             logger.error(f"Error applying parameters after validation: {e}", exc_info=True)
             messagebox.showerror("Parameter Error", f"Failed to apply parameters:\n{e}\nPlease check input values.")
             return False

    def update_gui_from_params(self):
        """Update GUI widget values from the self.params object."""
        logger.debug("Updating GUI from parameters...")
        params_dict = self.params.to_dict()
        for name, var in self.param_widgets.items():
            if name in params_dict:
                try:
                    var.set(params_dict[name])
                except (tk.TclError, ValueError) as e:
                    logger.error(f"Error setting GUI widget '{name}' from param value '{params_dict[name]}': {e}")
            else:
                 logger.warning(f"Parameter '{name}' exists in GUI but not in NetworkParameters object.")

    def save_parameters(self):
        """Save current parameters (validated from GUI) to a JSON file."""
        if not self.update_params_from_gui():
            logger.warning("Parameter validation failed. Cannot save invalid parameters.")
            return

        file_path = filedialog.asksaveasfilename(
            title="Save Parameters As",
            defaultextension=PARAMS_EXT,
            filetypes=[("JSON Parameter Files", f"*{PARAMS_EXT}"), ("All files", "*.*")],
            initialdir=os.path.abspath("."),
            initialfile="network_params.json"
        )
        if file_path:
            try:
                with open(file_path, 'w', encoding='utf-8') as f:
                    json.dump(self.params.to_dict(), f, indent=4)
                logger.info(f"Parameters saved successfully to {file_path}")
                messagebox.showinfo("Success", f"Parameters saved to:\n{file_path}")
            except IOError as e:
                logger.error(f"Error saving parameters to {file_path}: {e}")
                messagebox.showerror("File Error", f"Error saving parameters:\n{e}")
            except Exception as e:
                logger.error(f"Unexpected error saving parameters: {e}", exc_info=True)
                messagebox.showerror("Error", f"Unexpected error saving parameters:\n{e}")

    def load_parameters(self):
        """Load parameters from a JSON file and update GUI."""
        file_path = filedialog.askopenfilename(
            title="Load Parameters",
            filetypes=[("JSON Parameter Files", f"*{PARAMS_EXT}"), ("All files", "*.*")],
            initialdir=os.path.abspath(".")
        )
        if file_path:
            try:
                with open(file_path, 'r', encoding='utf-8') as f:
                    loaded_param_dict = json.load(f)
                # Create NetworkParameters instance using robust from_dict
                self.params = NetworkParameters.from_dict(loaded_param_dict)
                # Update GUI from the loaded params object
                self.update_gui_from_params()
                logger.info(f"Parameters loaded successfully from {file_path}")
                messagebox.showinfo("Success", f"Parameters loaded from:\n{file_path}")
            except FileNotFoundError:
                 messagebox.showerror("Error", f"File not found:\n{file_path}")
            except json.JSONDecodeError as e:
                 messagebox.showerror("JSON Error", f"Invalid JSON file format:\n{e}\nFile: {file_path}")
            except (ValueError, TypeError) as e: # Catch errors from NetworkParameters.from_dict
                 messagebox.showerror("Parameter Error", f"Invalid parameter data in file:\n{e}\nFile: {file_path}")
            except IOError as e:
                 messagebox.showerror("File Error", f"Error reading parameter file:\n{e}")
            except Exception as e:
                 logger.error(f"Unexpected error loading parameters from {file_path}: {e}", exc_info=True)
                 messagebox.showerror("Error", f"Unexpected error loading parameters:\n{e}")


    def _set_gui_running_state(self, running: bool):
        """Enable/disable widgets based on running state."""
        self.is_running = running
        state = tk.DISABLED if running else tk.NORMAL
        widgets_to_toggle = [
            self.place_entry, self.output_entry, self.browse_button,
            self.run_button, self.save_params_button, self.load_params_button
        ]
        # Disable/enable parameter widgets
        for widget in self.param_scrollable_frame.winfo_children():
            if isinstance(widget, ttk.LabelFrame): # Handle group frames
                for sub_widget in widget.winfo_children():
                     if isinstance(sub_widget, (ttk.Entry, ttk.Checkbutton)):
                         try: sub_widget.config(state=state)
                         except tk.TclError: pass # Ignore if widget destroyed
            elif isinstance(widget, (ttk.Entry, ttk.Checkbutton)): # Direct children (if any)
                 try: widget.config(state=state)
                 except tk.TclError: pass

        # Disable/enable top-level buttons and inputs
        for widget in widgets_to_toggle:
             try: widget.config(state=state)
             except tk.TclError: pass # Ignore errors if widget is destroyed

    def run_design(self):
        """Initiate the network design process in a separate thread after validation."""
        if self.is_running:
             messagebox.showwarning("Busy", "A design process is already running.")
             return

        place_name = self.place_var.get().strip()
        output_dir = self.output_dir_var.get().strip()

        # --- Input Validation ---
        if not place_name:
            messagebox.showerror("Input Error", "Please enter a Place Name.")
            self.place_entry.focus_set()
            return
        if not output_dir:
             messagebox.showerror("Input Error", "Please specify an Output Directory.")
             self.output_entry.focus_set()
             return
        if not os.path.exists(os.path.dirname(output_dir)) and not os.path.exists(output_dir):
            if messagebox.askyesno("Create Directory?", f"The parent directory for '{output_dir}' does not exist. Create it?"):
                 try: os.makedirs(os.path.dirname(output_dir), exist_ok=True)
                 except Exception as e: messagebox.showerror("Error", f"Could not create directory: {e}"); return
            else: return
        # Validate parameters from GUI *before* starting thread
        if not self.update_params_from_gui():
             logger.warning("Design run aborted due to invalid parameters in GUI.")
             return

        # --- Prepare GUI for Running State ---
        self._set_gui_running_state(True)
        self.status_var.set(f"Starting design for '{place_name}'...")
        self.progress_var.set(0)
        self.progress.config(mode='indeterminate')
        self.progress.start(10)
        # Clear previous log entries
        self.log_text.config(state='normal')
        self.log_text.delete(1.0, tk.END)
        self.log_text.config(state='disabled')
        logger.info(f"{'='*25} New GUI Design Run {'='*25}")

        # --- Run Core Logic in Background Thread ---
        params_copy = NetworkParameters(**self.params.to_dict()) # Pass validated copy
        design_thread = threading.Thread(
            target=self._run_design_thread_wrapper,
            args=(place_name, params_copy, output_dir),
            name="DesignWorker",
            daemon=True
        )
        design_thread.start()


    def _run_design_thread_wrapper(self, place_name: str, params_obj: NetworkParameters, output_dir: str):
        """Wrapper executed in background thread. Calls main design func and handles results."""
        result_dict = None
        error_obj = None
        try:
            result_dict = design_fibre_network(place_name, params_obj, output_dir)
        except Exception as e:
            logger.critical(f"Unhandled exception in design thread: {e}", exc_info=True)
            error_obj = e
        finally:
             # Schedule GUI update back on the main thread using root.after()
             if self.root.winfo_exists(): # Ensure root window wasn't closed
                  self.root.after(0, self._update_gui_on_completion, result_dict, error_obj)

    def _update_gui_on_completion(self, result: Optional[Dict], error: Optional[Exception]):
        """Updates GUI after design thread finishes. MUST run in main GUI thread via root.after()."""
        # --- Reset GUI Running State ---
        self.progress.stop()
        self.progress.config(mode='determinate')
        self._set_gui_running_state(False)

        # --- Process Results / Errors ---
        if error:
            self.status_var.set(f"Design failed! Check log.")
            self.progress_var.set(0)
            messagebox.showerror("Design Error", f"Network design encountered an error:\n\n{type(error).__name__}: {error}\n\nCheck log file for details.")
        elif result and isinstance(result, dict) and result.get("statistics"):
            self.status_var.set("Design completed successfully!")
            self.progress_var.set(100)
            stats = result.get("statistics")
            # Format concise success message
            cost_str = f"{stats.total_cost:,.2f}" if stats.total_cost is not None else "N/A"
            total_fibre_m = stats.total_drop_length + stats.total_distribution_length + stats.total_feeder_length
            success_msg = (
                f"Network design complete for '{self.place_var.get()}'.\n\n"
                f"Summary:\n"
                f"  Clients: {stats.total_clients}\n"
                f"  Zones: {stats.zones_count} (Failed: {stats.failed_zones})\n"
                f"  Manholes Placed: {stats.total_manholes}\n"
                f"  Splitters Used: {stats.total_splitters}\n"
                f"  Total Fibre Length: {total_fibre_m:,.1f} m\n"
                f"  Estimated Cost: {cost_str}\n\n"
                f"Output files saved in:\n{self.output_dir_var.get()}"
            )
            messagebox.showinfo("Design Complete", success_msg)
        else:
             # Finished without exception, but no valid result/stats
             self.status_var.set("Design finished, but no valid results produced.")
             self.progress_var.set(0)
             logger.error("Design process finished but returned None or invalid result structure.")
             messagebox.showwarning("Design Finished", "Design process finished but did not produce expected results.\nCheck log file for potential issues.")

    def on_close(self):
        """Handle window closing event."""
        if self.is_running:
            if messagebox.askyesno("Exit Confirmation", "A design process is currently running. Are you sure you want to exit? The process will be terminated."):
                # Add any necessary cleanup here if threads need explicit stopping
                self.root.destroy()
            else:
                return # Don't close if user cancels
        else:
            self.root.destroy()


# ---------------------------
# CLI IMPLEMENTATION
# ---------------------------
def run_cli(args: argparse.Namespace):
    """Run the tool in Command Line Interface (CLI) mode."""
    place_name = args.place
    output_dir = args.output

    print(f"\n--- {APP_NAME} v{VERSION} (CLI Mode) ---")
    logger.info(f"CLI Mode Execution Started.")
    logger.info(f"Place Name: '{place_name}'")
    logger.info(f"Output Directory: '{os.path.abspath(output_dir)}'")

    try:
        # --- Parameter Handling ---
        params = NetworkParameters() # Start with defaults
        if args.params_file:
            logger.info(f"Attempting to load parameters from: {args.params_file}")
            try:
                with open(args.params_file, 'r', encoding='utf-8') as f:
                    params = NetworkParameters.from_dict(json.load(f)) # Use robust loader
                logger.info(f"Parameters loaded successfully from {args.params_file}.")
            except FileNotFoundError: logger.error(f"Parameter file not found: {args.params_file}. Using defaults."); print(f"\nError: Parameter file not found: {args.params_file}", file=sys.stderr); return 1
            except json.JSONDecodeError as e: logger.error(f"Invalid JSON in parameter file {args.params_file}: {e}. Using defaults."); print(f"\nError: Invalid JSON: {e}", file=sys.stderr); return 1
            except (ValueError, TypeError) as e: logger.error(f"Invalid parameter data in file {args.params_file}: {e}. Using defaults."); print(f"\nError: Invalid parameter data: {e}", file=sys.stderr); return 1
            except IOError as e: logger.error(f"Cannot read parameter file {args.params_file}: {e}. Using defaults."); print(f"\nError: Cannot read file: {e}", file=sys.stderr); return 1
            except Exception as e: logger.error(f"Unexpected error loading params: {e}. Using defaults.", exc_info=True); print(f"\nError: Unexpected: {e}", file=sys.stderr); return 1

        # Override specific parameters from CLI flags
        overridden_params = []
        param_override_mapping = {
             'max_clients_per_splitter': 'max_clients_per_splitter',
             'max_drop_length': 'max_drop_length',
             'max_zone_area': 'max_zone_area',
             'manhole_cluster_distance': 'manhole_clustering_distance',
             'disable_manhole_optimization': ('optimize_manholes', False), # (attribute, value_if_flag_present)
             'disable_network_routing': ('use_network_routing', False),
             'disable_centrality': ('use_centrality_for_aggregation', False)
        }
        current_params_dict = params.to_dict()

        for arg_key, mapping in param_override_mapping.items():
            cli_value = getattr(args, arg_key, None)
            if cli_value is not None: # Flag was present or value was given
                if isinstance(mapping, tuple): # Boolean flag case
                    param_attr, value_if_present = mapping
                    current_value = current_params_dict.get(param_attr)
                    if cli_value and current_value != value_if_present: # Check if flag is True and changes value
                        setattr(params, param_attr, value_if_present)
                        overridden_params.append(f"{param_attr}={value_if_present} (from --{arg_key.replace('_','-')})")
                else: # Value argument case
                    param_attr = mapping
                    current_value = current_params_dict.get(param_attr)
                    expected_type = type(current_value)
                    try:
                        typed_cli_value = expected_type(cli_value)
                        if typed_cli_value != current_value:
                            setattr(params, param_attr, typed_cli_value)
                            overridden_params.append(f"{param_attr}={typed_cli_value} (from --{arg_key.replace('_','-')})")
                    except (ValueError, TypeError):
                        logger.warning(f"CLI override failed for {param_attr}: Cannot convert '{cli_value}' to {expected_type}.")
                        print(f"Warning: Ignoring invalid value '{cli_value}' for --{arg_key.replace('_','-')}.", file=sys.stderr)

        if overridden_params: logger.info(f"CLI parameter overrides applied: {', '.join(overridden_params)}")
        logger.info(f"Final parameters for run: {params.to_dict()}")

        # --- Run the Main Design Function ---
        print(f"\nStarting network design for '{place_name}'...")
        result = design_fibre_network(place_name, params, output_dir)

        # --- Process Results ---
        if result and isinstance(result, dict) and result.get("statistics"):
            print("\n--- Network Design Completed Successfully ---")
            final_stats = result["statistics"]
            print("\n" + "="*50 + "\n" + final_stats.generate_summary(params) + "\n" + "="*50)
            print("\nOutput files generated in:", os.path.abspath(output_dir))
            # List generated files
            for key, path in result.items():
                if key.endswith("_path") and path and os.path.exists(path):
                     print(f"  {key.replace('_path','').replace('_',' ').title()}: {os.path.basename(path)}")
            print("="*50)
            logger.info("CLI Mode Execution Finished Successfully.")
            return 0 # Success exit code
        else:
            logger.error("CLI design process failed or returned no valid results.")
            print("\nError: Network design process failed or did not produce results.", file=sys.stderr)
            print("Please check the log file for details.", file=sys.stderr)
            return 1 # Failure exit code

    except ValueError as ve: # Catch specific configuration/input errors
        logger.error(f"CLI Configuration Error: {ve}", exc_info=True)
        print(f"\nConfiguration Error: {ve}", file=sys.stderr)
        return 1
    except ImportError as ie: # Should be caught earlier, but handle here too
         logger.critical(f"Import Error: {ie}. Ensure libraries are installed.", exc_info=True)
         print(f"\nImport Error: {ie}.\nPlease install dependencies.", file=sys.stderr)
         return 1
    except Exception as e:
        logger.critical(f"An unexpected error occurred in CLI mode: {e}", exc_info=True)
        print(f"\nAn unexpected error occurred: {type(e).__name__}: {e}", file=sys.stderr)
        print("Check log file for detailed traceback.", file=sys.stderr)
        return 1

# ---------------------------
# MAIN ENTRY POINT & Argument Parsing
# ---------------------------
def check_dependencies():
    """Check for essential libraries and return list of missing ones."""
    required_libs = ['shapely', 'osmnx', 'networkx', 'simplekml', 'pyproj', 'numpy', 'sklearn', 'tkinter']
    missing_libs = []
    for lib_name in required_libs:
        try:
            __import__(lib_name)
        except ImportError:
             missing_libs.append(lib_name)
    return missing_libs

def main():
    """Main entry point: Checks dependencies, parses args, launches CLI or GUI."""
    parser = argparse.ArgumentParser(
        description=f"{APP_NAME} (v{VERSION}) - Design FTTH/FTTB networks.",
        formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    parser.add_argument('--version', action='version', version=f'%(prog)s {VERSION}')
    parser.add_argument("--cli", action="store_true", help="Run in Command Line Interface (CLI) mode.")

    # --- CLI Specific Arguments ---
    cli_group = parser.add_argument_group('CLI Mode Options (ignored in GUI mode)')
    cli_group.add_argument("--place", type=str, metavar="'Place Name'", help="Area name (e.g., 'City, Country'). Required for CLI.")
    cli_group.add_argument("--output", type=str, default=DEFAULT_OUTPUT_DIR, metavar="DIR", help="Output directory.")
    cli_group.add_argument("--params-file", metavar="FILE.json", type=str, help="Load parameters from JSON file.")
    # CLI Parameter Overrides
    cli_group.add_argument("--max-clients-per-splitter", type=int, metavar="N", help="Override: Max clients per splitter.")
    cli_group.add_argument("--max-drop-length", type=float, metavar="METERS", help="Override: Max drop length.")
    cli_group.add_argument("--max-zone-area", type=float, metavar="SQ_METERS", help="Override: Max zone area.")
    cli_group.add_argument("--manhole-cluster-distance", type=float, metavar="METERS", help="Override: DBSCAN distance.")
    cli_group.add_argument("--disable-manhole-optimization", action="store_true", help="Override: Disable manhole clustering.")
    cli_group.add_argument("--disable-network-routing", action="store_true", help="Override: Force straight-line distribution/feeder routes.")
    cli_group.add_argument("--disable-centrality", action="store_true", help="Override: Disable centrality calc for aggregation node (faster).")

    args = parser.parse_args()

    # --- Dependency Check ---
    missing = check_dependencies()
    if missing:
         print(f"Error: Missing required libraries: {', '.join(missing)}", file=sys.stderr)
         print("Please install them (e.g., 'pip install osmnx shapely networkx simplekml pyproj numpy scikit-learn')", file=sys.stderr)
         if 'tkinter' in missing: print("Note: 'tkinter' might require separate system installation (e.g., 'sudo apt-get install python3-tk' on Debian/Ubuntu).", file=sys.stderr)
         return 1

    # --- Execute based on mode ---
    if args.cli:
        if not args.place: parser.error("--place is required for --cli mode.")
        return run_cli(args) # Returns 0 for success, 1 for failure
    else:
        print("Launching GUI mode...")
        # Set high-DPI awareness on Windows if possible
        if sys.platform == "win32":
            try:
                from ctypes import windll
                windll.shcore.SetProcessDpiAwareness(1) # System Aware
            except Exception: pass # Non-critical if it fails

        try:
            root = tk.Tk()
            app = FibreNetworkDesignerGUI(root)
            root.mainloop()
            
            print("GUI Closed.")
            return 0 # Normal GUI exit
        except tk.TclError as tcl_e:
             print(f"\nGUI Error: Could not initialize Tkinter GUI: {tcl_e}", file=sys.stderr)
             print("Ensure a display environment is available.", file=sys.stderr)
             return 1
        except Exception as gui_e:
             print(f"\nUnexpected GUI error: {gui_e}", file=sys.stderr)
             # Log exception if possible (though logger might depend on GUI setup)
             try: logger.critical("Unexpected GUI Error", exc_info=True)
             except: pass
             return 1


if __name__ == "__main__":
    # Setup basic console logging BEFORE parsing args or running GUI/CLI,
    # in case of early errors. File logger is added later/inside functions.
    if not logger.handlers:
        console_handler = logging.StreamHandler(sys.stdout) # Use stdout for console
        console_handler.setFormatter(log_formatter)
        logger.addHandler(console_handler)

    sys.exit(main())