Domain: Business Operations & Analytics

Using the local files `orders.csv`, `order_items.csv`, and `products.csv`, compute a deterministic retail order-lineage quality ranking.

Read all three files as raw strings first.

Apply these rules exactly:

1. Deduplicate `orders.csv` by `order_id`, keeping the first row after sorting by:
   - `order_id` ascending
   - parsed `order_date` descending
   - parsed `required_date` descending
   - parsed `shipped_date` descending
   - `customer_id` ascending
   - `store_id` ascending
   - `staff_id` ascending
2. Deduplicate `order_items.csv` by `(order_id, item_id)`, keeping the first row after sorting by:
   - `order_id` ascending
   - `item_id` ascending
   - `abs(quantity_numeric) * abs(list_price_numeric)` descending, using `-1` for a missing parsed factor
   - `discount_numeric` descending
   - `product_id` ascending
3. Deduplicate `products.csv` by `product_id`, keeping the first row after sorting by:
   - `product_id` ascending
   - `model_year_numeric` descending
   - `list_price_numeric` descending
   - `product_name` ascending lexicographically
4. Parse dates strictly with the format `%Y-%m-%d`.
5. Parse numeric fields with normal decimal parsing.
6. Empty strings count as missing values. The literal text `NULL` is non-empty text and therefore is not treated as missing.
7. Before joining, compute `inconsistent_id_count` as:
   - number of distinct deduplicated line-item `order_id` values missing from deduplicated orders
   - plus number of distinct deduplicated line-item `product_id` values missing from deduplicated products
8. Left join deduplicated line items to deduplicated orders on `order_id`, then left join to deduplicated products on `product_id`.
9. Build row-level flags:
   - `id_issue = 1` if either join misses, else `0`
   - `sign_reversal_flag = 1` if parsed `quantity`, `list_price`, or `discount` is negative, else `0`
   - `discount_overflow_flag = 1` if parsed `discount > 1`, else `0`
   - `null_coercion_flag = 1` if any non-empty numeric/date text fails parsing, else `0`
   - `mixed_ts_flag = 1` if any non-empty date text does not exactly match `%Y-%m-%d`, else `0`
   - `lag_anomaly = 1` if `shipped_filled < order_date_parsed` or `required_date_parsed < order_date_parsed`, else `0`; use `required_date_parsed` when `shipped_date` is missing
10. Build cleaned fields:
   - `quantity_c = abs(quantity_numeric)` with missing parsed values coerced to `0`
   - `list_price_c = abs(list_price_numeric)` with missing parsed values coerced to `0`
   - `discount_c = discount_numeric` with missing parsed values coerced to `0`, then clipped to `[0, 1]`
11. A row is eligible iff all of the following hold:
   - `order_status` is `3` or `4`
   - parsed `order_date` is present
   - parsed `required_date` is present
   - `quantity_c > 0`
   - `list_price_c > 0`
   - `id_issue = 0`
   - `lag_anomaly = 0`
12. For every row, compute:
   - `line_net = quantity_c * list_price_c * (1 - discount_c)`
   - `catalog_gap = abs(list_price_c - product_list_price_numeric) / product_list_price_numeric` when catalog list price is positive, else `0`
13. Compute the eligible-row IQR outlier rule on `line_net` using Hyndman-Fan Type 7 quartiles:
   - `q1 = percentile_25`
   - `q3 = percentile_75`
   - `iqr = q3 - q1`
   - lower fence `= q1 - 1.5 * iqr`
   - upper fence `= q3 + 1.5 * iqr`
   - `outlier_flag = 1` for eligible rows with `line_net` outside the fences, else `0`
14. Define `error_flags` for each row as the sum of:
   - `id_issue`
   - `sign_reversal_flag`
   - `discount_overflow_flag`
   - `null_coercion_flag`
   - `mixed_ts_flag`
   - `lag_anomaly`
   - `outlier_flag`
15. Group eligible rows by `(store_id, category_id)` and compute:
   - `eligible_lines`
   - `net_revenue = sum(line_net)`
   - `error_lines = sum(error_flags)`
   - `outlier_lines = sum(outlier_flag)`
   - `avg_catalog_gap = mean(catalog_gap)`
16. For each group compute:
   - `error_rate = error_lines / eligible_lines`
   - `quality_score = net_revenue * (1 - error_rate) - 100 * outlier_lines - 1000 * avg_catalog_gap`
17. Rank groups by:
   - `quality_score` descending
   - `eligible_lines` descending
   - `avg_catalog_gap` ascending
   - `store_id` ascending
   - `category_id` ascending
18. Keep the top 3 ranked groups.
19. For each ranked row, format `net_revenue`, `error_rate`, and `quality_score` as strings with exactly 6 decimal places using round-half-to-even.
20. Define each ranked-row `signature` as the integer value of `md5(pipe_joined_fields)` modulo `1_000_000_000_000`, where the pipe-joined fields are:
   - `rank|store_id|category_id|eligible_lines|net_revenue_6dp|error_rate_6dp|quality_score_6dp|outlier_lines`
21. Define the audit row values:
   - `audit_inconsistent = inconsistent_id_count + sum(id_issue)`
   - `audit_sign = sum(sign_reversal_flag)`
   - `audit_null = sum(null_coercion_flag)`
   - `audit_mixed = sum(mixed_ts_flag)`
   - `audit_lag = sum(lag_anomaly)`
22. Define `global_hash` as the integer value of `md5(pipe_joined_fields)` modulo `1_000_000_000_000`, where the pipe-joined fields are:
   - the three ranked-row signatures in order
   - then `audit_inconsistent`, `audit_sign`, `audit_null`, `audit_mixed`, and `audit_lag`
23. Define `stability_token` as:
   - sum of the top-3 `eligible_lines`
   - plus sum of the top-3 `outlier_lines`
   - plus the five audit counts

Return exactly one Python list with 4 elements:

1. ranked row 1 as `[rank, store_id, category_id, eligible_lines, net_revenue_6dp, error_rate_6dp, quality_score_6dp, outlier_lines, signature]`
2. ranked row 2 in the same schema
3. ranked row 3 in the same schema
4. the audit row as `['AUDIT', audit_inconsistent, audit_sign, audit_null, audit_mixed, audit_lag, global_hash, stability_token]`

Print only that final Python list.

Grading rubric:
criterion: The final answer is a Python list with exactly 4 top-level elements, where the first three rows each have 9 elements and the audit row has 8 elements.
weight: 10
criterion: The first ranked row is exactly `[1, 2, 6, 757, '1737864.529000', '0.220608', '1338778.298692', 157, 881674604252]`.
weight: 15
criterion: The second ranked row is exactly `[2, 2, 3, 891, '628763.410900', '0.024691', '613038.388409', 2, 240297193208]`.
weight: 15
criterion: The third ranked row is exactly `[3, 2, 4, 172, '465285.173500', '0.127907', '403771.953634', 20, 983264525045]`.
weight: 15
criterion: The audit row is exactly `['AUDIT', 0, 0, 508, 508, 0, 685374557935, 3015]`.
weight: 20
criterion: All monetary and rate fields in the three ranked rows are formatted as strings with exactly 6 decimal places using round-half-to-even.
weight: 10

Golden answer:
# Code

```python
import csv
import hashlib
from decimal import Decimal, ROUND_HALF_EVEN
from pathlib import Path
from datetime import datetime


BASE = Path(__file__).resolve().parent


def read_csv(name):
    with (BASE / name).open(newline="", encoding="utf-8") as handle:
        return list(csv.DictReader(handle))


def parse_date(text):
    text = (text or "").strip()
    if text == "":
        return None
    try:
        return datetime.strptime(text, "%Y-%m-%d").date()
    except ValueError:
        return None


def parse_decimal(text):
    text = (text or "").strip()
    if text == "":
        return None
    try:
        return Decimal(text)
    except Exception:
        return None


def format_6(value):
    return str(value.quantize(Decimal("0.000001"), rounding=ROUND_HALF_EVEN))


def percentile_type7(values, p):
    if len(values) == 1:
        return values[0]
    position = Decimal(len(values) - 1) * p
    lower = int(position)
    upper = lower if position == lower else lower + 1
    if lower == upper:
        return values[lower]
    fraction = position - Decimal(lower)
    return values[lower] * (Decimal("1") - fraction) + values[upper] * fraction


def md5_mod(text):
    return int(hashlib.md5(text.encode("utf-8")).hexdigest(), 16) % 1_000_000_000_000


orders_raw = read_csv("orders.csv")
items_raw = read_csv("order_items.csv")
products_raw = read_csv("products.csv")

for row in orders_raw:
    row["_order_date"] = parse_date(row["order_date"])
    row["_required_date"] = parse_date(row["required_date"])
    row["_shipped_date"] = parse_date(row["shipped_date"])

orders_sorted = sorted(
    orders_raw,
    key=lambda row: (
        int(row["order_id"]),
        -((row["_order_date"] or datetime.min.date()).toordinal()),
        -((row["_required_date"] or datetime.min.date()).toordinal()),
        -((row["_shipped_date"] or datetime.min.date()).toordinal()),
        int(row["customer_id"]),
        int(row["store_id"]),
        int(row["staff_id"]),
    ),
)

orders = {}
for row in orders_sorted:
    orders.setdefault(row["order_id"], row)

for row in items_raw:
    quantity_num = parse_decimal(row["quantity"])
    list_price_num = parse_decimal(row["list_price"])
    discount_num = parse_decimal(row["discount"])
    row["_quantity_num"] = quantity_num
    row["_list_price_num"] = list_price_num
    row["_discount_num"] = discount_num
    left = abs(quantity_num) if quantity_num is not None else Decimal("-1")
    right = abs(list_price_num) if list_price_num is not None else Decimal("-1")
    row["_abs_line_base"] = left * right

items_sorted = sorted(
    items_raw,
    key=lambda row: (
        int(row["order_id"]),
        int(row["item_id"]),
        -row["_abs_line_base"],
        -(row["_discount_num"] if row["_discount_num"] is not None else Decimal("-Infinity")),
        int(row["product_id"]),
    ),
)

items = {}
for row in items_sorted:
    items.setdefault((row["order_id"], row["item_id"]), row)

for row in products_raw:
    row["_model_year_num"] = parse_decimal(row["model_year"])
    row["_product_list_price_num"] = parse_decimal(row["list_price"])

products_sorted = sorted(
    products_raw,
    key=lambda row: (
        int(row["product_id"]),
        -(row["_model_year_num"] if row["_model_year_num"] is not None else Decimal("-Infinity")),
        -(row["_product_list_price_num"] if row["_product_list_price_num"] is not None else Decimal("-Infinity")),
        row["product_name"],
    ),
)

products = {}
for row in products_sorted:
    products.setdefault(row["product_id"], row)

item_rows = list(items.values())
inconsistent_id_count = len({row["order_id"] for row in item_rows} - set(orders)) + len({row["product_id"] for row in item_rows} - set(products))

def is_bad_nonempty(text, parsed):
    text = (text or "").strip()
    return text != "" and parsed is None


def mixed_timestamp_flag(text):
    text = (text or "").strip()
    if text == "":
        return 0
    try:
        datetime.strptime(text, "%Y-%m-%d")
        return 0
    except ValueError:
        return 1


merged = []
for row in item_rows:
    order = orders.get(row["order_id"])
    product = products.get(row["product_id"])
    quantity_num = parse_decimal(row["quantity"])
    list_price_num = parse_decimal(row["list_price"])
    discount_num = parse_decimal(row["discount"])
    order_date = parse_date(order["order_date"] if order else "")
    required_date = parse_date(order["required_date"] if order else "")
    shipped_date = parse_date(order["shipped_date"] if order else "")

    id_issue = int(order is None or product is None)
    sign_reversal_flag = int(
        (quantity_num is not None and quantity_num < 0)
        or (list_price_num is not None and list_price_num < 0)
        or (discount_num is not None and discount_num < 0)
    )
    discount_overflow_flag = int(discount_num is not None and discount_num > 1)
    null_coercion_flag = int(
        is_bad_nonempty(row["quantity"], quantity_num)
        or is_bad_nonempty(row["list_price"], list_price_num)
        or is_bad_nonempty(row["discount"], discount_num)
        or is_bad_nonempty(order["order_date"] if order else "", order_date)
        or is_bad_nonempty(order["required_date"] if order else "", required_date)
        or is_bad_nonempty(order["shipped_date"] if order else "", shipped_date)
    )
    mixed_ts_flag = (
        mixed_timestamp_flag(order["order_date"] if order else "")
        or mixed_timestamp_flag(order["required_date"] if order else "")
        or mixed_timestamp_flag(order["shipped_date"] if order else "")
    )

    quantity_c = abs(quantity_num) if quantity_num is not None else Decimal("0")
    list_price_c = abs(list_price_num) if list_price_num is not None else Decimal("0")
    discount_c = discount_num if discount_num is not None else Decimal("0")
    if discount_c < 0:
        discount_c = Decimal("0")
    if discount_c > 1:
        discount_c = Decimal("1")

    shipped_filled = shipped_date if shipped_date is not None else required_date
    lag_anomaly = int(
        shipped_filled is None
        or order_date is None
        or required_date is None
        or shipped_filled < order_date
        or required_date < order_date
    )
    eligible = int(
        (order is not None)
        and order["order_status"] in {"3", "4"}
        and order_date is not None
        and required_date is not None
        and quantity_c > 0
        and list_price_c > 0
        and id_issue == 0
        and lag_anomaly == 0
    )

    line_net = quantity_c * list_price_c * (Decimal("1") - discount_c)
    product_list_price = product["_product_list_price_num"] if product else None
    if product_list_price is not None and product_list_price > 0:
        catalog_gap = abs(list_price_c - product_list_price) / product_list_price
    else:
        catalog_gap = Decimal("0")

    merged.append(
        {
            "store_id": order["store_id"] if order else None,
            "category_id": product["category_id"] if product else None,
            "id_issue": id_issue,
            "sign_reversal_flag": sign_reversal_flag,
            "discount_overflow_flag": discount_overflow_flag,
            "null_coercion_flag": null_coercion_flag,
            "mixed_ts_flag": int(mixed_ts_flag),
            "lag_anomaly": lag_anomaly,
            "eligible": eligible,
            "line_net": line_net,
            "catalog_gap": catalog_gap,
        }
    )

eligible_net = sorted(row["line_net"] for row in merged if row["eligible"] == 1)
q1 = percentile_type7(eligible_net, Decimal("0.25"))
q3 = percentile_type7(eligible_net, Decimal("0.75"))
iqr = q3 - q1
lower_fence = q1 - Decimal("1.5") * iqr
upper_fence = q3 + Decimal("1.5") * iqr

for row in merged:
    row["outlier_flag"] = int(row["eligible"] == 1 and (row["line_net"] < lower_fence or row["line_net"] > upper_fence))
    row["error_flags"] = (
        row["id_issue"]
        + row["sign_reversal_flag"]
        + row["discount_overflow_flag"]
        + row["null_coercion_flag"]
        + row["mixed_ts_flag"]
        + row["lag_anomaly"]
        + row["outlier_flag"]
    )

groups = {}
for row in merged:
    if row["eligible"] != 1:
        continue
    key = (int(row["store_id"]), int(row["category_id"]))
    group = groups.setdefault(
        key,
        {
            "eligible_lines": 0,
            "net_revenue": Decimal("0"),
            "error_lines": 0,
            "outlier_lines": 0,
            "catalog_gap_sum": Decimal("0"),
        },
    )
    group["eligible_lines"] += 1
    group["net_revenue"] += row["line_net"]
    group["error_lines"] += row["error_flags"]
    group["outlier_lines"] += row["outlier_flag"]
    group["catalog_gap_sum"] += row["catalog_gap"]

ranked = []
for (store_id, category_id), group in groups.items():
    avg_catalog_gap = group["catalog_gap_sum"] / Decimal(group["eligible_lines"])
    error_rate = Decimal(group["error_lines"]) / Decimal(group["eligible_lines"])
    quality_score = group["net_revenue"] * (Decimal("1") - error_rate) - Decimal("100") * Decimal(group["outlier_lines"]) - Decimal("1000") * avg_catalog_gap
    ranked.append(
        {
            "store_id": store_id,
            "category_id": category_id,
            "eligible_lines": group["eligible_lines"],
            "net_revenue": group["net_revenue"],
            "error_rate": error_rate,
            "quality_score": quality_score,
            "outlier_lines": group["outlier_lines"],
            "avg_catalog_gap": avg_catalog_gap,
        }
    )

ranked.sort(
    key=lambda row: (
        -row["quality_score"],
        -row["eligible_lines"],
        row["avg_catalog_gap"],
        row["store_id"],
        row["category_id"],
    )
)

top3 = ranked[:3]
rows = []
signatures = []

for index, row in enumerate(top3, start=1):
    net_revenue_6dp = format_6(row["net_revenue"])
    error_rate_6dp = format_6(row["error_rate"])
    quality_score_6dp = format_6(row["quality_score"])
    signature = md5_mod(
        f"{index}|{row['store_id']}|{row['category_id']}|{row['eligible_lines']}|{net_revenue_6dp}|{error_rate_6dp}|{quality_score_6dp}|{row['outlier_lines']}"
    )
    signatures.append(signature)
    rows.append(
        [
            index,
            row["store_id"],
            row["category_id"],
            row["eligible_lines"],
            net_revenue_6dp,
            error_rate_6dp,
            quality_score_6dp,
            row["outlier_lines"],
            signature,
        ]
    )

audit_inconsistent = inconsistent_id_count + sum(row["id_issue"] for row in merged)
audit_sign = sum(row["sign_reversal_flag"] for row in merged)
audit_null = sum(row["null_coercion_flag"] for row in merged)
audit_mixed = sum(row["mixed_ts_flag"] for row in merged)
audit_lag = sum(row["lag_anomaly"] for row in merged)
global_hash = md5_mod("|".join(str(value) for value in signatures + [audit_inconsistent, audit_sign, audit_null, audit_mixed, audit_lag]))
stability_token = sum(row[3] for row in rows) + sum(row[7] for row in rows) + audit_inconsistent + audit_sign + audit_null + audit_mixed + audit_lag

final_answer = rows + [["AUDIT", audit_inconsistent, audit_sign, audit_null, audit_mixed, audit_lag, global_hash, stability_token]]
print(final_answer)
```

# Reasoning

The code reads the three local CSV files as strings, applies the deterministic deduplication rules, and joins line items to orders and products. It parses dates strictly as `%Y-%m-%d`, so the literal `NULL` values in `shipped_date` fail parsing. Because those values are non-empty text, they contribute to both `null_coercion_flag` and `mixed_ts_flag`, which is why the audit row has `508` for both counts. The cleaned numeric fields use absolute values for quantity and price, clip discounts into `[0, 1]`, and use `required_date` when `shipped_date` is missing.

Eligible rows are grouped by `(store_id, category_id)`. The code computes line-level net revenue, catalog gaps, eligible-row IQR outliers using Hyndman-Fan Type 7 quartiles, and then group-level `error_rate` and `quality_score`. The top three groups are ranked deterministically, then the code formats the three numeric string fields with exactly 6 decimal places using round-half-to-even. Finally, it computes row signatures and the global audit hash with `md5(... ) mod 1_000_000_000_000`, and prints the required nested Python list.

# Verified Result

```python
[[1, 2, 6, 757, '1737864.529000', '0.220608', '1338778.298692', 157, 881674604252], [2, 2, 3, 891, '628763.410900', '0.024691', '613038.388409', 2, 240297193208], [3, 2, 4, 172, '465285.173500', '0.127907', '403771.953634', 20, 983264525045], ['AUDIT', 0, 0, 508, 508, 0, 685374557935, 3015]]
```