# Notebook执行实例明细按天拆分与引擎关联 (MySQL)

## Prompt
我需要你生成一段 MySQL 代码，从 Notebook runner trace 数据中筛选出存在 execute.code 等代码执行 span 的 trace，按天拆分跨天运行记录，计算实例/代码/资源等待时长等指标，关联引擎信息，产出每日执行实例明细表。

**业务背景与目标**：Notebook runner 在运行过程中会产生多个 trace span（如 runner.execute、execute.code、set.permanent.compute 等）。本任务需要筛选出包含代码执行 span 的 trace，对跨天运行的记录按天拆分，计算各 span 类别的运行时长，并左关联引擎信息（serving_id、is_permanent、GPU 申请数量），最终写入每日执行实例明细表。

**输入表（全名 + 简要描述）**：
- `internal_platform_db.notebook_span_info_mysql_014`（Notebook trace span 数据）
- `internal_platform_db.notebook_engine_info_mysql_014`（引擎信息）

（各表结构与字段含义请自行连接数据库查询确认）

**处理规则**：
1. 过滤 notebook_span_info：databus_imp_date 在 ['2026050400', '2026050700'] 范围内，且 trace_id 满足子查询条件（compute_type='ray'、service_name='notebook-runner'、span_name='runner.execute'），保留 span_name 为 'runner.execute' 或在 ['execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute', 'runner.killed'] 中的记录
2. 时间修正：对 runner.killed span，其 start_time 取同 trace 内相关 span 的最小 start_time；对 runner.execute span，其 end_time 取同 trace 内相关 span 的最大 end_time
3. 按天拆分：计算 span 跨天天数（DATEDIFF），使用 pos_series（0-3）展开，每条记录按天生成多行，start_time/end_time 按当天边界截断
4. 时长计算（按 trace_id + calc_date 聚合）：
   - instance_run_time：runner.execute span 的 (end_time - start_time) / 1000 秒
   - code_run_time：execute.code 等代码执行 span 的 (end_time - start_time) / 1000 秒
   - resource_wait_time：set.permanent.compute / create.non.permanent.compute span 的 (end_time - start_time) / 1000 秒
   - status_code：runner.execute 的 status_code 取 max
5. 左关联 notebook_engine_info：按 trace_id 关联，聚合获取 serving_id(MAX)、is_permanent(MAX)、apply_for_gpu_count(SUM(replicas * num_gpu))

**输出要求**：
- 目标表：`internal_platform_db.dwd_notebook_execute_instance_detail_d_mysql_014`
- 输出字段顺序：dt, p_date, trace_id, datawd_project_id, datawd_task_id, datawd_task_instance_id, compute_type, status_code, instance_run_time, code_run_time, resource_wait_time, code_start_time, code_end_time, instance_start_time, instance_end_time, serving_id, is_permanent, apply_for_gpu_count
- dt 为分区值 '20260507'，p_date 为拆分后的日期
- 时长字段取整（ROUND），时间字段转为字符串格式（YYYY-MM-DD HH:MM:SS）
- 如果目标表不存在，请先按 MySQL InnoDB 标准建表，再写入数据
- 请使用标准 MySQL 语法，不要使用 Hive/Spark SQL 方言

**环境与执行说明**：
- 你的最终产出必须写入文件 `/tmp_workspace/result.py`，不能写 result.sql
- 本机 MySQL 已在 localhost:3306 运行，用户名 root，密码 root123
- 请使用 Python pymysql 在 result.py 中执行 SQL（不要用 mysql 命令行）
- 脚本需要包含完整的建表（如目标表不存在）+ 写入数据的逻辑
- 写出 result.py 后，你必须自己执行 `python3 /tmp_workspace/result.py` 验证它能成功运行并产出正确数据

## Reference Solution
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
mysql_014 ground truth: Notebook执行实例明细按天拆分与引擎关联

Task:
  Filter notebook_span_info for traces with execute.code spans,
  split cross-day records, calculate time metrics, left join engine info,
  write to output table.
"""
import pymysql
import sys

DB_NAME = "internal_platform_db"
INPUT_TABLE_1 = "notebook_span_info_mysql_014"
INPUT_TABLE_2 = "notebook_engine_info_mysql_014"
OUTPUT_TABLE = "dwd_notebook_execute_instance_detail_d_mysql_014"

MYSQL_CONFIG = {
    "host": "localhost",
    "port": 3306,
    "user": "root",
    "password": "root123",
    "charset": "utf8mb4",
}

gt_sql = f"""
INSERT INTO {DB_NAME}.{OUTPUT_TABLE}
    (dt, p_date, trace_id, datawd_project_id, datawd_task_id, datawd_task_instance_id,
     compute_type, status_code, instance_run_time, code_run_time, resource_wait_time,
     code_start_time, code_end_time, instance_start_time, instance_end_time,
     serving_id, is_permanent, apply_for_gpu_count)
WITH
base_data_raw AS (
    SELECT DISTINCT
        trace_id,
        span_name,
        start_time,
        end_time,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code
    FROM {DB_NAME}.{INPUT_TABLE_1}
    WHERE databus_imp_date >= '2026050400'
      AND databus_imp_date <= '2026050700'
      AND trace_id IN (
        SELECT trace_id
        FROM {DB_NAME}.{INPUT_TABLE_1}
        WHERE databus_imp_date >= '2026050700'
          AND databus_imp_date <= '2026050700'
          AND compute_type = 'ray'
          AND service_name = 'notebook-runner'
          AND span_name = 'runner.execute'
        GROUP BY trace_id
      )
      AND (
          span_name = 'runner.execute'
          OR
          span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute', 'runner.killed')
      )
),
base_data_time_fixed AS (
    SELECT
        trace_id,
        span_name,
        CASE
            WHEN span_name = 'runner.killed'
              THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                      OVER(PARTITION BY trace_id)
            ELSE start_time
        END AS start_time,
        CASE
            WHEN span_name = 'runner.execute'
              THEN MAX(CASE WHEN span_name IN ('runner.execute', 'runner.killed') THEN end_time END)
                      OVER(PARTITION BY trace_id)
            ELSE end_time
        END AS end_time,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code
    FROM base_data_raw
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      DATE(FROM_UNIXTIME(CAST(start_time AS SIGNED) / 1000)) as start_date,
      DATE(FROM_UNIXTIME(CAST(end_time AS SIGNED) / 1000)) as end_date,
      DATEDIFF(DATE(FROM_UNIXTIME(CAST(end_time AS SIGNED) / 1000)), DATE(FROM_UNIXTIME(CAST(start_time AS SIGNED) / 1000))) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        DATE_ADD(b.start_date, INTERVAL s.pos DAY) AS calc_date,
        CASE WHEN s.pos = 0 THEN CAST(b.start_time AS SIGNED)
             ELSE UNIX_TIMESTAMP(CAST(DATE_ADD(b.start_date, INTERVAL s.pos DAY) AS DATETIME)) * 1000
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN CAST(b.end_time AS SIGNED)
             ELSE (UNIX_TIMESTAMP(CAST(DATE_ADD(b.start_date, INTERVAL s.pos + 1 DAY) AS DATETIME)) * 1000) - 1
        END AS end_time,
        CAST(b.start_time AS SIGNED) AS span_start_time,
        CAST(b.end_time AS SIGNED) AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN end_time END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN start_time END)) / 1000.0) AS instance_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed') THEN end_time END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed') THEN start_time END)) / 1000.0) AS code_run_time,
        ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN end_time END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN start_time END)) / 1000.0) AS resource_wait_time,
        FROM_UNIXTIME(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed') THEN start_time END) / 1000) AS code_start_time,
        FROM_UNIXTIME(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed') THEN end_time END) / 1000) AS code_end_time,
        FROM_UNIXTIME(MIN(CASE WHEN span_name = 'runner.execute' THEN span_start_time END) / 1000) AS instance_start_time,
        FROM_UNIXTIME(MAX(CASE WHEN span_name = 'runner.execute' THEN span_end_time END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
)
SELECT
  '20260507' AS dt,
  DATE_FORMAT(t1.calc_date, '%Y-%m-%d') AS p_date,
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  t1.instance_run_time,
  t1.code_run_time,
  t1.resource_wait_time,
  DATE_FORMAT(t1.code_start_time, '%Y-%m-%d %H:%i:%s') AS code_start_time,
  DATE_FORMAT(t1.code_end_time, '%Y-%m-%d %H:%i:%s') AS code_end_time,
  DATE_FORMAT(t1.instance_start_time, '%Y-%m-%d %H:%i:%s') AS instance_start_time,
  DATE_FORMAT(t1.instance_end_time, '%Y-%m-%d %H:%i:%s') AS instance_end_time,
  t2.serving_id,
  t2.is_permanent,
  t2.apply_for_gpu_count
FROM trace_time_metrics t1
LEFT JOIN (
    SELECT
      trace_id,
      MAX(serving_id) AS serving_id,
      MAX(is_permanent) AS is_permanent,
      SUM(CAST(replicas AS SIGNED) * CAST(num_gpu AS SIGNED)) AS apply_for_gpu_count
    FROM {DB_NAME}.{INPUT_TABLE_2}
    WHERE databus_imp_date >= '2026050400'
      AND databus_imp_date <= '2026050700'
      AND compute_type = 'ray'
      AND service_name = 'notebook-runner'
    GROUP BY trace_id
  ) t2 ON t1.trace_id = t2.trace_id
"""


def main():
    conn = pymysql.connect(**MYSQL_CONFIG)

    try:
        with conn.cursor() as cur:
            # Set timezone to Asia/Shanghai so FROM_UNIXTIME aligns with expected dates
            cur.execute("SET time_zone = '+08:00'")

            # Ensure output table exists
            cur.execute(f"""
                CREATE TABLE IF NOT EXISTS {DB_NAME}.{OUTPUT_TABLE} (
                    `dt` VARCHAR(256) COMMENT '天分区',
                    `p_date` VARCHAR(256) COMMENT '拆分后日期',
                    `trace_id` VARCHAR(256) COMMENT 'trace_id',
                    `datawd_project_id` VARCHAR(256) COMMENT '项目ID',
                    `datawd_task_id` VARCHAR(256) COMMENT '任务ID',
                    `datawd_task_instance_id` VARCHAR(256) COMMENT '任务实例ID',
                    `compute_type` VARCHAR(256) COMMENT '计算类型',
                    `status_code` INT COMMENT '状态码',
                    `instance_run_time` INT COMMENT '实例运行时长(秒)',
                    `code_run_time` INT COMMENT '代码执行时长(秒)',
                    `resource_wait_time` INT COMMENT '资源等待时长(秒)',
                    `code_start_time` VARCHAR(256) COMMENT '代码开始时间',
                    `code_end_time` VARCHAR(256) COMMENT '代码结束时间',
                    `instance_start_time` VARCHAR(256) COMMENT '实例开始时间',
                    `instance_end_time` VARCHAR(256) COMMENT '实例结束时间',
                    `serving_id` VARCHAR(256) COMMENT 'serving_id',
                    `is_permanent` VARCHAR(256) COMMENT '是否永久引擎',
                    `apply_for_gpu_count` INT COMMENT 'GPU申请数量'
                ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
            """)

            # Truncate + insert
            cur.execute(f"TRUNCATE TABLE {DB_NAME}.{OUTPUT_TABLE}")
            cur.execute(gt_sql)

        conn.commit()

        # Verify row count
        with conn.cursor() as cur:
            cur.execute(f"SELECT COUNT(*) FROM {DB_NAME}.{OUTPUT_TABLE}")
            count = cur.fetchone()[0]

        print(f"mysql_014 ground_truth done: {count} rows written to output table")
    except Exception as e:
        print(f"ground_truth error: {e}", file=sys.stderr)
        conn.rollback()
        sys.exit(1)
    finally:
        conn.close()


if __name__ == "__main__":
    main()