# 从 `internal_platform_db.notebook_span_info_query_engine (HiveSQL)

## Prompt
1) 任务目标
识别并统计 Notebook 管道中被异常 kill 且缺失正常完成信号的 Ray 实例，按天拆分计算运行时长、代码执行时长、资源等待时长及 GPU 申请数。

2) 输入
- `internal_platform_db.notebook_span_info_query_engine_007`
- `internal_platform_db.notebook_engine_info_query_engine_007`

3) 处理规则
- 目标实例筛选：从 span_info 筛选 `compute_type='ray'` 且 `service_name='notebook-runner'` 的实例，选取有 `span_name='runner.killed'` 但无 `span_name='runner.execute'` 的 `trace_id`。
- 合成运行区间：对每个目标 trace_id，取 `span_name='runner.start'` 的 start_time 作为实例开始时间，取 `span_name='runner.killed'` 的 end_time 作为实例结束时间，合成 `runner.execute` 区间，`status_code` 固定为 2。
- 跨天拆分：将合成区间按自然日拆分为多条记录，每条对应一天内的部分，`calc_date` 为拆分后的日期。
- 时长计算（按 trace_id 与 calc_date）：
  - instance_run_time：合成区间在当天的起止时间差（毫秒转秒）。
  - code_run_time：`span_name` 为 'execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed' 在当天的起止时间差（毫秒转秒）。
  - resource_wait_time：`span_name` 为 'set.permanent.compute', 'create.non.permanent.compute' 在当天的起止时间差（毫秒转秒）。
  - 输出 code_start_time/code_end_time/instance_start_time/instance_end_time（时间戳转字符串）。
- 关联引擎信息：左关联 engine_info（过滤 `compute_type='ray'` 且 `service_name='notebook-runner'`），关联键 `trace_id`，获取 serving_id、is_permanent，计算 `apply_for_gpu_count = SUM(replicas * num_gpu)`。

4) 输出要求
输出字段顺序：dt（STRING，分区）、p_date（STRING，calc_date）、trace_id（STRING）、datawd_project_id（STRING）、datawd_task_id（STRING）、datawd_task_instance_id（STRING）、compute_type（STRING）、status_code（INT）、instance_run_time（INT）、code_run_time（INT）、resource_wait_time（INT）、code_start_time（STRING）、code_end_time（STRING）、instance_start_time（STRING）、instance_end_time（STRING）、serving_id（STRING）、is_permanent（BOOLEAN）、apply_for_gpu_count（INT）。

5) 写入要求
目标表：`internal_platform_db.dwd_notebook_killed_instance_detail_d_copilot_cand_query_engine_007`。分区字段：dt。写入分区：`dt='20260507'`。写入模式：覆盖。

请将最终 HiveSQL 写入 result.sql 并执行。

## Reference Solution
INSERT overwrite TABLE internal_platform_db.dwd_notebook_killed_instance_detail_d_copilot_query_engine_007 PARTITION (dt = '20260507')
WITH base_trace AS (
    SELECT trace_id
    FROM internal_platform_db.notebook_span_info_query_engine_007
    WHERE databus_imp_date >= '2026050700'
      AND databus_imp_date <= '2026050700'
      AND compute_type = 'ray'
      AND service_name = 'notebook-runner'
      AND span_name IN ('runner.execute', 'runner.killed')
    GROUP BY trace_id
    HAVING
        COUNT(CASE WHEN span_name = 'runner.killed' THEN 1 END) > 0
        AND
        COUNT(CASE WHEN span_name = 'runner.execute' THEN 1 END) = 0
),
combined_spans AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM internal_platform_db.notebook_span_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.killed', 'runner.start', 'execute.code', 'execute.code.cell', 'client.execute.code', 'set.permanent.compute', 'create.non.permanent.compute')

  UNION ALL

  SELECT
      trace_id,
      'runner.execute' AS span_name,
      MAX(CASE WHEN span_name = 'runner.start' THEN start_time END) AS start_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN end_time END) AS end_time,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_project_id END) AS datawd_project_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_id END) AS datawd_task_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN datawd_task_instance_id END) AS datawd_task_instance_id,
      MAX(CASE WHEN span_name = 'runner.killed' THEN compute_type END) AS compute_type,
      2 AS status_code
  FROM internal_platform_db.notebook_span_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND trace_id IN (SELECT trace_id FROM base_trace)
    AND span_name IN ('runner.start', 'runner.killed')
  GROUP BY trace_id
),
base_data_time_fixed AS (
  SELECT
      trace_id,
      span_name,
      CASE
          WHEN span_name = 'runner.killed'
            THEN MIN(CASE WHEN span_name IN ('execute.code', 'execute.code.cell', 'client.execute.code', 'runner.killed') THEN start_time END)
                    OVER(PARTITION BY trace_id)
          ELSE start_time
      END AS start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code
  FROM combined_spans
),
base_data AS (
  SELECT
      trace_id,
      span_name,
      start_time,
      end_time,
      datawd_project_id,
      datawd_task_id,
      datawd_task_instance_id,
      compute_type,
      status_code,
      from_unixtime(CAST(start_time AS BIGINT) / 1000) as start_date,
      from_unixtime(CAST(end_time AS BIGINT) / 1000) as end_date,
      datediff(from_unixtime(CAST(end_time AS BIGINT) / 1000), from_unixtime(CAST(start_time AS BIGINT) / 1000)) AS diff_days
  FROM base_data_time_fixed
),
pos_series AS (
    SELECT 0 AS pos UNION ALL SELECT 1 AS pos UNION ALL SELECT 2 AS pos UNION ALL SELECT 3 AS pos
),
daily_split_spans AS (
    SELECT
        trace_id,
        span_name,
        datawd_project_id,
        datawd_task_id,
        datawd_task_instance_id,
        compute_type,
        status_code,
        start_date,
        end_date,
        diff_days,
        date_add(b.start_date, s.pos) AS calc_date,
        CASE WHEN s.pos = 0 THEN CAST(b.start_time AS BIGINT)
             ELSE unix_timestamp(cast(date_add(b.start_date, s.pos) as timestamp)) * 1000
        END AS start_time,
        CASE WHEN s.pos = b.diff_days THEN CAST(b.end_time AS BIGINT)
             ELSE (unix_timestamp(cast(date_add(b.start_date, s.pos + 1) as timestamp)) * 1000) - 1
        END AS end_time,
        CAST(b.start_time AS BIGINT) AS span_start_time,
        CAST(b.end_time AS BIGINT) AS span_end_time
    FROM base_data b
    INNER JOIN pos_series s ON s.pos <= b.diff_days
),
trace_time_metrics AS (
    SELECT
        trace_id,
        calc_date,
        MAX(datawd_project_id) AS datawd_project_id,
        MAX(datawd_task_id) AS datawd_task_id,
        MAX(datawd_task_instance_id) AS datawd_task_instance_id,
        MAX(compute_type) AS compute_type,
        MAX(MAX(CASE WHEN span_name = 'runner.execute' THEN status_code END)) OVER(PARTITION BY trace_id) AS status_code,
        CAST(ROUND((MAX(CASE WHEN span_name = 'runner.execute' THEN end_time END) -
               MIN(CASE WHEN span_name = 'runner.execute' THEN start_time END)) / 1000.0) AS INT) AS instance_run_time,
        CAST(ROUND((MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) -
               MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END)) / 1000.0) AS INT) AS code_run_time,
        CAST(ROUND((MAX(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN end_time END) -
               MIN(CASE WHEN span_name IN ('set.permanent.compute', 'create.non.permanent.compute') THEN start_time END)) / 1000.0) AS INT) AS resource_wait_time,
        from_unixtime(MIN(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN start_time END) / 1000) AS code_start_time,
        from_unixtime(MAX(CASE WHEN span_name IN ('execute.code', 'client.execute.code', 'execute.code.cell', 'runner.killed' ) THEN end_time END) / 1000) AS code_end_time,
        from_unixtime(MIN(CASE WHEN span_name = 'runner.execute' THEN span_start_time END) / 1000) AS instance_start_time,
        from_unixtime(MAX(CASE WHEN span_name = 'runner.execute' THEN span_end_time END) / 1000) AS instance_end_time
    FROM daily_split_spans
    GROUP BY trace_id, calc_date
)
SELECT
  t1.calc_date AS p_date,
  t1.trace_id,
  t1.datawd_project_id,
  t1.datawd_task_id,
  t1.datawd_task_instance_id,
  t1.compute_type,
  t1.status_code,
  t1.instance_run_time,
  t1.code_run_time,
  t1.resource_wait_time,
  t1.code_start_time,
  t1.code_end_time,
  t1.instance_start_time,
  t1.instance_end_time,
  t2.serving_id,
  CAST(t2.is_permanent AS BOOLEAN) AS is_permanent,
  CAST(t2.apply_for_gpu_count AS INT) AS apply_for_gpu_count
FROM trace_time_metrics t1 LEFT JOIN (
  SELECT
    trace_id,
    MAX(serving_id) AS serving_id,
    MAX(is_permanent) AS is_permanent,
    SUM(CAST(replicas AS INT) * CAST(num_gpu AS INT)) AS apply_for_gpu_count
  FROM internal_platform_db.notebook_engine_info_query_engine_007
  WHERE databus_imp_date >= '2026050400'
    AND databus_imp_date <= '2026050700'
    AND compute_type = 'ray'
    AND service_name = 'notebook-runner'
  GROUP BY trace_id
) t2 ON t1.trace_id = t2.trace_id
;