# Anti-Spam H5 Work Order Log Daily Detail Sync (PySpark)

## Prompt
I need you to generate a PySpark script that synchronizes the "Anti-Spam H5 Work Order Log" from ODS to the DWD detail table with full-field passthrough, and adds a run date column.

**Business Background and Objective**: The anti-spam H5 work order service writes each appeal/report operation log to the raw `log_80001099` table (48 business fields + `databus_imp_date` partition). The downstream DWD wants to transport the full-field detail of the current day's partition every day, and additionally add a column `imp_date` (run date, YYYYMMDD) for convenient subsequent filtering by run date. The task has no aggregation and no joins — pure detail transport + one derived column.

**Input Table**:
- `internal_platform_db.caseR18_ods_log_80001099_v3`

**Data Range and Filter Conditions (business description)**:
  - The original task in production reads from the full table (without `imp_date` filter); the sandbox v3 table is loaded by `ds` for the corresponding date, so the SELECT has no WHERE clause and reads the full table.
  - The run date `ds` is fixed as `'20260513'` and will be written as the new column `imp_date`.

**Table Join Relationships**: (No joins, single-table passthrough.)

**Aggregation and Computation Rules (must be reflected in the SQL)**:
  - Derived column `imp_date = '20260513'` (run date constant), placed as the first column in the SELECT.
  - The following 49 columns are passed through as-is: `databus_imp_date` + 48 business fields, in the same order as the DDL.
  - Use `INSERT OVERWRITE TABLE` + explicit column name list (50 columns) to avoid column order misalignment.

**Output Requirements**:
  - Target table: `internal_platform_db.caseR18_dwd_log_80001099_daily_v3`
  - Output table schema (50 columns, in this order, with the following semantics; the first 2 columns are run/original partition date, the remaining 48 are business fields):
    1. `imp_date` STRING: run date YYYYMMDD = `'20260513'`
    2. `databus_imp_date` STRING: original partition date (passed through from input)
    3. `logid` BIGINT, 4. `svrtime` STRING, 5. `svrip` STRING, 6. `module` STRING
    7. `form_id` STRING, 8. `form_type` BIGINT, 9. `action` BIGINT, 10. `retcode` BIGINT
    11. `vid` BIGINT, 12. `corpid` BIGINT, 13. `gid` BIGINT
    14. `appeal_kind` BIGINT (appeal problem type), 15. `report_kind` BIGINT (report problem type), 16. `fraud_kind` BIGINT (fraud type), 17. `loss_amount` BIGINT (loss amount)
    18. `suspect_vid` BIGINT, 19. `suspect_corpid` BIGINT
    20. `order_id` STRING (work order number), 21. `order_result` BIGINT, 22. `order_source` BIGINT, 23. `order_action` BIGINT
    24. `match_rule` STRING, 25. `roomid` BIGINT (problematic group number), 26. `create_vid` BIGINT, 27. `openid` STRING
    28. `isfromwx` BIGINT (interception party ww/wx), 29. `spamtype` STRING, 30. `auto_finish_order_rule` STRING
    31. `action_time` BIGINT (action occurrence time), 32. `industry_name` STRING, 33. `second_industry_name` STRING, 34. `is_ka` BIGINT
    35. `create_time` STRING, 36. `urgent_time` STRING, 37. `result_time` STRING, 38. `reopen_time` BIGINT
    39. `block_uin` BIGINT, 40. `block_user_id_type` BIGINT, 41. `version` STRING (v1/v2 differentiation)
    42. `h5_source` BIGINT (0: frontline customer service, 1: user self-service), 43. `h5_platform` BIGINT (1: BizComm, 2: PlatformW), 44. `template_type` BIGINT
    45. `current_owner` STRING, 46. `oid` BIGINT (appeal work order numeric ID), 47. `reason` STRING (QA closing evidence reason, base64)
    48. `order_finish_type` STRING, 49. `reply_content` STRING (scripted reply, base64), 50. `match_rule_scene_id` BIGINT
  - Write strategy: `INSERT OVERWRITE TABLE` + explicit 50-column name list.
  - Sandbox note: The original task is encapsulated in SparkBase (SparkJob class), using the `self.load_dw_data().createOrReplaceTempView()` + `self.save_dw_data()` chain; the sandbox version is rewritten as a top-level SparkSession + a single `INSERT OVERWRITE TABLE ... SELECT`.
  - If the target table does not exist, first create it using standard Hive format (ORC storage), then write the data.

## Reference Solution
# caseR18 rewrite-gt
# source: data/v7_gt_codes/case_row19_case_0011.py (v7 row 18, manifest_row 19)
# task_name: t_dwd_ww_log_80001099_daily_copilot
# rewrite rules:
#   1) internal_platform_db.log_80001099_copilot
#       -> internal_platform_db.caseR18_ods_log_80001099_v3
#   2) internal_platform_db.dwd_ww_log_80001099_daily_copilot
#       -> internal_platform_db.caseR18_dwd_log_80001099_daily_v3
#   3) 删除 dw_spark_base_python3.SparkBase 框架封装：
#      - SparkJob(SparkBase) class -> 直接顶层 SparkSession
#      - self.load_dw_data(db, tbl).createOrReplaceTempView() -> 直接在 sql 里写全名
#      - self.save_dw_data(df, db_name, tbl_name) -> spark.sql('INSERT OVERWRITE TABLE ...')
#      - self.statdate -> 硬编码 '20260513'
#      - self.spark -> 顶层 spark 变量
#   4) INSERT 用显式列名避免列序错位（50 列：imp_date + databus_imp_date + 48 业务列）
#   5) 沙箱 v3 表无分区，databus_imp_date 在输入表中已沙箱化为普通列
# ---
from __future__ import print_function

import sys
from pyspark.sql import SparkSession


if __name__ == "__main__":
    ds = '20260513'

    spark = (
        SparkSession.builder
        .appName("caseR18_dwd_log_80001099_daily")
        .enableHiveSupport()
        .getOrCreate()
    )

    # 确保输出表以正确 schema 存在
    spark.sql("DROP TABLE IF EXISTS internal_platform_db.caseR18_dwd_log_80001099_daily_v3")
    spark.sql('''
CREATE TABLE internal_platform_db.caseR18_dwd_log_80001099_daily_v3 (
    `imp_date` STRING COMMENT '运行日期(YYYYMMDD)',
    `databus_imp_date` STRING COMMENT '原分区日期(从输入透传)',
    `logid` BIGINT COMMENT 'logid',
    `svrtime` STRING COMMENT 'svrtime',
    `svrip` STRING COMMENT 'svrip',
    `module` STRING COMMENT 'module',
    `form_id` STRING COMMENT '表单唯一码',
    `form_type` BIGINT COMMENT '表单类型',
    `action` BIGINT COMMENT '动作类型',
    `retcode` BIGINT COMMENT '返回值',
    `vid` BIGINT COMMENT '用户帐号',
    `corpid` BIGINT COMMENT '用户公司账号',
    `gid` BIGINT COMMENT '用户gid',
    `appeal_kind` BIGINT COMMENT '申诉问题类型',
    `report_kind` BIGINT COMMENT '举报问题类型',
    `fraud_kind` BIGINT COMMENT '诈骗类型',
    `loss_amount` BIGINT COMMENT '损失金额',
    `suspect_vid` BIGINT COMMENT '举报嫌疑vid',
    `suspect_corpid` BIGINT COMMENT '举报嫌疑corpid',
    `order_id` STRING COMMENT '工单编号',
    `order_result` BIGINT COMMENT '工单结果',
    `order_source` BIGINT COMMENT '工单来源类型',
    `order_action` BIGINT COMMENT '工单动作',
    `match_rule` STRING COMMENT '工单相关的拦截规则',
    `roomid` BIGINT COMMENT '出问题的群号',
    `create_vid` BIGINT COMMENT '工单创建者vid',
    `openid` STRING COMMENT 'openid',
    `isfromwx` BIGINT COMMENT '拦截方:ww or wx',
    `spamtype` STRING COMMENT '审核定性',
    `auto_finish_order_rule` STRING COMMENT '自动结单策略',
    `action_time` BIGINT COMMENT '行为发生时间',
    `industry_name` STRING COMMENT '一级行业名称',
    `second_industry_name` STRING COMMENT '二级行业名称',
    `is_ka` BIGINT COMMENT 'corpid 是否为 ka',
    `create_time` STRING COMMENT '建单时间',
    `urgent_time` STRING COMMENT '催单时间',
    `result_time` STRING COMMENT '结单时间',
    `reopen_time` BIGINT COMMENT '重开时间(时间戳秒)',
    `block_uin` BIGINT COMMENT '实际拦截UIN',
    `block_user_id_type` BIGINT COMMENT '拦截UIN类型',
    `version` STRING COMMENT 'v1/v2 区分',
    `h5_source` BIGINT COMMENT '0:一线客服 1:用户自助',
    `h5_platform` BIGINT COMMENT '1:企业通讯平台B 2:平台W',
    `template_type` BIGINT COMMENT '在线申诉模板类型',
    `current_owner` STRING COMMENT '当前工单操作人',
    `oid` BIGINT COMMENT '申诉工单数字ID',
    `reason` STRING COMMENT 'QA结单证据原因(base64)',
    `order_finish_type` STRING COMMENT '工单结单类型',
    `reply_content` STRING COMMENT '话术回复(base64)',
    `match_rule_scene_id` BIGINT COMMENT '策略打击场景ID'
) STORED AS ORCFILE
''')

    # 反垃圾H5工单日志的每日明细同步
    sql = """
    INSERT OVERWRITE TABLE internal_platform_db.caseR18_dwd_log_80001099_daily_v3 (
        imp_date, databus_imp_date, logid, svrtime, svrip, module, form_id, form_type, action, retcode,
        vid, corpid, gid, appeal_kind, report_kind, fraud_kind, loss_amount, suspect_vid, suspect_corpid, order_id,
        order_result, order_source, order_action, match_rule, roomid, create_vid, openid, isfromwx, spamtype, auto_finish_order_rule,
        action_time, industry_name, second_industry_name, is_ka, create_time, urgent_time, result_time, reopen_time, block_uin, block_user_id_type,
        version, h5_source, h5_platform, template_type, current_owner, oid, reason, order_finish_type, reply_content, match_rule_scene_id
    )
    select '{0}' as imp_date,
        databus_imp_date,
        logid,
        svrtime,
        svrip,
        module,
        form_id,
        form_type,
        action,
        retcode,
        vid,
        corpid,
        gid,
        appeal_kind,
        report_kind,
        fraud_kind,
        loss_amount,
        suspect_vid,
        suspect_corpid,
        order_id,
        order_result,
        order_source,
        order_action,
        match_rule,
        roomid,
        create_vid,
        openid,
        isfromwx,
        spamtype,
        auto_finish_order_rule,
        action_time,
        industry_name,
        second_industry_name,
        is_ka,
        create_time,
        urgent_time,
        result_time,
        reopen_time,
        block_uin,
        block_user_id_type,
        version,
        h5_source,
        h5_platform,
        template_type,
        current_owner,
        oid,
        reason,
        order_finish_type,
        reply_content,
        match_rule_scene_id
    from internal_platform_db.caseR18_ods_log_80001099_v3
    """.format(ds)
    print("sql:")
    print(sql)
    spark.sql(sql)
    print("saved")

    spark.stop()
    sys.exit(0)