# 被动客服满意度归因环比对比 (PySpark)

## Prompt
我需要你生成一段 PySpark 代码，产出"被动客服会话满意度归因因素"的环比对比表。

**业务背景与目标**：被动客服每天都会按 reason / reason_detail / session_type 维度产出一份满意度归因明细（dwd_ww_kf_session_satify_attrib）。运营希望对每一条归因因素同时做两个维度的对比：(1) 与 14 天前同一 reason+reason_detail+session_type 的指标做差值；(2) 与上月相同类型日期（工作日 vs 工作日、周末 vs 周末）的平均指标做差值。本任务把当天、14 天前、上月这三组数据拉齐后，逐条计算满意度差值、占比差值、贡献率，输出到下游对比表里供运营复盘。

**输入表**：
- `internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_v3`

**数据范围与过滤条件（业务说法）**：
- 运行日期 statdate 固定为 '20260513'（沙箱化日期，原任务由调度框架传入）。
- 14 天前日期 = statdate - 14 天 = '20260429'。
- 上月相同类型日期：根据 statdate 的工作日/周末属性，从上一个自然月里取出所有同类型日期组成日期列表（例如 statdate 是周三，则取上月所有工作日）；用该列表 IN 过滤主表得到"上月同类型样本"。
- 三组切片都从同一张明细表过滤得到，分别注册成临时视图 today / 14d / month_avg。

**表关联关系**：
- left join：当天视图 t0 ⋈ 14 天前视图 t1 on (t0.reason=t1.reason AND t0.reason_detail=t1.reason_detail AND t0.session_type=t1.session_type) → df_compare_14d。
- left join：当天视图 t0 ⋈ 上月平均视图 t1 on 同上 → df_compare_month。
- df_compare_14d 与 df_compare_month 通过 union all 拼接成 df_all 一次性写入。

**聚合与计算规则（需体现在 SQL/PySpark 中）**：
1. **上月平均聚合**：对上月切片，先按 session_type 计算月级总量 `all_month_session_cnt = sum(all_session_cnt)` 和加权满意度 `all_month_session_score = sum(all_session_satify_score*all_session_cnt) / sum(all_session_cnt)`；再按 (session_type, reason, reason_detail) 聚合，得到月度 satify_score = sum(session_cnt*satify_score)/sum(session_cnt)、session_rate = sum(session_cnt)/all_month_session_cnt、satify_attrib = (sum(session_cnt*satify_score*session_rate/100)/sum(session_cnt))/all_month_session_score - 1。
2. **派生差值字段**（每条 reason+reason_detail+session_type 行同时计算 14 天对比和上月对比两组）：
   - all_session_satify_diff = t0.all_session_satify_score - t1.all_session_satify_score
   - satify_score_diff = t0.satify_score - t1.satify_score
   - session_rate_diff = t0.session_rate - t1.session_rate
   - satify_attrib_diff = (t0.satify_score*t0.session_rate - t1.satify_score*t1.session_rate) / 100
   - satify_attrib_rate = satify_attrib_diff / all_session_satify_diff（贡献率）
   - self_satify_attrib_rate = (t0.satify_score*t0.session_rate) / (t1.satify_score*t1.session_rate) - 1（自身环比）
   - compare_satify_attrib_diff = t0.satify_attrib - t1.satify_attrib
3. **compare_interval 字段**：14 天对比那批固定写 '14天前'；上月对比那批固定写 '上月'。
4. **输出 imp_date** 全部填当天 statdate '20260513'。

**输出要求**：
- 目标表：`internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_compare_v3`
- 写入策略：INSERT OVERWRITE TABLE，使用显式列名清单（22 列）避免列序错位。
- 如果目标表不存在，请先按 Hive 标准建表（ORC 存储），再写入数据。

## Reference Solution
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
caseR1: 被动客服满意度归因 - 14天/上月对比 (SparkBase -> SparkSession 改写)
- 沙箱无分区版：原 part_name=last14day / part_name=part_names 改成 WHERE imp_date='...' / WHERE imp_date IN (...)
- 原 self.statdate 硬编码为 '20260513'（沙箱化日期）
- 原 self.save_dw_data 改为 spark.sql INSERT OVERWRITE TABLE
"""

import datetime
from pyspark.sql import SparkSession

spark = (
    SparkSession.builder
    .appName('caseR1_satify_attrib_compare')
    .enableHiveSupport()
    .getOrCreate()
)

# 沙箱化运行日期（fixture 模板硬编码）
statdate = '20260513'

# ---------------------------- 14 天前 ----------------------------
last14day = (datetime.datetime.strptime(statdate, "%Y%m%d") - datetime.timedelta(days=14)).strftime("%Y%m%d")
print(statdate, last14day)

# 当天视图
spark.sql("""
SELECT * FROM internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_v3
WHERE imp_date = '{0}'
""".format(statdate)).createOrReplaceTempView("dwd_ww_kf_session_satify_attrib_today")

# 14 天前视图
spark.sql("""
SELECT * FROM internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_v3
WHERE imp_date = '{0}'
""".format(last14day)).createOrReplaceTempView("dwd_ww_kf_session_satify_attrib_14d")

sql_14d = """
SELECT
    "{0}" AS imp_date
    ,'14天前' AS compare_interval
    ,t0.reason
    ,t0.reason_detail
    ,CAST(t0.session_cnt AS DOUBLE) AS session_cnt
    ,t0.satify_score
    ,t0.session_rate
    ,t0.all_session_satify_score
    ,t0.satify_attrib
    ,t1.all_session_satify_score AS compare_all_session_satify_score
    ,t1.satify_score AS compare_satify_score
    ,t1.session_rate AS compare_session_rate
    ,t0.all_session_satify_score - t1.all_session_satify_score AS all_session_satify_diff
    ,t0.satify_score - t1.satify_score AS satify_score_diff
    ,t0.session_rate - t1.session_rate AS session_rate_diff
    ,(t0.satify_score*t0.session_rate - t1.satify_score*t1.session_rate)/100 AS satify_attrib_diff
    ,((t0.satify_score*t0.session_rate - t1.satify_score*t1.session_rate)/100)/(t0.all_session_satify_score - t1.all_session_satify_score) AS satify_attrib_rate
    ,(t0.satify_score*t0.session_rate)/(t1.satify_score*t1.session_rate) - 1 AS self_satify_attrib_rate
    ,CAST(t1.session_cnt AS DOUBLE) AS compare_session_cnt
    ,t1.satify_attrib AS compare_satify_attrib
    ,t0.satify_attrib - t1.satify_attrib AS compare_satify_attrib_diff
    ,t0.session_type
FROM dwd_ww_kf_session_satify_attrib_today t0
LEFT JOIN dwd_ww_kf_session_satify_attrib_14d t1
    ON t0.reason = t1.reason AND t0.reason_detail = t1.reason_detail AND t0.session_type = t1.session_type
""".format(statdate)
df_compare_14d = spark.sql(sql_14d).cache()

# ---------------------------- 上月对比 ----------------------------
# 纯 datetime 实现 lastmonth_firstday，避免依赖 dateutil
_today = datetime.datetime.strptime(statdate, "%Y%m%d")
_thismonth_first = _today.replace(day=1)
# 上月第 1 天 = 本月第 1 天 - 1 天 → 取该日期所在月的第 1 天
_lastmonth_anchor = _thismonth_first - datetime.timedelta(days=1)
lastmonth_firstday = _lastmonth_anchor.replace(day=1).strftime("%Y%m%d")
thismonth_firstday = statdate[0:6] + "01"
interval = (datetime.datetime.strptime(thismonth_firstday, "%Y%m%d")
            - datetime.datetime.strptime(lastmonth_firstday, "%Y%m%d"))

weekday_list = []
holiday_list = []
for i in range(interval.days):
    date = (datetime.datetime.strptime(lastmonth_firstday, "%Y%m%d")
            + datetime.timedelta(days=i)).strftime("%Y%m%d")
    week_flag = datetime.datetime.strptime(date, "%Y%m%d").weekday()
    if week_flag in (0, 1, 2, 3, 4):
        weekday_list.append(date)
    else:
        holiday_list.append(date)

today_week = datetime.datetime.strptime(statdate, "%Y%m%d").weekday()
part_names = weekday_list if today_week in (0, 1, 2, 3, 4) else holiday_list

# 上月对应类型日期视图（多分区联合）
in_clause = ",".join(["'%s'" % d for d in part_names])
spark.sql("""
SELECT * FROM internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_v3
WHERE imp_date IN ({0})
""".format(in_clause)).createOrReplaceTempView("dwd_ww_kf_session_satify_attrib_lastmonth")

# 上月统计
spark.sql("""
SELECT
    session_type
    ,count(distinct imp_date) AS date_cnt
    ,sum(all_session_cnt) AS all_month_session_cnt
    ,sum(all_session_satify_score*all_session_cnt)/sum(all_session_cnt) AS all_month_session_score
FROM (
    SELECT DISTINCT imp_date,session_type,all_session_satify_score,all_session_cnt
    FROM dwd_ww_kf_session_satify_attrib_lastmonth
)
GROUP BY session_type
""").createOrReplaceTempView("lastmonth_stat")

# 每类因素的上月平均
spark.sql("""
SELECT
    t0.session_type
    ,reason
    ,reason_detail
    ,all_month_session_cnt
    ,all_month_session_score AS all_session_satify_score
    ,sum_session_cnt
    ,sum_session_cnt/date_cnt session_cnt
    ,satify_score
    ,sum_session_cnt*100/all_month_session_cnt AS session_rate
    ,(satify_score - all_month_session_score)*(sum_session_cnt/all_month_session_cnt) AS satify_attrib
FROM (
    SELECT
        session_type
        ,reason
        ,reason_detail
        ,sum(session_cnt) sum_session_cnt
        ,sum(session_cnt*satify_score)/sum(session_cnt) AS satify_score
    FROM dwd_ww_kf_session_satify_attrib_lastmonth
    GROUP BY session_type,reason,reason_detail
) t0
LEFT JOIN lastmonth_stat t1 ON t0.session_type = t1.session_type
""").createOrReplaceTempView("month_avg_stat")

sql_month = """
SELECT
    "{0}" AS imp_date
    ,'上月' AS compare_interval
    ,t0.reason
    ,t0.reason_detail
    ,CAST(t0.session_cnt AS DOUBLE) AS session_cnt
    ,t0.satify_score
    ,t0.session_rate
    ,t0.all_session_satify_score
    ,t0.satify_attrib
    ,t1.all_session_satify_score AS compare_all_session_satify_score
    ,t1.satify_score AS compare_satify_score
    ,t1.session_rate AS compare_session_rate
    ,t0.all_session_satify_score - t1.all_session_satify_score AS all_session_satify_diff
    ,t0.satify_score - t1.satify_score AS satify_score_diff
    ,t0.session_rate - t1.session_rate AS session_rate_diff
    ,(t0.satify_score*t0.session_rate - t1.satify_score*t1.session_rate)/100 AS satify_attrib_diff
    ,((t0.satify_score*t0.session_rate - t1.satify_score*t1.session_rate)/100)/(t0.all_session_satify_score - t1.all_session_satify_score) AS satify_attrib_rate
    ,(t0.satify_score*t0.session_rate)/(t1.satify_score*t1.session_rate) - 1 AS self_satify_attrib_rate
    ,CAST(t1.session_cnt AS DOUBLE) AS compare_session_cnt
    ,t1.satify_attrib AS compare_satify_attrib
    ,t0.satify_attrib - t1.satify_attrib AS compare_satify_attrib_diff
    ,t0.session_type
FROM dwd_ww_kf_session_satify_attrib_today t0
LEFT JOIN month_avg_stat t1
    ON t0.reason = t1.reason AND t0.reason_detail = t1.reason_detail AND t0.session_type = t1.session_type
""".format(statdate)
df_compare_month = spark.sql(sql_month).cache()

df_all = df_compare_month.union(df_compare_14d)
df_all.createOrReplaceTempView("df_all_view")

# 确保输出表以正确 schema 存在（agent 可能未建表或建了错误 schema）
spark.sql("DROP TABLE IF EXISTS internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_compare_v3")
spark.sql('''
CREATE TABLE internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_compare_v3 (
    `compare_interval` STRING,
    `reason` STRING,
    `reason_detail` STRING,
    `session_cnt` DOUBLE,
    `satify_score` DOUBLE,
    `session_rate` DOUBLE,
    `all_session_satify_score` DOUBLE,
    `satify_attrib` DOUBLE,
    `compare_all_session_satify_score` DOUBLE,
    `compare_satify_score` DOUBLE,
    `compare_session_rate` DOUBLE,
    `all_session_satify_diff` DOUBLE,
    `satify_score_diff` DOUBLE,
    `session_rate_diff` DOUBLE,
    `satify_attrib_diff` DOUBLE,
    `satify_attrib_rate` DOUBLE,
    `self_satify_attrib_rate` DOUBLE,
    `compare_session_cnt` DOUBLE,
    `compare_satify_attrib` DOUBLE,
    `compare_satify_attrib_diff` DOUBLE,
    `session_type` BIGINT,
    `imp_date` STRING
) STORED AS ORC
''')

# 落地：DDL 列序 (compare_interval, reason, ..., session_type, imp_date)
spark.sql("""
INSERT OVERWRITE TABLE internal_platform_db.caseR1_dwd_ww_kf_session_satify_attrib_compare_v3
(
    compare_interval, reason, reason_detail, session_cnt, satify_score, session_rate,
    all_session_satify_score, satify_attrib, compare_all_session_satify_score,
    compare_satify_score, compare_session_rate, all_session_satify_diff, satify_score_diff,
    session_rate_diff, satify_attrib_diff, satify_attrib_rate, self_satify_attrib_rate,
    compare_session_cnt, compare_satify_attrib, compare_satify_attrib_diff, session_type, imp_date
)
SELECT
    compare_interval, reason, reason_detail, session_cnt, satify_score, session_rate,
    all_session_satify_score, satify_attrib, compare_all_session_satify_score,
    compare_satify_score, compare_session_rate, all_session_satify_diff, satify_score_diff,
    session_rate_diff, satify_attrib_diff, satify_attrib_rate, self_satify_attrib_rate,
    compare_session_cnt, compare_satify_attrib, compare_satify_attrib_diff, session_type, imp_date
FROM df_all_view
""")

print("caseR1 done")