# 代码评审组织排行 (PySpark)

## Prompt
我需要你生成一段 PySpark 代码，产出代码协作平台代码评审（CR）按组织维度的排名表。

**业务背景与目标**：代码协作平台每天都会产生代码评审数据，包括评审注释（line_note）和评审耗时（used_time）。运营希望按公司/部门/中心/小组四级组织维度，对成员进行两个维度的 Top10 排名：(1) 按评审注释数降序排名（notes_create_desc）；(2) 按评审投入时间降序排名（cr_used_time_desc）。通过 rank_space 交叉连接生成完整的排名空间，左连接实际数据，产出组织排名明细表。

**输入表**：
- `internal_platform_db.dim_org_framework_org_d_f_pyspark_220`
- `internal_platform_db.dim_code_employee_user_org_relation_bare_d_f_pyspark_220`
- `internal_platform_db.dual_pyspark_220`
- `internal_platform_db.dwm_code_review_note_user_stat_d_f_pyspark_220`
- `internal_platform_db.dwd_code_review_used_time_d_i_pyspark_220`

**数据范围与过滤条件**：
- 运行日期 ds 固定为 '20260513'
- 月份范围：month_start='2026-05-01', month_end='2026-06-01'
- org 数据过滤 CONCAT(year,month,day) = ds
- note 数据过滤 concat(year,month,day) = ds，且 note_crt_date 在 [month_start, month_end) 区间
- used_time 数据过滤 dt 在 [month_start, month_end) 区间，且 (is_reviewer=1 OR is_fileowner=1)

**处理逻辑**：
1. 从 dim_org_framework_org 构建四级组织 CTE（cp 公司, dp 部门, ct 中心, gp 小组），每级提取对应的层级 ID
2. 将员工关联到各层级组织（cp_user, dp_user, ct_user, gp_user）
3. 计算 line_note_list（按 author_id 汇总 line_human_note_cnt）和 used_time_list（按 reviewer_id 汇总 duration_today）
4. 对每级组织的用户按两个 ranktype 分别做 rank() 窗口排序
5. 生成 rank_space（ranktype × rank 1-10 的笛卡尔积）
6. 各组织级别 cross join rank_space，left join 对应的排名结果
7. 四级结果 union all 后写入目标表

**输出要求**：
- 目标表：`internal_platform_db.ads_code_code_platform_mon_cr_org_rank_d_i_pyspark_220`
- 写入策略：INSERT OVERWRITE TABLE
- 输出列：month_name, month_start, month_end, org_id, org_type, ranktype, rank, user_id, user_name, value_string, value_int, value_float, value_datetime, created_at

## Reference Solution
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
pyspark_220: 代码协作平台CR组织排名 - Code Review Org Ranking (L3)
- 沙箱化改写：硬编码日期 '20260513'，移除 pytoolkit/dateutil 依赖
- 使用 spark.sql() 执行全部 CTE 逻辑
"""

import datetime
from pyspark.sql import SparkSession

spark = (
    SparkSession.builder
    .appName('pyspark_016_cr_org_rank')
    .enableHiveSupport()
    .getOrCreate()
)

# Hardcoded dates
ds = '20260513'
year = '2026'
month = '05'
day = '13'
month_start = '2026-05-01'
month_end = '2026-06-01'

db = 'internal_platform_db'
tbl = 'ads_code_code_platform_mon_cr_org_rank_d_i_pyspark_220'

# Drop and create output table
spark.sql('''
DROP TABLE IF EXISTS {db}.{tbl}
'''.format(db=db, tbl=tbl))

spark.sql('''
CREATE TABLE {db}.{tbl} (
    `month_name` STRING COMMENT 'cr创建时间转7位月份，yyyy-mm',
    `month_start` STRING COMMENT '月份开始时点',
    `month_end` STRING COMMENT '月份结束时点',
    `org_id` BIGINT COMMENT '组织id',
    `org_type` STRING COMMENT '组织类型',
    `ranktype` STRING COMMENT '排名类型',
    `rank` BIGINT COMMENT '排名',
    `user_id` BIGINT COMMENT '代码协作平台user_id',
    `user_name` STRING COMMENT 'user_name',
    `value_string` STRING COMMENT '排名所用文本',
    `value_int` BIGINT COMMENT '排名所用整数',
    `value_float` DOUBLE COMMENT '排名所用浮点数',
    `value_datetime` STRING COMMENT '排名所用时间',
    `created_at` STRING COMMENT '数据生成时间'
)
PARTITIONED BY (
    `year` STRING,
    `month` STRING,
    `day` STRING
)
STORED AS ORC
'''.format(db=db, tbl=tbl))

# Main ETL query
query_sql = """
    WITH org_list AS (
        SELECT org_id AS id, org_name AS name, company_id, company_name,
               org_bg_01_id, org_bg_02_id,
               business_line_01_id, business_line_02_id,
               department_01_id, department_02_id, department_03_id, department_04_id,
               center_01_id, center_02_id, center_03_id,
               group_01_id, group_02_id, group_03_id, group_04_id,
               group_05_id, group_06_id, group_07_id, group_08_id
        FROM internal_platform_db.dim_org_framework_org_d_f_pyspark_220
        WHERE CONCAT(year,month,day) = '{ds}'
    ),
    cp AS (
        SELECT o.id org_id,'company' org_type
            ,company_id
            ,-10000 org_bg_id
            ,-10000 business_line_id
            ,-10000 department_id
            ,-10000 center_id
            ,-10000 group_id
        FROM org_list AS o
        WHERE id = company_id
    ),
    dp AS (
        SELECT o.id org_id,'department' org_type
            ,company_id
            ,org_bg_01_id org_bg_id
            ,business_line_01_id business_line_id
            ,department_01_id department_id
            ,-10000 center_id
            ,-10000 group_id
        FROM org_list as o
        WHERE id = department_01_id
    ),
    ct AS (
        SELECT o.id org_id,'center' org_type
            ,company_id
            ,org_bg_01_id org_bg_id
            ,business_line_01_id business_line_id
            ,department_01_id department_id
            ,center_01_id center_id
            ,-10000 group_id
        FROM org_list as o
        WHERE id = center_01_id
    ),
    gp AS (
        SELECT o.id org_id,'group' org_type
            ,company_id
            ,org_bg_01_id org_bg_id
            ,business_line_01_id business_line_id
            ,department_01_id department_id
            ,center_01_id center_id
            ,group_01_id group_id
        FROM org_list as o
        WHERE id = group_01_id
    ),
    user_org as (
        select user_id,
            user_name,
            org_id,
            company_id,
            org_bg_id,
            business_line_id,
            department_id,
            center_id,
            group_id
        from internal_platform_db.dim_code_employee_user_org_relation_bare_d_f_pyspark_220
        where concat(year, month, day) = '{ds}'
    ),
    rank_space as (
        select t1.ranktype,t2.rank
        from (
            select 'notes_create_desc' as ranktype from internal_platform_db.dual_pyspark_220 union all
            select 'cr_used_time_desc' as ranktype from internal_platform_db.dual_pyspark_220
        ) as t1
        cross join (
            select 1  as rank from internal_platform_db.dual_pyspark_220 union all
            select 2  as rank from internal_platform_db.dual_pyspark_220 union all
            select 3  as rank from internal_platform_db.dual_pyspark_220 union all
            select 4  as rank from internal_platform_db.dual_pyspark_220 union all
            select 5  as rank from internal_platform_db.dual_pyspark_220 union all
            select 6  as rank from internal_platform_db.dual_pyspark_220 union all
            select 7  as rank from internal_platform_db.dual_pyspark_220 union all
            select 8  as rank from internal_platform_db.dual_pyspark_220 union all
            select 9  as rank from internal_platform_db.dual_pyspark_220 union all
            select 10 as rank from internal_platform_db.dual_pyspark_220
        ) as t2 on 1 = 1
    ),
    line_note_list as (
        select author_id user_id
            ,sum(line_human_note_cnt) line_human_note_cnt
        from internal_platform_db.dwm_code_review_note_user_stat_d_f_pyspark_220
        where concat(year,month,day) = '{ds}'
            and note_crt_date >= '{month_start}'
            and note_crt_date < '{month_end}'
        group by author_id
    ),
    used_time_list as (
        select reviewer_id user_id
            , sum(duration_today) duration_today
        from internal_platform_db.dwd_code_review_used_time_d_i_pyspark_220
        where concat(year,'-', month,'-', day) >= '{month_start}'
            and concat(year,'-', month,'-', day) < '{month_end}'
            and (is_reviewer = 1 or is_fileowner = 1)
        group by reviewer_id
    ),
    cp_user as (
        select t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
        from cp as t1 left join user_org as t2 on t1.org_id = t2.company_id
        group by t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
    ),
    dp_user as (
        select t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
        from dp as t1 left join user_org as t2 on t1.org_id = t2.department_id
        group by t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
    ),
    ct_user as (
        select t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
        from ct as t1 left join user_org as t2 on t1.org_id = t2.center_id
        group by t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
    ),
    gp_user as (
        select t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
        from gp as t1 left join user_org as t2 on t1.org_id = t2.group_id
        group by t1.org_id,t1.org_type,t1.company_id,t1.org_bg_id,t1.business_line_id,t1.department_id,t1.center_id,t1.group_id,t2.user_id,t2.user_name
    ),
    cp_rank as (
        select t1.*
            ,'notes_create_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.line_human_note_cnt  desc) rank
            ,null value_string
            ,t2.line_human_note_cnt value_int
            ,null value_float
            ,null value_datetime
        from cp_user as t1
        inner join line_note_list as t2 on t1.user_id = t2.user_id and t2.line_human_note_cnt > 0

        union all
        select t1.*
            ,'cr_used_time_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.duration_today  desc) rank
            ,null value_string
            ,t2.duration_today value_int
            ,null value_float
            ,null value_datetime
        from cp_user as t1
        inner join used_time_list as t2 on t1.user_id = t2.user_id and t2.duration_today > 0
    ),
    dp_rank as (
        select t1.*
            ,'notes_create_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.line_human_note_cnt  desc) rank
            ,null value_string
            ,t2.line_human_note_cnt value_int
            ,null value_float
            ,null value_datetime
        from dp_user as t1
        inner join line_note_list as t2 on t1.user_id = t2.user_id and t2.line_human_note_cnt > 0

        union all
        select t1.*
            ,'cr_used_time_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.duration_today  desc) rank
            ,null value_string
            ,t2.duration_today value_int
            ,null value_float
            ,null value_datetime
        from dp_user as t1
        inner join used_time_list as t2 on t1.user_id = t2.user_id and t2.duration_today > 0
    ),
    ct_rank as (
        select t1.*
            ,'notes_create_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.line_human_note_cnt  desc) rank
            ,null value_string
            ,t2.line_human_note_cnt value_int
            ,null value_float
            ,null value_datetime
        from ct_user as t1
        inner join line_note_list as t2 on t1.user_id = t2.user_id and t2.line_human_note_cnt > 0

        union all
        select t1.*
            ,'cr_used_time_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.duration_today  desc) rank
            ,null value_string
            ,t2.duration_today value_int
            ,null value_float
            ,null value_datetime
        from ct_user as t1
        inner join used_time_list as t2 on t1.user_id = t2.user_id and t2.duration_today > 0
    ),
    gp_rank as (
        select t1.*
            ,'notes_create_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.line_human_note_cnt  desc) rank
            ,null value_string
            ,t2.line_human_note_cnt value_int
            ,null value_float
            ,null value_datetime
        from gp_user as t1
        inner join line_note_list as t2 on t1.user_id = t2.user_id and t2.line_human_note_cnt > 0

        union all
        select t1.*
            ,'cr_used_time_desc' ranktype
            ,rank() over(partition by t1.org_id order by t2.duration_today  desc) rank
            ,null value_string
            ,t2.duration_today value_int
            ,null value_float
            ,null value_datetime
        from gp_user as t1
        inner join used_time_list as t2 on t1.user_id = t2.user_id and t2.duration_today > 0
    )
    INSERT overwrite TABLE {db}.{tbl} partition(YEAR='{year}',MONTH='{month}',DAY='{day}')
    SELECT /*+ REPARTITION(64) */
        *
    from (
        select '{year}-{month}' month_name
            ,'{month_start} 00:00:00' month_start
            ,'{month_end} 00:00:00'   month_end
            ,t1.org_id
            ,t1.org_type
            ,t2.ranktype
            ,t2.rank
            ,t3.user_id
            ,t3.user_name
            ,t3.value_string
            ,t3.value_int
            ,t3.value_float
            ,t3.value_datetime
            ,CAST('2026-05-13 00:00:00' AS STRING) AS created_at
        from cp as t1
        cross join rank_space as t2 on 1 = 1
        left join cp_rank as t3 on t1.org_id = t3.org_id and t2.rank = t3.rank and t2.ranktype = t3.ranktype

        union all
        select '{year}-{month}' month_name
            ,'{month_start} 00:00:00' month_start
            ,'{month_end} 00:00:00'   month_end
            ,t1.org_id
            ,t1.org_type
            ,t2.ranktype
            ,t2.rank
            ,t3.user_id
            ,t3.user_name
            ,t3.value_string
            ,t3.value_int
            ,t3.value_float
            ,t3.value_datetime
            ,CAST('2026-05-13 00:00:00' AS STRING) AS created_at
        from dp as t1
        cross join rank_space as t2 on 1 = 1
        left join dp_rank as t3 on t1.org_id = t3.org_id and t2.rank = t3.rank and t2.ranktype = t3.ranktype

        union all
        select '{year}-{month}' month_name
            ,'{month_start} 00:00:00' month_start
            ,'{month_end} 00:00:00'   month_end
            ,t1.org_id
            ,t1.org_type
            ,t2.ranktype
            ,t2.rank
            ,t3.user_id
            ,t3.user_name
            ,t3.value_string
            ,t3.value_int
            ,t3.value_float
            ,t3.value_datetime
            ,CAST('2026-05-13 00:00:00' AS STRING) AS created_at
        from ct as t1
        cross join rank_space as t2 on 1 = 1
        left join ct_rank as t3 on t1.org_id = t3.org_id and t2.rank = t3.rank and t2.ranktype = t3.ranktype

        union all
            select '{year}-{month}' month_name
            ,'{month_start} 00:00:00' month_start
            ,'{month_end} 00:00:00'   month_end
            ,t1.org_id
            ,t1.org_type
            ,t2.ranktype
            ,t2.rank
            ,t3.user_id
            ,t3.user_name
            ,t3.value_string
            ,t3.value_int
            ,t3.value_float
            ,t3.value_datetime
            ,CAST('2026-05-13 00:00:00' AS STRING) AS created_at
        from gp as t1
        cross join rank_space as t2 on 1 = 1
        left join gp_rank as t3 on t1.org_id = t3.org_id and t2.rank = t3.rank and t2.ranktype = t3.ranktype
    )
""".format(
    db=db, tbl=tbl,
    year=year, month=month, day=day, ds=ds,
    month_start=month_start, month_end=month_end,
)

spark.sql(query_sql)

print("pyspark_016 done")

spark.stop()