# Sync Cloud-Check User Active Status to Application Layer (PySpark)

## Prompt
I need you to generate a PySpark script that synchronizes the "user cloud-check active status" detail from ODS to the application layer DWV, and applies compliance encryption to the phone number to generate a unionid.

**Business Background and Objective**: In the upstream ODS, the `phone_enc` field is the bottom-layer encrypted phone number. The downstream application layer wants to further convert it into a `unionid` column (referred to as `phone` in business terms) using a deterministic one-way encryption. In addition to encryption, the active status fields (inactive days, sleep flag, first/recent/last cloud-check dates) must be passed through as-is, written out by `ds`.

**Input Table**:
- `internal_platform_db.case15_ods_phone_active_df_v3`

**Data Range and Filter Conditions (business description)**:
  - Only take the `ds` = current day partition; for this run, `ds` is fixed as `'20260513'` (the original task passes this via `sys.argv[1]`; TaskType=63 does not pass command-line arguments, so it is hardcoded to avoid IndexError).

**Table Join Relationships**: (No joins, single-table extraction.)

**Aggregation and Computation Rules (must be reflected in the SQL)**:
  1. Derived column `phone`: The original task encrypts `phone_enc` via the `OpenidCrypt` UDF; this UDF's jar (`hdfs://cluster-alpha/...`) is unavailable in the sandbox (tested exit code 1), so `sha2(phone_enc, 256)` is used as a replacement — also a deterministic one-way encryption, with equivalent business semantics.
  2. Column renaming (alias during passthrough):
     - `his_call_date_cloud` → `his_call_date_total`
     - `first_call_date_cloud` → `first_call_date_total`
     - `last_call_date_cloud` → `last_call_date_total`
  3. Other fields (`phone_type`, `not_active_days`, `is_sleep_flag`, `ds`) are passed through as-is.

**Output Requirements**:
  - Target table: `internal_platform_db.case15_dwv_phone_regul_unionid_df_v3`
  - Output table schema (in this order, with the following semantics):
    1. `phone` STRING: encrypted unionid
    2. `phone_type` STRING: phone number type
    3. `not_active_days` INT: recent consecutive inactive days
    4. `is_sleep_flag` STRING: whether currently in sleep state
    5. `his_call_date_total` STRING: historical first query date
    6. `first_call_date_total` STRING: recent first query date
    7. `last_call_date_total` STRING: last query date
    8. `ds` BIGINT: business date (originally a partition column, sandboxed as a regular column)
  - Write strategy: `INSERT OVERWRITE TABLE`, using an explicit column name list (8 columns) to avoid column order misalignment; config `spark.sql.storeAssignmentPolicy=LEGACY` to accommodate implicit type conversion.
  - Sandbox note: The original `PARTITION(ds={ds})` write is changed to a regular column + `WHERE ds = {ds}` regular column filter; the `OpenidCrypt` UDF is replaced with `sha2(phone_enc, 256)`; `jar_addr` addJar and `registerJavaFunction` are removed.
  - If the target table does not exist, first create it using standard Hive format (ORC storage), then write the data.

## Reference Solution
# data-mocker rewrite-gt
# source: v7 xlsx row 16 (case15)
# task_name: t_app_regul_user_comm_security_cloud_active_status_phone_uid_df_copilot
# rewrite rules:
#   1) internal_platform_db.t_app_comm_security_user_cloud_active_status_phone_df_copilot
#       -> internal_platform_db.case15_ods_phone_active_df_v3
#   2) internal_platform_db.t_app_regul_user_comm_security_cloud_active_status_phone_uid_df_copilot
#       -> internal_platform_db.case15_dwv_phone_regul_unionid_df_v3
#   3) PARTITION(ds=...) 删除；output 表 ds 改为普通列，SELECT 头部补 ds 列
#      WHERE ds = {ds} 保留（普通列过滤）
#      INSERT 用显式列名避免列序错位
#   4) sys.argv[1] -> 硬编码 '20260513'
#      (datawd PySpark TaskType=63 不直传命令行参数)
#   5) OpenidCrypt UDF -> sha2(phone_enc, 256)
#      (原 jar hdfs://cluster-alpha/... 在沙箱不可用，实测 exit code 1；
#       sha2 同样是确定性单向加密，业务语义等价)
#   6) 移除 jar_addr addJar + registerJavaFunction（不再需要）
# ---
import sys
from pyspark.sql import SparkSession

"""

脚本说明：API-SDK云查（sha2 加密版）

"""


# 生成和配置spark实例
def get_spark_cli():
    spark = SparkSession.builder \
        .enableHiveSupport() \
        .config("spark.sql.storeAssignmentPolicy", "LEGACY") \
        .getOrCreate()
    return spark


# 生成spark sql语句
def get_sql_str():
    ds = '20260513'  # was: sys.argv[1]
    sql = f"""
INSERT OVERWRITE TABLE internal_platform_db.case15_dwv_phone_regul_unionid_df_v3
(phone, phone_type, not_active_days, is_sleep_flag, his_call_date_total, first_call_date_total, last_call_date_total, ds)
SELECT
sha2(phone_enc, 256) as phone,
phone_type,
not_active_days,
is_sleep_flag,
his_call_date_cloud as his_call_date_total,
first_call_date_cloud as first_call_date_total,
last_call_date_cloud as last_call_date_total,
ds
FROM internal_platform_db.case15_ods_phone_active_df_v3
where ds = {ds}
"""
    print(f"sql: {sql}")
    return sql


# 执行
if __name__ == "__main__":
    spark = get_spark_cli()
    # 确保输出表以正确 schema 存在
    spark.sql("DROP TABLE IF EXISTS internal_platform_db.case15_dwv_phone_regul_unionid_df_v3")
    spark.sql('''
CREATE TABLE internal_platform_db.case15_dwv_phone_regul_unionid_df_v3 (
    `phone` STRING COMMENT '加密后的 unionid',
    `phone_type` STRING COMMENT '号码类型',
    `not_active_days` INT COMMENT '最近连续不活跃天数',
    `is_sleep_flag` STRING COMMENT '当前是否处于休眠状态',
    `his_call_date_total` STRING COMMENT '历史首次查询日期',
    `first_call_date_total` STRING COMMENT '最近首次查询日期',
    `last_call_date_total` STRING COMMENT '最后查询日期',
    `ds` BIGINT COMMENT 'partition fields（原为分区列）'
) STORED AS ORCFILE
''')
    SQL = get_sql_str()
    spark.sql(SQL)