# 用户标签同步ETL (PySpark)

## Prompt
我需要你生成一段 PySpark 代码，完成"用户标签同步ETL"的数据加工任务。

**业务背景与目标**：安全平台每天从 event_log 日志中采集用户设备与标签数据（t_sh_event_log_v2_xab00014231），数据中包含手机号、IMEI、IMSI、AndroidID 等敏感信息，这些字段在原始日志中已经过在线加密（OnlineCrypt）存储为加密字段。本任务需要将加密字段解密后，使用加密算法重新加密，同时对数据类型字段做 INT 转换，最终写入下游用户标签同步表（t_ods_comm_security_user_tag_sync_hi）供安全分析使用。

**输入表**：
- `internal_platform_db.t_sh_event_log_v2_xab00014231_pyspark_254`

**数据范围与过滤条件（业务说法）**：
- 运行日期 ds 固定为 '20260513'（沙箱化日期，原任务由调度框架传入）。
- 从输入表中读取 ds = '20260513' 分区的全量数据，不做额外过滤。

**计算规则（需体现在 SQL/PySpark 中）**：
1. **字段映射与类型转换**：
   - server_ip、tag_name、ip、ip_md5、ip_c_segment、manufacture、model、language、region、sdk_version、ip_country、ip_province、ip_city、ip_district → 原样透传
   - `time` → log_time（字段重命名）
   - src → CAST(src AS INT)
   - phonetype → CAST(phonetype AS INT)
   - apn → CAST(apn AS INT)
   - auth_type → CAST(auth_type AS INT)
   - call_type → CAST(call_type AS INT)
   - duration → CAST(duration AS INT)

2. **敏感字段加密**（原逻辑：OnlineCrypt 解密 → IdCrypt 重新加密；沙箱改写为 sha2 哈希）：
   - phonenum → sha2(phonenum, 256) AS phonenum_enc
   - self_phone → sha2(self_phone, 256) AS self_phone_enc
   - imei → sha2(imei, 256) AS imei_enc
   - imsi → sha2(imsi, 256) AS imsi_enc
   - phonenum_reg → sha2(phonenum_reg, 256) AS phonenum_reg_enc
   - android_id → sha2(android_id, 256) AS android_id_enc

**输出要求**：
- 目标表：`internal_platform_db.t_ods_comm_security_user_tag_sync_hi_pyspark_254`
- 写入策略：INSERT OVERWRITE TABLE，使用显式分区写入 partition(ds='20260513')
- 如果目标表不存在，请先按 Hive 标准建表（ORC 存储，分区列 ds），再写入数据。

## Reference Solution
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
pyspark_015: 用户标签同步ETL (Java UDF -> sha2 改写)
- 原 pytoolkit/IdCrypt/OnlineCrypt 替换为内置 sha2() 函数
- 原 self.statdate 硬编码为 '20260513'（沙箱化日期）
- 原 self.save_dw_data 改为 spark.sql INSERT OVERWRITE TABLE
"""

from pyspark.sql import SparkSession

spark = (
    SparkSession.builder
    .appName('pyspark_015_user_tag_sync')
    .enableHiveSupport()
    .getOrCreate()
)

ds = '20260513'

# Create output table DDL
spark.sql('DROP TABLE IF EXISTS internal_platform_db.t_ods_comm_security_user_tag_sync_hi_pyspark_254')
spark.sql('''
CREATE TABLE internal_platform_db.t_ods_comm_security_user_tag_sync_hi_pyspark_254 (
  server_ip STRING,
  log_time STRING,
  src INT,
  phonenum_enc STRING,
  tag_name STRING,
  self_phone_enc STRING,
  ip STRING,
  imei_enc STRING,
  imsi_enc STRING,
  phonenum_reg_enc STRING,
  phonetype INT,
  apn INT,
  ip_md5 STRING,
  ip_c_segment STRING,
  auth_type INT,
  android_id_enc STRING,
  manufacture STRING,
  model STRING,
  language STRING,
  region STRING,
  sdk_version STRING,
  call_type INT,
  duration INT,
  ip_country STRING,
  ip_province STRING,
  ip_city STRING,
  ip_district STRING
) PARTITIONED BY (ds STRING)
STORED AS ORC
''')

# Business logic: read from source, apply transformations, write to target
# Original: OnlineCrypt decrypt encrypted fields -> IdCrypt re-encrypt
# Rewrite: sha2() on plaintext fields to produce encrypted output
spark.sql("""
INSERT OVERWRITE TABLE internal_platform_db.t_ods_comm_security_user_tag_sync_hi_pyspark_254 partition(ds='{ds}')
SELECT  server_ip
        ,`time` AS log_time
        ,CAST(src AS INT) AS src
        ,sha2(phonenum, 256) AS phonenum_enc
        ,tag_name
        ,sha2(self_phone, 256) AS self_phone_enc
        ,ip
        ,sha2(imei, 256) AS imei_enc
        ,sha2(imsi, 256) AS imsi_enc
        ,sha2(phonenum_reg, 256) AS phonenum_reg_enc
        ,CAST(phonetype AS INT) AS phonetype
        ,CAST(apn AS INT) AS apn
        ,ip_md5
        ,ip_c_segment
        ,CAST(auth_type AS INT) AS auth_type
        ,sha2(android_id, 256) AS android_id_enc
        ,manufacture
        ,model
        ,language
        ,region
        ,sdk_version
        ,CAST(call_type AS INT) AS call_type
        ,CAST(duration AS INT) AS duration
        ,ip_country
        ,ip_province
        ,ip_city
        ,ip_district
FROM    internal_platform_db.t_sh_event_log_v2_xab00014231_pyspark_254
WHERE   ds = '{ds}'
""".format(ds=ds))

print("pyspark_015 done")

spark.stop()