# Anomaly Detection Analysis (PySpark)

## Prompt
I need you to generate a PySpark script that implements anomaly detection analysis based on business data.

**Business Background and Objective**: In the advertising delivery business, operations needs to monitor changes in key metrics (such as `real_cost`) across different industry (`industry_for_business`) dimensions. By comparing and analyzing metric data between the pre-period and post-period, the metrics are summarized by industry dimension, labeling whether the data belongs to the pre-analysis period (`is_post=0`) or the post-analysis period (`is_post=1`). The anomaly detection analysis result table is produced for downstream analysis modules to use.

**Input Table**:
- `internal_platform_db.f_account_manage_analysis_d_pyspark_208`

**Data Range and Filter Conditions**:
- Run date `ds` is fixed as `'20260513'`
- Pre-period: `process_time BETWEEN '20260510' AND '20260512'`
- Post-period: `process_time BETWEEN '20260513' AND '20260513'`
- Filter `partition_time = '20260513'`

**Processing Logic**:
1. Read data from the input table where `partition_time='20260513'`
2. Divide into pre-period (pre, `is_post=0`) and post-period (post, `is_post=1`) by `process_time`
3. Aggregate by the `industry_for_business` dimension, computing `SUM(real_cost)` as the metric value
4. Union the pre and post results and write to the target table

**Output Requirements**:
- Target table: `internal_platform_db.abnormal_analysis_result_pyspark_208`
- Write strategy: `INSERT OVERWRITE TABLE`
- Output columns: `is_post` (INT, 0=pre-period, 1=post-period), `industry_for_business` (STRING, industry), `real_cost` (DOUBLE, sum of `real_cost`)

## Reference Solution
#!/usr/bin/env python3
"""pyspark_017 ground truth: anomaly detection analysis (simplified)"""
from pyspark.sql import SparkSession

spark = SparkSession.builder \
    .appName('dataclaw_eval_gt_pyspark_017') \
    .enableHiveSupport() \
    .config('spark.sql.warehouse.dir', '/tmp/hive_warehouse') \
    .getOrCreate()

# Hardcoded task configuration (originally from MySQL t_abnormal_detection_and_analysis_task_conf)
# task_type=8 means use f_account_manage_analysis_d as source table
task_conf = {
    'id': 1,
    'pre_start_date': '20260510',
    'pre_end_date': '20260512',
    'post_start_date': '20260513',
    'post_end_date': '20260513',
    'metrics': 'real_cost',
    'dimensions': 'industry_for_business',
    'conditions': '[]',
    'partition_time': '20260513',
    'task_type': 8
}

partition = task_conf['partition_time']
pre_start = task_conf['pre_start_date']
pre_end = task_conf['pre_end_date']
post_start = task_conf['post_start_date']
post_end = task_conf['post_end_date']
metrics = task_conf['metrics']
dimension_col = task_conf['dimensions']

# Create output table DDL
spark.sql('''
DROP TABLE IF EXISTS internal_platform_db.abnormal_analysis_result_pyspark_208
''')
spark.sql('''
CREATE TABLE IF NOT EXISTS internal_platform_db.abnormal_analysis_result_pyspark_208 (
    is_post INT COMMENT '0=pre period, 1=post period',
    industry_for_business STRING COMMENT '行业',
    real_cost DOUBLE COMMENT 'real_cost日均'
)
STORED AS ORC
''')

# Read data from input table filtered by partition_time
input_df = spark.sql(f'''
SELECT
    process_time,
    industry_for_business,
    real_cost
FROM internal_platform_db.f_account_manage_analysis_d_pyspark_208
WHERE partition_time = '{partition}'
    AND process_time BETWEEN '{pre_start}' AND '{post_end}'
''')

# Compute pre-period metrics (simple aggregation: sum real_cost per industry_for_business)
pre_df = spark.sql(f'''
SELECT
    0 AS is_post,
    industry_for_business,
    SUM(real_cost) AS real_cost
FROM internal_platform_db.f_account_manage_analysis_d_pyspark_208
WHERE partition_time = '{partition}'
    AND process_time BETWEEN '{pre_start}' AND '{pre_end}'
GROUP BY industry_for_business
''')

# Compute post-period metrics
post_df = spark.sql(f'''
SELECT
    1 AS is_post,
    industry_for_business,
    SUM(real_cost) AS real_cost
FROM internal_platform_db.f_account_manage_analysis_d_pyspark_208
WHERE partition_time = '{partition}'
    AND process_time BETWEEN '{post_start}' AND '{post_end}'
GROUP BY industry_for_business
''')

# Union pre and post results
result_df = pre_df.union(post_df)
result_df.show()

# Write result to output table
result_df.write.format("orc").mode('overwrite').saveAsTable(
    'internal_platform_db.abnormal_analysis_result_pyspark_208'
)

print('Ground truth computation complete')
spark.stop()