Repository: aws/aws-sdk-pandas

Issue #2826: athena.to_iceberg function is not deleting temp_table_xxxxx properly in Athena
Created: 2024-05-21T04:38:58Z
Describe the bug
When calling to_iceberg function, I found out that some of the temp table (ex. temp_table_xxxxx) created by the function persists in Athena.
Upon checking the CloudTrail, the DeleteTable glue API has failed to delete the temp table with EntityNotFoundException.
The CreateTable API and DeleteTable API was called nearly at the same time, and since the CreateTable API is async, I assume that the DeleteTable API failed because the temp table was not created yet.
I suggest to add a logic to check whether table was created or not before calling DeleteTable API in to_iceberg function.
How to Reproduce
*P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.*

wr.athena.to_iceberg(
        df=df,
        database="sample_database",
        table="sample_iceberg_table",
        temp_path="s3://sample_bucket/temp_dir",
        keep_files=False
)

Expected behavior
The temp_table_xxxxx has to be deleted properly in Athena
Your project
No response
Screenshots
No response
OS
Window
Python version



AWS SDK for pandas version



Additional context
No response

Duplicate issue #2925: Iceberg commit errors thrown when using overwrite partition somehow fails to clean up temp table
Created: 2024-08-07T16:20:58Z
Describe the bug
When concurrent processes attempt to update the same iceberg table and one hits an ICEBERG_COMMIT_ERROR, sometimes the temp_table created by delete_from_iceberg_table fails to get cleaned up. Reading through the source code, I don't see a clear path for how this would be possible, since the code seems to correctly catch the exception and clean up in finally, but nonetheless the temp tables are getting left behind somehow.
I have not been able to create a reliable minimum code sample to replicate this behavior consistently, but in production we occasionally are hitting commit errors and accumulating temp tables in the glue catalog as a result:
awswrangler.exceptions.QueryFailed: ICEBERG_COMMIT_ERROR: Failed to commit Iceberg update to table
--
-- | -- wr.athena.to_iceberg(**write_args) |  
-- | --
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/_config.py", line 715, in wrapper |  
  |   | 2024-08-07 08:06:24 | return function(**args) |  
  |   | 2024-08-07 08:06:24 | ^^^^^^^^^^^^^^^^ |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/_utils.py", line 178, in inner |  
  |   | 2024-08-07 08:06:24 | return func(*args, **kwargs) |  
  |   | 2024-08-07 08:06:24 | ^^^^^^^^^^^^^^^^^^^^^ |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/athena/_write_iceberg.py", line 452, in to_iceberg |  
  |   | 2024-08-07 08:06:24 | delete_from_iceberg_table( |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/_config.py", line 715, in wrapper |  
  |   | 2024-08-07 08:06:24 | return function(**args) |  
  |   | 2024-08-07 08:06:24 | ^^^^^^^^^^^^^^^^ |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/_utils.py", line 178, in inner |  
  |   | 2024-08-07 08:06:24 | return func(*args, **kwargs) |  
  |   | 2024-08-07 08:06:24 | ^^^^^^^^^^^^^^^^^^^^^ |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/athena/_write_iceberg.py", line 680, in delete_from_iceberg_table |  
  |   | 2024-08-07 08:06:24 | wait_query(query_execution_id=query_execution_id, boto3_session=boto3_session) |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/_config.py", line 715, in wrapper |  
  |   | 2024-08-07 08:06:24 | return function(**args) |  
  |   | 2024-08-07 08:06:24 | ^^^^^^^^^^^^^^^^ |  
  |   | 2024-08-07 08:06:24 | File "/usr/local/lib/python3.11/site-packages/awswrangler/athena/_executions.py", line 237, in wait_query |  
  |   | 2024-08-07 08:06:24 | raise exceptions.QueryFailed(response["Status"].get("StateChangeReason"))

How to Reproduce
Broadly:
Have two processes call to_iceberg on the same catalog table simultaneously, with write params similar to:
{
    "df": to_write.copy(),
    "mode": "overwrite_partitions",
    "partition_cols": self.partition_cols,
    "database": self.glue_database_name,
    "table": self.table_name,
    "table_location": self.table_s3_path(),
    "temp_path": tmp_table_path,
    "boto3_session": self.boto3_client_wrapper.get_session(),
    "schema_evolution": True,
    "fill_missing_columns_in_df": True,
    "keep_files": False, 
}

If a commit error occurs, sometimes a temp table is left behind.
Expected behavior
No response
Your project
No response
Screenshots
No response
OS
Linux
Python version
3.11
AWS SDK for pandas version
3.7.3
Additional context
No response

Closing comment (2024-08-13T14:23:48Z): This seems similar to #2826, looks like it was hard to replicate back then too