Repository: aws/aws-sdk-pandas

Issue #998: Key in dictionary does not always exist
Created: 2021-11-10T17:15:02Z
Describe the bug
In the following line of the code:
https://github.com/awslabs/aws-data-wrangler/blob/main/awswrangler/catalog/_create.py#L53 (And/Or Line 96)
If the Glue table actually exists and does not have any tags, it will not return table_inputs["Parameters"] and there will be a Key Error at the above lines.
Environment
Python:
 File "test.py", line 345, in main
    upload_data_to_s3(entries, bucket, database, "ticket_detailed_entries", 'modified_date', log)
  File "test.py", line 201, in upload_data_to_s3
    wr.s3.to_parquet(
  File "/apollo/env/HWEngDSADataIngester/lib/python3.8/site-packages/awswrangler/_config.py", line 417, in wrapper
    return function(**args)
  File "/apollo/env/HWEngDSADataIngester/lib/python3.8/site-packages/awswrangler/s3/_write_parquet.py", line 582, in to_parquet
    catalog._create_parquet_table(  # pylint: disable=protected-access
  File "/apollo/env/HWEngDSADataIngester/lib/python3.8/site-packages/awswrangler/catalog/_create.py", line 262, in _create_parquet_table
    _create_table(
  File "/apollo/env/HWEngDSADataIngester/lib/python3.8/site-packages/awswrangler/catalog/_create.py", line 96, in _create_table
    table_input["Parameters"]["projection.enabled"] = "false"

To Reproduce
Create a simple glue table with no tags, call the following function:
wr.s3.to_parquet(
    df=dataframe,
    path=f"s3://{output_bucket}/{output_s3_prefix}",
    index=False,
    dataset=True,
    mode='overwrite_partitions',
    partition_cols=[partition],
    concurrent_partitioning=True,
    database=database,  # Athena/Glue database
    table=table,  # Athena/Glue table
    dtype=all_types
)

Duplicate issue #1059: Glue writer expects Parameters key to present in the Glue table
Created: 2021-12-08T19:00:35Z
Describe the bug
When trying to perform an s3 write to parquet it fails with
KeyError: 'Parameters'
Diving into the code I could smell something there:
https://github.com/awslabs/aws-data-wrangler/blob/f82b7e12d4126ec63f739f6f172139ed2e7d73ac/awswrangler/catalog/_get.py#L20
and
https://github.com/awslabs/aws-data-wrangler/blob/f82b7e12d4126ec63f739f6f172139ed2e7d73ac/awswrangler/catalog/_create.py#L96
Environment
alabaster==0.7.12
appnope==0.1.2
argon2-cffi==21.1.0
asn1crypto==1.4.0
attrs==21.2.0
aws-lambda-powertools==1.22.0
aws-xray-sdk==2.9.0
awswrangler==2.12.0
Babel==2.9.1
backcall==0.2.0
backports.entry-points-selectable==1.1.1
beautifulsoup4==4.10.0
bleach==4.1.0
boto3==1.20.21
botocore==1.23.21
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.9
debugpy==1.5.1
decorator==5.1.0
defusedxml==0.7.1
distlib==0.3.4
docutils==0.17.1
entrypoints==0.3
et-xmlfile==1.1.0
fastjsonschema==2.15.1
filelock==3.4.0
future==0.18.2
idna==3.3
imagesize==1.3.0
ipykernel==6.6.0
ipython==7.30.1
ipython-genutils==0.2.0
ipywidgets==7.6.5
jedi==0.18.1
Jinja2==3.0.3
jmespath==0.10.0
jsonpath-ng==1.5.3
jsonschema==4.2.1
jupyter==1.0.0
jupyter-client==7.1.0
jupyter-console==6.4.0
jupyter-core==4.9.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.2
lxml==4.6.4
MarkupSafe==2.0.1
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.5.9
nbconvert==6.3.0
nbformat==5.1.3
nest-asyncio==1.5.4
notebook==6.4.6
numpy==1.21.4
openpyxl==3.0.9
opensearch-py==1.0.0
packaging==21.3
pandas==1.3.4
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pg8000==1.21.3
pickleshare==0.7.5
platformdirs==2.4.0
pluggy==1.0.0
ply==3.11
progressbar2==3.55.0
prometheus-client==0.12.0
prompt-toolkit==3.0.23
ptyprocess==0.7.0
py==1.11.0
pyarrow==5.0.0
pycparser==2.21
Pygments==2.10.0
PyMySQL==1.0.2
pyparsing==3.0.6
pyrsistent==0.18.0
python-dateutil==2.8.2
python-utils==2.5.6
pytz==2021.3
pyzmq==22.3.0
qtconsole==5.2.1
QtPy==1.11.3
redshift-connector==2.0.901
requests==2.26.0
requests-aws4auth==1.1.1
s3transfer==0.5.0
scramp==1.4.1
Send2Trash==1.8.0
six==1.16.0
snowballstemmer==2.2.0
soupsieve==2.3.1
Sphinx==4.3.1
sphinx-bootstrap-theme==0.8.0
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
terminado==0.12.1
testpath==0.5.0
toml==0.10.2
tornado==6.1
tox==3.24.4
traitlets==5.1.1
urllib3==1.26.7
virtualenv==20.10.0
wcwidth==0.2.5
webencodings==0.5.1
widgetsnbextension==3.5.2
wrapt==1.13.3

To Reproduce
Create a Parquet Table in Glue without Paramaters and try to write to this table with wr.s3.to_parquet and specify database and table properties
P.S. Please do not attach files as it's considered a security risk. Add code snippets directly in the message body as much as possible.

Closing comment (2021-12-08T20:08:22Z): duplicate of #998