Issue 1:
Checklist

I have checked the issues list

for similar or identical enhancement to an existing feature.

I have checked the pull requests list

for existing proposed enhancements.

I have checked the commit log

to find out if the if the same enhancement was already implemented in the

master branch.

I have included all related issues and possible duplicate issues in this issue

(If there are none, check this box anyway).

Related Issues and Possible Duplicates

Related Issues

#6622

Possible Duplicates

None

Brief Summary

https://github.com/celery/celery/blob/v5.2.7/celery/backends/dynamodb.py#L204

This always calls CreatTable even the table existed already.

Each call will have an AWS CloudTrail management event for CreateTable API call with ResourceInUseException error.

Each CouldTrail management event cost 0.00002 USD in AWS US West (Oregon) region.

I have 10M+ tasks everyday and they produce 10M+ such events, and AWS charges me for 200+ USD.

Design

Architectural Considerations

None

Proposed Behavior

Just like what proposed in #6622, check existence of table with another boto3 call first instead of just calling create table

Proposed UI/UX

Diagrams

N/A

Alternatives

None

Issue 2:
For some reason after running for about a day our worker loses it's connection to rabbitmq and is unable to reconnect.

I initially encountered this issue with 4.2, so I upgraded to the latest version (4.3.0) and am still encountering the issue. I've also found several issues related to this, but since they are all closed I am opening this new one (and referencing them below).

The application I am encountering this with is open source, so you can view it in the Nebula repository.

Checklist

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

#4457

#5499

Possible Duplicates

#4457

Environment & Settings

Celery version: 4.3.0

celery report Output:

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.7

Minimal Celery Version: 4.2.0

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: Docker

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

https://raw.githubusercontent.com/tedivm/nebula/master/requirements.txt

Other Dependencies

N/A

Minimally Reproducible Test Case

I can't reproduce on demand unfortunately.

Expected Behavior

The worker should stay connected to the broker, or it should reconnect successfully.

Actual Behavior

consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 596, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python3.6/site-packages/celery/worker/loops.py", line 50, in asynloop
obj.controller.register_with_event_loop(hub)
File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 219, in register_with_event_loop
description='hub.register',
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 151, in send_all
fun(parent, *args)
File "/usr/local/lib/python3.6/site-packages/celery/worker/components.py", line 179, in register_with_event_loop
w.pool.register_with_event_loop(hub)
File "/usr/local/lib/python3.6/site-packages/celery/concurrency/prefork.py", line 134, in register_with_event_loop
return reg(loop)
File "/usr/local/lib/python3.6/site-packages/celery/concurrency/asynpool.py", line 482, in register_with_event_loop
for fd in self._fileno_to_outq]
File "/usr/local/lib/python3.6/site-packages/celery/concurrency/asynpool.py", line 482, in <listcomp>
for fd in self._fileno_to_outq]
File "/usr/local/lib/python3.6/site-packages/kombu/asynchronous/hub.py", line 216, in add_reader
return self.add(fds, callback, READ | ERR, args)
File "/usr/local/lib/python3.6/site-packages/kombu/asynchronous/hub.py", line 167, in add
self.poller.register(fd, flags)
File "/usr/local/lib/python3.6/site-packages/kombu/utils/eventio.py", line 67, in register
self._epoll.register(fd, events)
OSError: [Errno 9] Bad file descriptor