Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version:

celery report Output:

software -> celery:4.4.3 (cliffs) kombu:4.6.9 py:3.7.6
billiard:3.6.3.0 py-amqp:2.6.0
platform -> system:Linux arch:64bit, ELF
kernel version:4.15.0-101-generic imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:cassandra

Queue: <class 'kombu.entity.Queue'>
RoundRobinPolicy: <class 'cassandra.policies.RoundRobinPolicy'>
broker_url: 'amqp://guest:********@localhost:5672//'
cassandra_auth_kwargs: {
'password': '********', 'username': 'cassandra'}
cassandra_auth_provider: 'PlainTextAuthProvider'
cassandra_entry_ttl: 86400
cassandra_keyspace: '********'
cassandra_options: {
'load_balancing_policy': <cassandra.policies.RoundRobinPolicy object at 0x7f09de9a34d0>}
cassandra_servers: ['127.0.0.1']
cassandra_table: 'celery_jobs_result'
ignore_result: False
result_backend: 'cassandra'
worker_concurrency: 4

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

amqp==2.6.0
attrs==19.3.0
billiard==3.6.3.0
case==1.5.3
cassandra-driver==3.20.2
future==0.18.2
gevent==20.5.2
greenlet==0.4.15
importlib-metadata==1.6.0
kombu==4.6.9
linecache2==1.0.0
mock==4.0.2
more-itertools==8.3.0
nose==1.3.7
packaging==20.4
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.7
pytest==5.4.2
pytz==2020.1
redis==3.5.2
six==1.15.0
traceback2==1.4.0
unittest2==1.1.0
vine==1.3.0
wcwidth==0.1.9
zipp==3.1.0
zope.event==4.4
zope.interface==5.1.0

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

CassandraBackend work fine

Actual Behavior

in prefork pool or solo:

it can work, just not good, i think

in threads or gevent pool:

get these error frequently

[2020-06-02 21:52:19,592: ERROR/MainProcess] Task add[4bd528a0-c1ba-456c-b74e-7dee054616bb] raised unexpected: AttributeError("'NoneType' object has no attribute 'execute'")
Traceback (most recent call last):
File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
uuid, retval, task_request, publish_result,
File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
self.store_result(task_id, result, state, request=request)
File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
request=request, **kwargs)
File "/home/bx/project/celery/celery/backends/cassandra.py", line 201, in _store_result
self._session.execute(self._write_stmt, (
AttributeError: 'NoneType' object has no attribute 'execute'

and

[2020-06-02 21:52:29,773: ERROR/MainProcess] Task add[27c980d3-126a-430d-a8e0-92d749ceb735] raised unexpected: OperationTimedOut("errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None")
Traceback (most recent call last):
File "/home/bx/project/celery/celery/app/trace.py", line 480, in trace_task
uuid, retval, task_request, publish_result,
File "/home/bx/project/celery/celery/backends/base.py", line 158, in mark_as_done
self.store_result(task_id, result, state, request=request)
File "/home/bx/project/celery/celery/backends/base.py", line 443, in store_result
request=request, **kwargs)
File "/home/bx/project/celery/celery/backends/cassandra.py", line 207, in _store_result
buf_t(self.encode(self.current_task_children(request)))
File "cassandra/cluster.py", line 2345, in cassandra.cluster.Session.execute
File "cassandra/cluster.py", line 4304, in cassandra.cluster.ResponseFuture.result
cassandra.OperationTimedOut: errors=errors={'Connection defunct by heartbeat': 'Client request timeout. See Session.execute[_async](timeout)'}, last_host=127.0.0.1:9042, last_host=None

Fix it

remove method(CassandraBackend.process_cleanup), it works fine.

because.

all tasks should share one instance of cassandra.cluster.Cluster and session, not create one ervery time. that is the correct way to use cassandra-driver

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

DefectDojo/django-DefectDojo#3582

Possible Duplicates

Environment & Settings

Steps to Reproduce

I have come across this when running using OpenShift restricted SCC

oc logs defect-dojo-defectdojo-celery-worker-78dbb86b48-2rxn2
uid=1000640000(1000640000) gid=0(root) groups=0(root),1000640000
Waiting for database to be reachable [06/Jan/2021 13:40:58] INFO [dojo.models:3474] enabling audit logging
[06/Jan/2021 13:41:00] INFO [dojo.models:3474] enabling audit logging
Running a worker with superuser privileges when the
worker accepts messages serialized with pickle is a very bad idea!
If you really want to continue then you have to set the C_FORCE_ROOT
environment variable (but please think about this before you do).
User information: uid=1000640000 euid=1000640000 gid=0 egid=0

Required Dependencies

Minimal Python Version: N/A

Minimal Celery Version: 4.1.1

Minimal Kombu Version: 4.1

Minimal Broker Version: N/A

Minimal Result Backend Version: N/A

Minimal OS and/or Kernel Version: N/A

Minimal Broker Client Version: N/A

Minimal Result Backend Client Version: N/A

Python Packages

Other Dependencies

Minimally Reproducible Test Case

Expected Behavior

I believe that GID=0 doesn't give superuser privileges, so am not sure why the celery checks are implemented in this manner.

Actual Behavior

The gid (and egid) being 0 does not give superuser prvilieges

as stated - https://unix.stackexchange.com/questions/44077/what-does-it-mean-to-be-in-group-0 and https://docs.celeryproject.org/en/stable/userguide/daemonizing.html#running-the-worker-with-superuser-privileges-root