Issue 1:
我在使用cerely4.4.7+redis3.5.3的时候，单独一个线程进行测试使用.get()能正常取到返回值，

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery. (I cannot test this because it's a production server)

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery. (I cannot test this because it's a production server)

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

#4480 (But that issue is about the AMQP broker, and I'm using Redis)

Environment & Settings

Celery version: 4.4.7 (cliffs)

celery report Output:

software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.6.9
billiard:3.6.3.0 redis:3.5.3
platform -> system:Linux arch:64bit, ELF
kernel version:4.15.0-115-generic imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:redis results:redis+socket:///var/run/redis/redis-server.sock

ABSOLUTE_URL_OVERRIDES: {
}
ADMINS: []
ALLOWED_HOSTS: ['proj.com']
APPEND_SLASH: True
AUTHENTICATION_BACKENDS: ['django.contrib.auth.backends.ModelBackend']
AUTH_PASSWORD_VALIDATORS: '********'
AUTH_USER_MODEL: 'accounts.CustomUser'
BASE_DIR: '/home/proj/website'
CACHES: {
'default': { 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': 'unix:/var/run/memcached/memcached.sock'}}
CACHE_MIDDLEWARE_ALIAS: 'default'
CACHE_MIDDLEWARE_KEY_PREFIX: '********'
CACHE_MIDDLEWARE_SECONDS: 600
CELERY_BROKER_URL: 'redis+socket:///var/run/redis/redis-server.sock'
CELERY_RESULT_BACKEND: 'redis+socket:///var/run/redis/redis-server.sock'
CSRF_COOKIE_AGE: 31449600
CSRF_COOKIE_DOMAIN: None
CSRF_COOKIE_HTTPONLY: False
CSRF_COOKIE_NAME: 'csrftoken'
CSRF_COOKIE_PATH: '/'
CSRF_COOKIE_SAMESITE: 'Lax'
CSRF_COOKIE_SECURE: False
CSRF_FAILURE_VIEW: 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME: 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS: []
CSRF_USE_SESSIONS: False
DATABASES: {
'default': { 'ATOMIC_REQUESTS': False,
'AUTOCOMMIT': True,
'CONN_MAX_AGE': 0,
'ENGINE': 'django.db.backends.postgresql',
'HOST': 'localhost',
'NAME': 'proj',
'OPTIONS': {},
'PASSWORD': '********',
'PORT': '',
'TEST': { 'CHARSET': None,
'COLLATION': None,
'MIRROR': None,
'NAME': None},
'TIME_ZONE': None,
'USER': 'postgres'}}
DATABASE_ROUTERS: '********'
DATA_UPLOAD_MAX_MEMORY_SIZE: 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS: 1000
DATETIME_FORMAT: 'N j, Y, P'
DATETIME_INPUT_FORMATS: ['%Y-%m-%d %H:%M:%S',
'%Y-%m-%d %H:%M:%S.%f',
'%Y-%m-%d %H:%M',
'%Y-%m-%d',
'%m/%d/%Y %H:%M:%S',
'%m/%d/%Y %H:%M:%S.%f',
'%m/%d/%Y %H:%M',
'%m/%d/%Y',
'%m/%d/%y %H:%M:%S',
'%m/%d/%y %H:%M:%S.%f',
'%m/%d/%y %H:%M',
'%m/%d/%y']
DATE_FORMAT: 'N j, Y'
DATE_INPUT_FORMATS: ['%Y-%m-%d',
'%m/%d/%Y',
'%m/%d/%y',
'%b %d %Y',
'%b %d, %Y',
'%d %b %Y',
'%d %b, %Y',
'%B %d %Y',
'%B %d, %Y',
'%d %B %Y',
'%d %B, %Y']
DEBUG: False
DEBUG_PROPAGATE_EXCEPTIONS: False
DECIMAL_SEPARATOR: '.'
DEFAULT_CHARSET: 'utf-8'
DEFAULT_EXCEPTION_REPORTER_FILTER: 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE: 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL: 'PROJ <noreply@proj.com>'
DEFAULT_INDEX_TABLESPACE: ''
DEFAULT_TABLESPACE: ''
DISALLOWED_USER_AGENTS: []
EMAIL_BACKEND: 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST: 'localhost'
EMAIL_HOST_PASSWORD: '********'
EMAIL_HOST_USER: ''
EMAIL_PORT: 25
EMAIL_SSL_CERTFILE: None
EMAIL_SSL_KEYFILE: '********'
EMAIL_SUBJECT_PREFIX: '[Django] '
EMAIL_TIMEOUT: None
EMAIL_USE_LOCALTIME: False
EMAIL_USE_SSL: False
EMAIL_USE_TLS: False
FILE_CHARSET: 'utf-8'
FILE_UPLOAD_DIRECTORY_PERMISSIONS: None
FILE_UPLOAD_HANDLERS: ['django.core.files.uploadhandler.MemoryFileUploadHandler',
'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE: 2621440
FILE_UPLOAD_PERMISSIONS: 420
FILE_UPLOAD_TEMP_DIR: None
FIRST_DAY_OF_WEEK: 0
FIXTURE_DIRS: []
FORCE_SCRIPT_NAME: None
FORMAT_MODULE_PATH: None
FORM_RENDERER: 'django.forms.renderers.TemplatesSetting'
IGNORABLE_404_URLS: []
INSTALLED_APPS: ['proj.apps.SuitConfig',
'fullurl',
'django_telegrambot',
'django_jalali',
'django_extensions',
'debug_toolbar',
'django_user_agents',
'htmlemailer',
'qsessions',
'django.forms',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.humanize',
# .... my proj apps here
]
INTERNAL_IPS: ['127.0.0.1']
LANGUAGES:
(('fa', 'Farsi'), ('en', 'انگلیسی'))
LANGUAGES_BIDI: ['he', 'ar', 'fa', 'ur']
LANGUAGE_CODE: 'fa'
LANGUAGE_COOKIE_AGE: None
LANGUAGE_COOKIE_DOMAIN: None
LANGUAGE_COOKIE_HTTPONLY: False
LANGUAGE_COOKIE_NAME: 'django_language'
LANGUAGE_COOKIE_PATH: '/'
LANGUAGE_COOKIE_SAMESITE: None
LANGUAGE_COOKIE_SECURE: False
LOCALE_PATHS: []
LOGGING: {
'disable_existing_loggers': False,
'formatters': { 'standard': { 'format': '%(asctime)s [%(levelname)s] '
'%(name)s: %(message)s'}},
'handlers': { 'file': { 'backupCount': 10,
'class': 'logging.handlers.RotatingFileHandler',
'filename': '/var/log/proj/website.log',
'formatter': 'standard',
'level': 'INFO',
'maxBytes': 10485760}},
'loggers': { 'django': { 'handlers': ['file'],
'level': 'INFO',
'propagate': True}},
'version': 1}
LOGGING_CONFIG: 'logging.config.dictConfig'
LOGIN_REDIRECT_URL: 'dashboard'
LOGIN_URL: 'login'
LOGOUT_REDIRECT_URL: 'index'
MANAGERS: []
MAX_STRING_LENGTH: 65536
MEDIA_ROOT: '/home/proj/website/media'
MEDIA_URL: '/media/'
MESSAGE_STORAGE: 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS: {
25: 'success', 40: 'danger'}
MIDDLEWARE: ['htmlmin.middleware.MarkRequestMiddleware',
'htmlmin.middleware.HtmlMinifyMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware',
'django.middleware.security.SecurityMiddleware',
'qsessions.middleware.SessionMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django_user_agents.middleware.UserAgentMiddleware']
MIGRATION_MODULES: {
}
MONTH_DAY_FORMAT: 'F j'
NUMBER_GROUPING: 0
PASSWORD_HASHERS: '********'
PASSWORD_RESET_TIMEOUT_DAYS: '********'
PREPEND_WWW: False
ROOT_URLCONF: 'proj.urls'
SECRET_KEY: '********'
SECURE_BROWSER_XSS_FILTER: False
SECURE_CONTENT_TYPE_NOSNIFF: True
SECURE_HSTS_INCLUDE_SUBDOMAINS: False
SECURE_HSTS_PRELOAD: False
SECURE_HSTS_SECONDS: 0
SECURE_PROXY_SSL_HEADER: None
SECURE_REDIRECT_EXEMPT: []
SECURE_REFERRER_POLICY: None
SECURE_SSL_HOST: None
SECURE_SSL_REDIRECT: False
SENTRY_API_KEY: '********'
SENTRY_TRACES_SAMPLE_RATE: 0.0001
SERVER_EMAIL: 'root@localhost'
SESSION_CACHE_ALIAS: 'default'
SESSION_COOKIE_AGE: 1209600
SESSION_COOKIE_DOMAIN: None
SESSION_COOKIE_HTTPONLY: True
SESSION_COOKIE_NAME: 'sessionid'
SESSION_COOKIE_PATH: '/'
SESSION_COOKIE_SAMESITE: 'Lax'
SESSION_COOKIE_SECURE: False
SESSION_ENGINE: 'qsessions.backends.cached_db'
SESSION_EXPIRE_AT_BROWSER_CLOSE: False
SESSION_FILE_PATH: None
SESSION_SAVE_EVERY_REQUEST: False
SESSION_SERIALIZER: 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE: 'proj.settings'
SHORT_DATETIME_FORMAT: 'm/d/Y P'
SHORT_DATE_FORMAT: 'm/d/Y'
SIGNING_BACKEND: 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS: []
STATICFILES_DIRS: ['/home/proj/website/proj/static']
STATICFILES_FINDERS: ['django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATICFILES_STORAGE: 'proj.storages.ForgivingManifestStaticFilesStorage'
STATIC_ROOT: '/home/proj/website/static'
STATIC_URL: '/static/'
TEMPLATES: [{'APP_DIRS': True,
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': ['/home/proj/website/templates'],
'OPTIONS': {'context_processors': ['django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages']}}]
TEST_NON_SERIALIZED_APPS: []
TEST_RUNNER: 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR: ','
TIME_FORMAT: 'P'
TIME_INPUT_FORMATS: ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE: 'Asia/Tehran'
USE_I18N: True
USE_L10N: True
USE_THOUSAND_SEPARATOR: False
USE_TZ: True
USE_X_FORWARDED_HOST: False
USE_X_FORWARDED_PORT: False
WSGI_APPLICATION: 'proj.wsgi.application'
X_FRAME_OPTIONS: 'DENY'
YEAR_MONTH_FORMAT: 'F Y'
is_overridden: <bound method Settings.is_overridden of <Settings "proj.settings">>

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.6

Minimal Celery Version: 4.4

Minimal Kombu Version: 4.6.11

Minimal Broker Version: 4.0.9

Minimal Result Backend Version: 4.0.9

Minimal OS and/or Kernel Version: Ubuntu 18.04

Minimal Broker Client Version: 3.5.3

Minimal Result Backend Client Version: 3.5.3

Python Packages

pip freeze Output:

aiohttp==3.6.2
amqp==2.6.1 (I don't think that I'm using this)
asgiref==3.2.5
async-timeout==3.0.1
attrs==19.3.0
beautifulsoup4==4.8.2
billiard==3.6.3.0
celery==4.4.7
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
commonmark==0.9.1
commonmarkextensions==0.0.5
cryptography==2.8
cssutils==1.0.2
decorator==4.4.2
Django==3.0.8
django-debug-toolbar==2.2
django-extensions==2.2.8
django-forgiving-collectstatic==1.0.0
django-fullurl==1.1
django-html-emailer==0.0.8.1
django-htmlmin==0.11.0
django-ipware==3.0.1
django-jalali==4.0.0
django-qsessions==1.0.0
django-suit @ https://github.com/darklow/django-suit/tarball/v2
-e git://github.com/telebotter/django-telegrambot.git@76742ec4cff775bf4e31df481face7662f5f412f#egg=django_telegrambot
django-user-agents==0.4.0
future==0.18.2
geoip2==4.0.2
html5lib==1.0.1
htpasswd==2.3
idna==2.10
idna-ssl==1.1.0
importlib-metadata==1.7.0
jdatetime==3.6.2
kombu==4.6.11
maxminddb==2.0.2
multidict==4.7.6
orderedmultidict==1.0.1
pkg-resources==0.0.0
psycopg2-binary==2.8.5
pycparser==2.20
pynliner==0.8.0
python-memcached==1.59
python-telegram-bot==12.4.2
pytz==2019.3
redis==3.5.3
requests==2.24.0
sentry-sdk==0.17.3
six==1.14.0
soupsieve==2.0
sqlparse==0.3.1
tornado==6.0.4
typing-extensions==3.7.4.2
ua-parser==0.10.0
urllib3==1.25.10
user-agents==2.1
vine==1.3.0
webencodings==0.5.1
yarl==1.5.1
zipp==3.1.0

Other Dependencies

I'm using celery with Django, placed behind an Apache2 server, using the WSGI mod.
The entries for WSGI are:

WSGIDaemonProcess proj python-home=/home/proj/website/.venv user=proj group=proj socket-user=proj processes=3 threads=15
WSGIProcessGroup proj
WSGIScriptAlias / /home/proj/website/proj/wsgi.py

Minimally Reproducible Test Case

Just use celery with Redis and Django behind apache2, in multi-threaded mode.

Expected Behavior

I expect that celery works every time I run a task.

Actual Behavior

Most of the time starting a task works fine, but sometimes I get this error (or something similar with different values inside ProtocolError):

InvalidResponse: Protocol Error: b'1'
File "django/core/handlers/exception.py", line 34, in inner
response = get_response(request)

Full stacktrace here