Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug

Related Issues

#4116

#4292

Environment & Settings

4.3.0 (rhubarb):

Celery version:

celery report Output:

software -> celery:4.3.0 (rhubarb) kombu:4.5.0 py:3.8.2

billiard:3.6.3.0 py-amqp:2.5.2

platform -> system:Linux arch:64bit, ELF

kernel version:4.15.0-88-generic imp:CPython

loader -> celery.loaders.app.AppLoader

settings -> transport:amqp results:amqp://guest:**@localhost//

ABSOLUTE_URL_OVERRIDES: {

}

ACCOUNT_ADAPTER: 'lancium_box.users.adapters.AccountAdapter'

ACCOUNT_ALLOW_REGISTRATION: True

ACCOUNT_AUTHENTICATION_METHOD: 'username'

ACCOUNT_EMAIL_REQUIRED: True

ACCOUNT_EMAIL_VERIFICATION: 'mandatory'

ACCOUNT_FORMS: {

'login': 'lancium_box.users.forms.CustomLoginForm'}

ADMINS: [('Vitor de Miranda Henrique',

'vitor.henrique@lancium.com')]

ADMIN_URL: 'admin/'

ALLOWED_HOSTS: []

APPEND_SLASH: True

APPS_DIR: Path:/home/skywalker/lancium_box/lancium_box

AUTHENTICATION_BACKENDS: ['lancium_box.users.auth_backends.CustomAuthenticationBackend',

'django.contrib.auth.backends.ModelBackend',

'allauth.account.auth_backends.AuthenticationBackend']

AUTH_PASSWORD_VALIDATORS: ''

AUTH_USER_MODEL: 'users.User'

CACHES: {

'default': { 'BACKEND': 'django_redis.cache.RedisCache',

'LOCATION': 'redis://127.0.0.1:6379/1',

'OPTIONS': { 'CLIENT_CLASS': 'django_redis.client.DefaultClient',

'IGNORE_EXCEPTIONS': True}}}

CACHE_MIDDLEWARE_ALIAS: 'default'

CACHE_MIDDLEWARE_KEY_PREFIX: ''

CACHE_MIDDLEWARE_SECONDS: 600

CELERY_ACCEPT_CONTENT: ['json']

CELERY_BROKER_URL: 'amqp://guest:@localhost:5672//'

CELERY_IMPORTS: ['lancium_box.monitoring.tasks']

CELERY_RESULT_BACKEND: 'amqp://guest:@localhost//'

CELERY_RESULT_SERIALIZER: 'json'

CELERY_TASK_SERIALIZER: 'json'

CELERY_TASK_SOFT_TIME_LIMIT: 60

CELERY_TASK_TIME_LIMIT: 300

CELERY_TIMEZONE: 'UTC'

COMPRESSORS: {

'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}

COMPRESS_CACHEABLE_PRECOMPILERS:

()

COMPRESS_CACHE_BACKEND: 'default'

COMPRESS_CACHE_KEY_FUNCTION: ''

COMPRESS_CLEAN_CSS_ARGUMENTS: ''

COMPRESS_CLEAN_CSS_BINARY: 'cleancss'

COMPRESS_CLOSURE_COMPILER_ARGUMENTS: ''

COMPRESS_CLOSURE_COMPILER_BINARY: 'java -jar compiler.jar'

COMPRESS_CSS_HASHING_METHOD: 'mtime'

COMPRESS_DATA_URI_MAX_SIZE: 1024

COMPRESS_DEBUG_TOGGLE: None

COMPRESS_ENABLED: True

COMPRESS_FILTERS: {

'css': ['compressor.filters.css_default.CssAbsoluteFilter'],

'js': ['compressor.filters.jsmin.JSMinFilter']}

COMPRESS_JINJA2_GET_ENVIRONMENT: <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f3a8e76fee0>

COMPRESS_MINT_DELAY: 30

COMPRESS_MTIME_DELAY: 10

COMPRESS_OFFLINE: True

COMPRESS_OFFLINE_CONTEXT: {

'STATIC_URL': '/static/'}

COMPRESS_OFFLINE_MANIFEST: 'manifest.json'

COMPRESS_OFFLINE_TIMEOUT: 31536000

COMPRESS_OUTPUT_DIR: 'CACHE'

COMPRESS_PARSER: 'compressor.parser.AutoSelectParser'

COMPRESS_PRECOMPILERS:

()

COMPRESS_REBUILD_TIMEOUT: 2592000

COMPRESS_ROOT: '/home/skywalker/lancium_box/staticfiles'

COMPRESS_STORAGE: 'compressor.storage.GzipCompressorFileStorage'

COMPRESS_TEMPLATE_FILTER_CONTEXT: {

'STATIC_URL': '/static/'}

COMPRESS_URL: '/static/'

COMPRESS_URL_PLACEHOLDER: '/compressor_url_placeholder/'

COMPRESS_VERBOSE: False

COMPRESS_YUGLIFY_BINARY: 'yuglify'

COMPRESS_YUGLIFY_CSS_ARGUMENTS: '--terminal'

COMPRESS_YUGLIFY_JS_ARGUMENTS: '--terminal'

COMPRESS_YUI_BINARY: 'java -jar yuicompressor.jar'

COMPRESS_YUI_CSS_ARGUMENTS: ''

COMPRESS_YUI_JS_ARGUMENTS: ''

CRISPY_TEMPLATE_PACK: 'bootstrap4'

CSRF_COOKIE_AGE: 31449600

CSRF_COOKIE_DOMAIN: None

CSRF_COOKIE_HTTPONLY: True

CSRF_COOKIE_NAME: 'csrftoken'

CSRF_COOKIE_PATH: '/'

CSRF_COOKIE_SAMESITE: 'Lax'

CSRF_COOKIE_SECURE: False

CSRF_FAILURE_VIEW: 'django.views.csrf.csrf_failure'

CSRF_HEADER_NAME: 'HTTP_X_CSRFTOKEN'

CSRF_TRUSTED_ORIGINS: []

CSRF_USE_SESSIONS: False

DATABASES: {

'default': { 'ATOMIC_REQUESTS': True,

'AUTOCOMMIT': True,

'CONN_MAX_AGE': None,

'ENGINE': 'django.db.backends.postgresql',

'HOST': '10.2.40.205',

'NAME': 'lanciumdb',

'OPTIONS': {},

'PASSWORD': '',

'PORT': 5432,

'TEST': { 'CHARSET': None,

'COLLATION': None,

'MIRROR': None,

'NAME': None},

'TIME_ZONE': None,

'USER': 'lancium_user'}}

DATABASE_ROUTERS: ''

DATACENTER_ROOM: None

DATA_UPLOAD_MAX_MEMORY_SIZE: 2621440

DATA_UPLOAD_MAX_NUMBER_FIELDS: 1000

DATETIME_FORMAT: 'N j, Y, P'

DATETIME_INPUT_FORMATS: ['%Y-%m-%d %H:%M:%S',

'%Y-%m-%d %H:%M:%S.%f',

'%Y-%m-%d %H:%M',

'%Y-%m-%d',

'%m/%d/%Y %H:%M:%S',

'%m/%d/%Y %H:%M:%S.%f',

'%m/%d/%Y %H:%M',

'%m/%d/%Y',

'%m/%d/%y %H:%M:%S',

'%m/%d/%y %H:%M:%S.%f',

'%m/%d/%y %H:%M',

'%m/%d/%y']

DATE_FORMAT: 'N j, Y'

DATE_INPUT_FORMATS: ['%Y-%m-%d',

'%m/%d/%Y',

'%m/%d/%y',

'%b %d %Y',

'%b %d, %Y',

'%d %b %Y',

'%d %b, %Y',

'%B %d %Y',

'%B %d, %Y',

'%d %B %Y',

'%d %B, %Y']

DEBUG: False

DEBUG_PROPAGATE_EXCEPTIONS: False

DECIMAL_SEPARATOR: '.'

DEFAULT_CHARSET: 'utf-8'

DEFAULT_CONTENT_TYPE: 'text/html'

DEFAULT_EXCEPTION_REPORTER_FILTER: 'django.views.debug.SafeExceptionReporterFilter'

DEFAULT_FILE_STORAGE: 'django.core.files.storage.FileSystemStorage'

DEFAULT_FROM_EMAIL: 'Smart Response noreply@lancium.com'

DEFAULT_INDEX_TABLESPACE: ''

DEFAULT_TABLESPACE: ''

DISABLE_DASHBOARD: False

DISALLOWED_USER_AGENTS: []

DJANGO_APPS: ['django.contrib.auth',

'django.contrib.contenttypes',

'django.contrib.sessions',

'django.contrib.sites',

'django.contrib.messages',

'django.contrib.staticfiles',

'django.contrib.humanize',

'django.contrib.admin']

EMAIL_BACKEND: 'django.core.mail.backends.smtp.EmailBackend'

EMAIL_HOST: 'smtp.gmail.com'

EMAIL_PORT: 587

EMAIL_SSL_CERTFILE: None

EMAIL_SSL_KEYFILE: ''

EMAIL_SUBJECT_PREFIX: '[Smart Response]'

EMAIL_TIMEOUT: 10

EMAIL_USE_LOCALTIME: False

EMAIL_USE_SSL: False

EMAIL_USE_TLS: True

FILE_CHARSET: 'utf-8'

FILE_UPLOAD_DIRECTORY_PERMISSIONS: None

FILE_UPLOAD_HANDLERS: ['django.core.files.uploadhandler.MemoryFileUploadHandler',

'django.core.files.uploadhandler.TemporaryFileUploadHandler']

FILE_UPLOAD_MAX_MEMORY_SIZE: 2621440

FILE_UPLOAD_PERMISSIONS: None

FILE_UPLOAD_TEMP_DIR: None

FIRST_DAY_OF_WEEK: 0

FIXTURE_DIRS:

('/home/skywalker/lancium_box/lancium_box/fixtures',)

FORCE_SCRIPT_NAME: None

FORMAT_MODULE_PATH: None

FORM_RENDERER: 'django.forms.renderers.DjangoTemplates'

IGNORABLE_404_URLS: []

INSTALLED_APPS: ['django.contrib.auth',

'django.contrib.contenttypes',

'django.contrib.sessions',

'django.contrib.sites',

'django.contrib.messages',

'django.contrib.staticfiles',

'django.contrib.humanize',

'django.contrib.admin',

'crispy_forms',

'allauth',

'allauth.account',

'allauth.socialaccount',

'rest_framework',

'rest_framework.authtoken',

'django_celery_beat',

'sekizai',

'lancium_box.users.apps.UsersConfig',

'lancium_box.datacenter.apps.DatacenterConfig',

'lancium_box.monitoring.apps.MonitoringConfig',

'lancium_box.cryptocurrency.apps.CryptocurrencyConfig',

'lancium_box.dashboard.apps.DashboardConfig',

'lancium_box.remotebox.apps.RemoteboxConfig',

'lancium_box.api.apps.ApiConfig',

'lancium_box.event.apps.EventConfig',

'lancium_box.control.apps.ControlConfig',

'lancium_box.demand_response.apps.DemandResponseConfig',

'lancium_box.inventory.apps.InventoryConfig',

'compressor']

INTERNAL_IPS: []

LANGUAGES: [('af', 'Afrikaans'),

('ar', 'Arabic'),

('ast', 'Asturian'),

('az', 'Azerbaijani'),

('bg', 'Bulgarian'),

('be', 'Belarusian'),

('bn', 'Bengali'),

('br', 'Breton'),

('bs', 'Bosnian'),

('ca', 'Catalan'),

('cs', 'Czech'),

('cy', 'Welsh'),

('da', 'Danish'),

('de', 'German'),

('dsb', 'Lower Sorbian'),

('el', 'Greek'),

('en', 'English'),

('en-au', 'Australian English'),

('en-gb', 'British English'),

('eo', 'Esperanto'),

('es', 'Spanish'),

('es-ar', 'Argentinian Spanish'),

('es-co', 'Colombian Spanish'),

('es-mx', 'Mexican Spanish'),

('es-ni', 'Nicaraguan Spanish'),

('es-ve', 'Venezuelan Spanish'),

('et', 'Estonian'),

('eu', 'Basque'),

('fa', 'Persian'),

('fi', 'Finnish'),

('fr', 'French'),

('fy', 'Frisian'),

('ga', 'Irish'),

('gd', 'Scottish Gaelic'),

('gl', 'Galician'),

('he', 'Hebrew'),

('hi', 'Hindi'),

('hr', 'Croatian'),

('hsb', 'Upper Sorbian'),

('hu', 'Hungarian'),

('hy', 'Armenian'),

('ia', 'Interlingua'),

('id', 'Indonesian'),

('io', 'Ido'),

('is', 'Icelandic'),

('it', 'Italian'),

('ja', 'Japanese'),

('ka', 'Georgian'),

('kab', 'Kabyle'),

('kk', 'Kazakh'),

('km', 'Khmer'),

('kn', 'Kannada'),

('ko', 'Korean'),

('lb', 'Luxembourgish'),

('lt', 'Lithuanian'),

('lv', 'Latvian'),

('mk', 'Macedonian'),

('ml', 'Malayalam'),

('mn', 'Mongolian'),

('mr', 'Marathi'),

('my', 'Burmese'),

('nb', 'Norwegian Bokmål'),

('ne', 'Nepali'),

('nl', 'Dutch'),

('nn', 'Norwegian Nynorsk'),

('os', 'Ossetic'),

('pa', 'Punjabi'),

('pl', 'Polish'),

('pt', 'Portuguese'),

('pt-br', 'Brazilian Portuguese'),

('ro', 'Romanian'),

('ru', 'Russian'),

('sk', 'Slovak'),

('sl', 'Slovenian'),

('sq', 'Albanian'),

('sr', 'Serbian'),

('sr-latn', 'Serbian Latin'),

('sv', 'Swedish'),

('sw', 'Swahili'),

('ta', 'Tamil'),

('te', 'Telugu'),

('th', 'Thai'),

('tr', 'Turkish'),

('tt', 'Tatar'),

('udm', 'Udmurt'),

('uk', 'Ukrainian'),

('ur', 'Urdu'),

('vi', 'Vietnamese'),

('zh-hans', 'Simplified Chinese'),

('zh-hant', 'Traditional Chinese')]

LANGUAGES_BIDI: ['he', 'ar', 'fa', 'ur']

LANGUAGE_CODE: 'en-us'

LANGUAGE_COOKIE_AGE: None

LANGUAGE_COOKIE_DOMAIN: None

LANGUAGE_COOKIE_NAME: 'django_language'

LANGUAGE_COOKIE_PATH: '/'

LOCALE_PATHS: [Path:/home/skywalker/lancium_box/locale]

LOCAL_APPS: ['lancium_box.users.apps.UsersConfig',

'lancium_box.datacenter.apps.DatacenterConfig',

'lancium_box.monitoring.apps.MonitoringConfig',

'lancium_box.cryptocurrency.apps.CryptocurrencyConfig',

'lancium_box.dashboard.apps.DashboardConfig',

'lancium_box.remotebox.apps.RemoteboxConfig',

'lancium_box.api.apps.ApiConfig',

'lancium_box.event.apps.EventConfig',

'lancium_box.control.apps.ControlConfig',

'lancium_box.demand_response.apps.DemandResponseConfig',

'lancium_box.inventory.apps.InventoryConfig']

LOGGING: {

'disable_existing_loggers': False,

'filters': { 'require_debug_false': { '()': 'django.utils.log.RequireDebugFalse'}},

'formatters': { 'verbose': { 'format': '%(levelname)s %(asctime)s '

'%(module)s %(process)d '

'%(thread)d %(message)s'}},

'handlers': { 'console': { 'class': 'logging.StreamHandler',

'formatter': 'verbose',

'level': 'DEBUG'},

'mail_admins': { 'class': 'django.utils.log.AdminEmailHandler',

'filters': ['require_debug_false'],

'level': 'ERROR'}},

'loggers': { 'django.request': { 'handlers': ['mail_admins'],

'level': 'ERROR',

'propagate': True},

'django.security.DisallowedHost': { 'handlers': [ 'console',

'mail_admins'],

'level': 'ERROR',

'propagate': True}},

'root': {'handlers': ['console'], 'level': 'INFO'},

'version': 1}

LOGGING_CONFIG: 'logging.config.dictConfig'

LOGIN_REDIRECT_URL: 'users:redirect'

LOGIN_URL: 'account_login'

LOGOUT_REDIRECT_URL: None

MANAGERS: [('Vitor de Miranda Henrique',

'vitor.henrique@lancium.com')]

MEDIA_ROOT: '/home/skywalker/lancium_box/lancium_box/media'

MEDIA_URL: '/media/'

MESSAGE_STORAGE: 'django.contrib.messages.storage.fallback.FallbackStorage'

MIDDLEWARE: ['whitenoise.middleware.WhiteNoiseMiddleware',

'django.contrib.sessions.middleware.SessionMiddleware',

'django.middleware.locale.LocaleMiddleware',

'django.middleware.common.CommonMiddleware',

'django.middleware.csrf.CsrfViewMiddleware',

'django.contrib.auth.middleware.AuthenticationMiddleware',

'django.contrib.messages.middleware.MessageMiddleware',

'django.middleware.clickjacking.XFrameOptionsMiddleware']

MIGRATION_MODULES: {

'sites': 'lancium_box.contrib.sites.migrations'}

MONTH_DAY_FORMAT: 'F j'

NUMBER_GROUPING: 0

PASSWORD_HASHERS: ''

PASSWORD_RESET_TIMEOUT_DAYS: ''

PREPEND_WWW: False

READ_DOT_ENV_FILE: False

REST_FRAMEWORK: {

'DEFAULT_AUTHENTICATION_CLASSES': [ 'rest_framework.authentication.TokenAuthentication']}

ROOT_DIR: Path:/home/skywalker/lancium_box

ROOT_URLCONF: 'config.urls'

SECRET_KEY: '********'

SECURE_BROWSER_XSS_FILTER: True

SECURE_CONTENT_TYPE_NOSNIFF: False

SECURE_HSTS_INCLUDE_SUBDOMAINS: False

SECURE_HSTS_PRELOAD: False

SECURE_HSTS_SECONDS: 0

SECURE_PROXY_SSL_HEADER: None

SECURE_REDIRECT_EXEMPT: []

SECURE_SSL_HOST: None

SECURE_SSL_REDIRECT: False

SERVER_EMAIL: 'Smart Response noreply@lancium.com'

SESSION_CACHE_ALIAS: 'default'

SESSION_COOKIE_AGE: 1209600

SESSION_COOKIE_DOMAIN: None

SESSION_COOKIE_HTTPONLY: True

SESSION_COOKIE_NAME: 'sessionid'

SESSION_COOKIE_PATH: '/'

SESSION_COOKIE_SAMESITE: 'Lax'

SESSION_COOKIE_SECURE: False

SESSION_ENGINE: 'django.contrib.sessions.backends.db'

SESSION_EXPIRE_AT_BROWSER_CLOSE: False

SESSION_FILE_PATH: None

SESSION_SAVE_EVERY_REQUEST: False

SESSION_SERIALIZER: 'django.contrib.sessions.serializers.JSONSerializer'

SETTINGS_MODULE: 'config.settings.production'

SHORT_DATETIME_FORMAT: 'm/d/Y P'

SHORT_DATE_FORMAT: 'm/d/Y'

SIGNING_BACKEND: 'django.core.signing.TimestampSigner'

SILENCED_SYSTEM_CHECKS: []

SITE_ID: 1

SOCIALACCOUNT_ADAPTER: 'lancium_box.users.adapters.SocialAccountAdapter'

STATICFILES_DIRS: ['/home/skywalker/lancium_box/lancium_box/static']

STATICFILES_FINDERS: ['django.contrib.staticfiles.finders.FileSystemFinder',

'django.contrib.staticfiles.finders.AppDirectoriesFinder',

'compressor.finders.CompressorFinder']

STATICFILES_STORAGE: 'whitenoise.storage.CompressedManifestStaticFilesStorage'

STATIC_ROOT: '/home/skywalker/lancium_box/staticfiles'

STATIC_URL: '/static/'

TEMPLATES: [{'BACKEND': 'django.template.backends.django.DjangoTemplates',

'DIRS': ['/home/skywalker/lancium_box/lancium_box/templates'],

'OPTIONS': {'context_processors': ['django.template.context_processors.debug',

'django.template.context_processors.request',

'django.contrib.auth.context_processors.auth',

'django.template.context_processors.i18n',

'django.template.context_processors.media',

'django.template.context_processors.static',

'django.template.context_processors.tz',

'django.contrib.messages.context_processors.messages',

'lancium_box.utils.context_processors.settings_context',

'sekizai.context_processors.sekizai'],

'loaders': [('django.template.loaders.cached.Loader',

['django.template.loaders.filesystem.Loader',

'django.template.loaders.app_directories.Loader'])]}}]

TEST_NON_SERIALIZED_APPS: []

TEST_RUNNER: 'django.test.runner.DiscoverRunner'

THIRD_PARTY_APPS: ['crispy_forms',

'allauth',

'allauth.account',

'allauth.socialaccount',

'rest_framework',

'rest_framework.authtoken',

'django_celery_beat',

'sekizai']

THOUSAND_SEPARATOR: ','

TIME_FORMAT: 'P'

TIME_INPUT_FORMATS: ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']

TIME_ZONE: 'UTC'

USE_I18N: True

USE_L10N: True

USE_THOUSAND_SEPARATOR: False

USE_TZ: True

USE_X_FORWARDED_HOST: False

USE_X_FORWARDED_PORT: False

WSGI_APPLICATION: 'config.wsgi.application'

X_FRAME_OPTIONS: 'DENY'

YEAR_MONTH_FORMAT: 'F Y'

is_overridden: <bound method Settings.is_overridden of <Settings "config.settings.production">>

Steps to Reproduce

Hello everyone,

On my project I have 5 workers on running that periodically check or insert data on database (postgress) every 5 seconds . My problem is that celery is opening too many connections to the database. I can see at least 20 connections for the same tasks hitting the database.

Is there way to improve this with Django? I tried setting up CONN_MAX_AGE = None on django, to have the database persist the same connection but that didn't help.

I also saw a pull request and some comments on reuse the database on previous mentioned issues but they were reverted because they caused regression problems.

Python Packages

pip freeze Output:

amqp==2.5.2

argon2-cffi==19.2.0

bcrypt==3.1.7

beautifulsoup4==4.7.1

billiard==3.6.3.0

blockchain==1.4.4

boto3==1.12.47

botocore==1.15.47

celery==4.3.0

certifi==2020.4.5.1

cffi==1.14.0

chardet==3.0.4

coreapi==2.3.3

coreschema==0.0.4

cryptography==2.9.2

defusedxml==0.6.0

Django==2.2.8

django-allauth==0.40.0

django-anymail==7.0.0

django-appconf==1.0.4

django-celery-beat==1.5.0

django-classy-tags==0.8.0

django-compressor==2.3

django-crispy-forms==1.8.1

django-environ==0.4.5

django-model-utils==3.2.0

django-pandas==0.6.0

django-redis==4.10.0

django-sekizai==1.0.0

django-storages==1.9.1

django-timezone-field==4.0

djangorestframework==3.10.3

docutils==0.15.2

enum-compat==0.0.3

et-xmlfile==1.0.1

future==0.18.2

gevent==20.4.0

greenlet==0.4.15

gunicorn==20.0.4

idna==2.8

itypes==1.2.0

jdcal==1.4.1

Jinja2==2.11.2

jmespath==0.9.5

kombu==4.5.0

MarkupSafe==1.1.1

numpy==1.16.2

oauthlib==3.1.0

openpyxl==2.6.3

pandas==0.24.2

parallel-ssh==1.9.1

paramiko==2.7.1

Pillow==6.2.1

psycopg2-binary==2.8.4

pycparser==2.20

pyModbusTCP==0.1.8

PyNaCl==1.3.0

pysftp==0.2.9

python-crontab==2.4.2

python-dateutil==2.8.0

python-slugify==4.0.0

python3-openid==3.1.0

pytz==2019.3

rcssmin==1.0.6

redis==3.3.11

requests==2.21.0

requests-oauthlib==1.3.0

rjsmin==1.1.0

s3transfer==0.3.3

six==1.14.0

soupsieve==2.0

sqlparse==0.3.1

ssh2-python==0.18.0.post1

text-unidecode==1.3

uritemplate==3.0.1

urllib3==1.24.3

vine==1.3.0

whitenoise==4.1.4

Issue 2:
Checklist

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

#5597

Environment & Settings

Celery version: 4.3.0 (rhubarb)

celery report Output:

software -> celery:4.3.0 (rhubarb) kombu:4.4.0 py:3.6.7
billiard:3.6.0.0 py-amqp:2.4.1
platform -> system:Linux arch:64bit
kernel version:4.18.0-25-generic imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

task_queues: [<unbound Queue test -> <unbound Exchange test(direct)> -> test>]

Steps to Reproduce

Required Dependencies

Minimal Python Version: 3.6.7

Minimal Celery Version: 4.3.0

Minimal Kombu Version: 4.4.0

Minimally Reproducible Test Case

import celery
from kombu import Exchange
from kombu import Queue

class Config:
task_queues = [
Queue('test', Exchange('test'), routing_key='test', queue_arguments={'x-max-priority': 3})
] # yapf: disable

app = celery.Celery('test')
app.config_from_object(Config)

@app.task(bind=True)
def task(self):
print(self.request.delivery_info['priority'])
self.retry(countdown=1)

if __name__ == '__main__':
task.s().apply_async(priority=1, queue='test')

Expected Behavior

Expect task to have a priority 1 after self.retry()

Expected output:

[tasks]
. test.task

[2019-07-24 22:31:50,810: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-07-24 22:31:50,823: INFO/MainProcess] mingle: searching for neighbors
[2019-07-24 22:31:51,855: INFO/MainProcess] mingle: all alone
[2019-07-24 22:31:51,869: INFO/MainProcess] celery@ArtSobes-Home-Ubuntu-18 ready.
[2019-07-24 22:31:51,870: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2]
[2019-07-24 22:31:51,973: WARNING/ForkPoolWorker-16] 1
[2019-07-24 22:31:51,995: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] ETA:[2019-07-24 19:31:52.974357+00:00]
[2019-07-24 22:31:51,995: INFO/ForkPoolWorker-16] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:54,828: WARNING/ForkPoolWorker-2] 1
[2019-07-24 22:31:54,850: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] ETA:[2019-07-24 19:31:55.829945+00:00]
[2019-07-24 22:31:54,850: INFO/ForkPoolWorker-2] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:56,831: WARNING/ForkPoolWorker-4] 1
[2019-07-24 22:31:56,853: INFO/MainProcess] Received task: test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] ETA:[2019-07-24 19:31:57.832523+00:00]
[2019-07-24 22:31:56,853: INFO/ForkPoolWorker-4] Task test.task[df27b1cc-6a9e-4fdb-aa46-dd02d15e4df2] retry: Retry in 1s
[2019-07-24 22:31:58,833: WARNING/ForkPoolWorker-6] 1

Actual Behavior

On the first call task priority is 1, but after retry it is None

[tasks]
. test.task

[2019-07-24 22:30:51,901: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672//
[2019-07-24 22:30:51,913: INFO/MainProcess] mingle: searching for neighbors
[2019-07-24 22:30:52,940: INFO/MainProcess] mingle: all alone
[2019-07-24 22:30:52,956: INFO/MainProcess] celery@ArtSobes-Home-Ubuntu-18 ready.
[2019-07-24 22:30:52,957: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e]
[2019-07-24 22:30:53,060: WARNING/ForkPoolWorker-16] 1
[2019-07-24 22:30:53,083: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] ETA:[2019-07-24 19:30:54.062046+00:00]
[2019-07-24 22:30:53,083: INFO/ForkPoolWorker-16] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:54,958: WARNING/ForkPoolWorker-2] None
[2019-07-24 22:30:54,979: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] ETA:[2019-07-24 19:30:55.959481+00:00]
[2019-07-24 22:30:54,979: INFO/ForkPoolWorker-2] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:56,960: WARNING/ForkPoolWorker-4] None
[2019-07-24 22:30:56,982: INFO/MainProcess] Received task: test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] ETA:[2019-07-24 19:30:57.962134+00:00]
[2019-07-24 22:30:56,983: INFO/ForkPoolWorker-4] Task test.task[bc93cbd9-9a20-41cb-aaef-55c71245038e] retry: Retry in 1s
[2019-07-24 22:30:58,963: WARNING/ForkPoolWorker-6] None