Issue 1:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version: celery:5.1.0b1 (singularity)

celery report Output:

software -> celery:5.1.0b1 (singularity) kombu:5.1.0b1 py:3.9.2
billiard:3.6.4.0 py-amqp:5.0.6
platform -> system:Darwin arch:64bit
kernel version:20.3.0 imp:CPython
loader -> celery.loaders.default.Loader
settings -> transport:amqp results:disabled

backend: 'amqp://guest:********@test//'
broker: 'redis://test/'
deprecated_settings: None

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

Other Dependencies

N/A

Minimally Reproducible Test Case

Expected Behavior

config_from_object work correctly.

Actual Behavior

I want to use config_from_object to load my configurations from a python module, but it seems not work, and get the output:

I define broker and backend in celeryconfig.py:

backend = 'amqp://guest:guest@test//'. # test is hostname, == 127.0.0.1

broker = 'redis://test/'

It works fine if I give the broker and backend directly while define celery instance.

Issue 2:
Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Optional Debugging Information

I have tried reproducing the issue on more than one Python version

and/or implementation.

I have tried reproducing the issue on more than one message broker and/or

result backend.

I have tried reproducing the issue on more than one version of the message

broker and/or result backend.

I have tried reproducing the issue on more than one operating system.

I have tried reproducing the issue on more than one workers pool.

I have tried reproducing the issue with autoscaling, retries,

ETA/Countdown & rate limits disabled.

I have tried reproducing the issue after downgrading

and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

None

Possible Duplicates

None

Environment & Settings

Celery version: 5.0.4 (singularity)

celery report Output:

software -> celery:5.0.4 (singularity) kombu:5.0.2 py:3.9.0
billiard:3.6.3.0 py-amqp:5.0.2
platform -> system:Linux arch:64bit, ELF
kernel version:5.9.12-arch1-1 imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:redis://:**@****:6379/10

broker_url: 'amqp://**:********@*****:5672/**'
result_backend: 'redis://:********@*****:6379/10'
deprecated_settings: None

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

amqp==5.0.2
billiard==3.6.3.0
celery @ git+https://github.com/celery/celery.git@420e3931a63538bd225ef57916deccf53cbcb57a // == master, tried also 5.0.4
click==7.1.2
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.1.6
kombu==5.0.2
prompt-toolkit==3.0.8
pytz==2020.4
redis==3.5.3
six==1.15.0
vine==5.0.0
wcwidth==0.2.5

Other Dependencies

N/A

Minimally Reproducible Test Case

#!/usr/bin/env python3

from celery import Celery

app = Celery(
'tasks',
broker='amqp://user:***@**:5672/**',
backend='redis://:**@**:6379/1',
)

@app.task(ignore_result=False)
def test(*args, **kwargs):
return 'x' * 536870911 # 512MB

Expected Behavior

I'd except an exception or error. There is no point in retrying storing result to Redis when it simply never fits there. String limit is 512 MB.

I could check size of the data I'm returning in task. However, Celery adds additional metadata to my result so I do not know to what size should I limit my result.

Actual Behavior

[2020-12-09 08:38:52,786: ERROR/ForkPoolWorker-8] Connection to Redis lost: Retry (0/20) now.
[2020-12-09 08:38:53,328: ERROR/ForkPoolWorker-8] Connection to Redis lost: Retry (1/20) in 1.00 second.
[2020-12-09 08:38:54,940: ERROR/ForkPoolWorker-8] Connection to Redis lost: Retry (2/20) in 1.00 second.
[2020-12-09 08:38:56,547: ERROR/ForkPoolWorker-8] Connection to Redis lost: Retry (3/20) in 1.00 second.
.... and so on

Redis fails with the following error when you try to set string larger than 512 MB:

raise ConnectionError("Error %s while writing to socket. %s." %
redis.exceptions.ConnectionError: Error 32 while writing to socket. Broken pipe.

Tried via another script with direct Redis connection. This says nothing about input size therefore I think value size should be checked before sending data to Redis.