Issue 1:
The option app.conf.worker_pool is not honored. As I've been requested in previous issues to follow the template, I will try to.

Checklist

I have verified that the issue exists against the master branch of Celery.

This has already been asked to the discussion group first: I didn't ask. This is a super easily reproducible bug, and I git bisect to identify the commit responsible.

I have read the relevant section in the

contribution guide

on reporting bugs.

I have checked the issues list

for similar or identical bug reports.

I have checked the pull requests list

for existing proposed fixes.

I have checked the commit log

to find out if the bug was already fixed in the master branch.

I have included all related issues and possible duplicate issues

in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

I have verified that the issue exists against the master branch of Celery.

I have included the contents of pip freeze in the issue.

I have included all the versions of all the external dependencies required

to reproduce this bug.

Related Issues

None

Possible Duplicates

None

Environment & Settings

5.0.5 (singularity)

celery report Output:

software -> celery:5.0.5 (singularity) kombu:5.0.2 py:3.8.2
billiard:3.6.3.0 py-amqp:5.0.5
platform -> system:Linux arch:64bit, ELF
kernel version:4.19.121-linuxkit imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

deprecated_settings: None
worker_concurrency: 1
worker_pool: 'solo'

Steps to Reproduce

Required Dependencies

Minimal Python Version: N/A or Unknown

Minimal Celery Version: N/A or Unknown

Minimal Kombu Version: N/A or Unknown

Minimal Broker Version: N/A or Unknown

Minimal Result Backend Version: N/A or Unknown

Minimal OS and/or Kernel Version: N/A or Unknown

Minimal Broker Client Version: N/A or Unknown

Minimal Result Backend Client Version: N/A or Unknown

Python Packages

pip freeze Output:

amqp 5.0.5
billiard 3.6.3.0
celery 5.0.5 /tmp/celery
click 7.1.2
click-didyoumean 0.0.3
click-plugins 1.1.1
click-repl 0.1.6
kombu 5.0.2
pip 21.0.1
prompt-toolkit 3.0.18
pytz 2021.1
setuptools 54.1.2
six 1.15.0
vine 5.0.0
wcwidth 0.2.5
wheel 0.36.2

Other Dependencies

N/A

Minimally Reproducible Test Case

from celery import Celery

app = Celery(
'worker',
)

app.conf.update(
worker_concurrency=1,
worker_pool='solo',
)

Expected Behavior

The banner displayed by celery -A worker worker should contain concurrency: 1 (solo)

Actual Behavior

The banner contains concurrency: 1 (prefork). The parameter worker_concurrency is honored, but worker_pool is ignored.

The problem apparently comes from 01651d2

Issue 2:
Checklist

I'm pip installing Celery from master.

Steps to reproduce

Submit a task to an SQS broker and verify that there are 1 messages available and 0 messages "in flight" (total messages: 1). Launch a worker connected to that broker and now verify that there are 0 messages available and 1 messages "in flight" (total messages: 1). Shutdown the worker before the task completes (via Ctrl + C). The task will then be resubmitted with the following message:

Restoring 1 unacknowledged message(s).

Verify that the total number of messages in SQS is now 2 (1 available and 1 "in flight").

Repeat.

Expected behavior

I expect the total number of messages in the SQS broker to remain constant, as is the behavior I observe when I perform the repro with a Redis broker.

Actual behavior

Messages are duplicated, which can cause the queue to grow in an unbounded way.

Adding to this, I have long-running tasks (~45 min) and notice that I quite frequently see this error message

[INFO] botocore.vendored.requests.packages.urllib3.connectionpool: Resetting dropped connection: sqs.us-west-2.amazonaws.com

in conjunction with Restoring N unacknowledged messages.

It seems that dying SQS connections are causing workers to restart, which is in turn causing my queue to grow. (FYI: overnight, it grew from ~10 jobs to 13,000 because of this behavior when running a cluster of ~20 machines)