Issue 1:
Checklist

I have included the output of celery -A proj report in the issue.

(if you are not able to do this, then at least specify the Celery

version affected).

$ celery -A app report

software -> celery:4.0.2 (latentcall) kombu:4.0.2 py:3.4.3
billiard:3.5.0.2 redis:2.10.5
platform -> system:Darwin arch:64bit imp:CPython
loader -> celery.loaders.app.AppLoader
settings -> transport:redis results:disabled

broker_url: 'redis://127.0.0.1:6379/3'

I have verified that the issue exists against the master branch of Celery.

Steps to reproduce

Tasks that finish and succeed after the warm shutdown signal are not acked if acks_late=True. To reproduce:

Create app.py:

from time import sleep

from celery import Celery

app = Celery('foo', broker='redis://127.0.0.1:6379/3')

@app.task(acks_late=True)
def foo():
sleep(5)
return 'foo'

Queue up a task:

$ python -c "from app import foo; foo.apply_async()"

Start the worker and issue a warm shutdown (CTRL+C) before the task finishes.

$ celery -A app worker --loglevel=debug
[2017-01-30 16:23:46,092: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2017-01-30 16:23:46,094: DEBUG/MainProcess] | Worker: Building graph...
[2017-01-30 16:23:46,094: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Pool, Autoscaler, StateDB, Beat, Consumer}
[2017-01-30 16:23:46,108: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2017-01-30 16:23:46,109: DEBUG/MainProcess] | Consumer: Building graph...
[2017-01-30 16:23:46,124: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Gossip, Agent, Tasks, Control, Heart, event loop}

celery@foobar v4.0.2 (latentcall)

Darwin-16.3.0-x86_64-i386-64bit 2017-01-30 16:23:46

[config]
.> app: foo:0x1034ddc88
.> transport: redis://127.0.0.1:6379/3
.> results: disabled://
.> concurrency: 4 (prefork)
.> task events: OFF (enable -E to monitor tasks in this worker)

[queues]
.> celery exchange=celery(direct) key=celery

[tasks]
. app.foo
. celery.accumulate
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap

[2017-01-30 16:23:46,139: DEBUG/MainProcess] | Worker: Starting Hub
[2017-01-30 16:23:46,139: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:46,140: DEBUG/MainProcess] | Worker: Starting Pool
[2017-01-30 16:23:46,248: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:46,249: DEBUG/MainProcess] | Worker: Starting Consumer
[2017-01-30 16:23:46,250: DEBUG/MainProcess] | Consumer: Starting Connection
[2017-01-30 16:23:46,268: INFO/MainProcess] Connected to redis://127.0.0.1:6379/3
[2017-01-30 16:23:46,268: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:46,269: DEBUG/MainProcess] | Consumer: Starting Events
[2017-01-30 16:23:46,278: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:46,278: DEBUG/MainProcess] | Consumer: Starting Mingle
[2017-01-30 16:23:46,278: INFO/MainProcess] mingle: searching for neighbors
[2017-01-30 16:23:47,309: INFO/MainProcess] mingle: all alone
[2017-01-30 16:23:47,309: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:47,309: DEBUG/MainProcess] | Consumer: Starting Gossip
[2017-01-30 16:23:47,316: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:47,316: DEBUG/MainProcess] | Consumer: Starting Tasks
[2017-01-30 16:23:47,320: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:47,320: DEBUG/MainProcess] | Consumer: Starting Control
[2017-01-30 16:23:47,325: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:47,325: DEBUG/MainProcess] | Consumer: Starting Heart
[2017-01-30 16:23:47,329: DEBUG/MainProcess] ^-- substep ok
[2017-01-30 16:23:47,329: DEBUG/MainProcess] | Consumer: Starting event loop
[2017-01-30 16:23:47,329: INFO/MainProcess] celery@foobar ready.
[2017-01-30 16:23:47,330: DEBUG/MainProcess] | Worker: Hub.register Pool...
[2017-01-30 16:23:47,330: DEBUG/MainProcess] basic.qos: prefetch_count->16
[2017-01-30 16:23:48,093: INFO/MainProcess] Received task: app.foo[a1d83951-0e24-488b-b400-0ddbd4161254]
[2017-01-30 16:23:48,093: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x10357c598> (args:('app.foo', 'a1d83951-0e24-488b-b400-0ddbd4161254', {'kwargsrepr': '{}', 'id': 'a1d83951-0e24-488b-b400-0ddbd4161254', 'task': 'app.foo', 'expires': None, 'reply_to': '6de20f21-2460-3189-b534-29ae63c15d1c', 'lang': 'py', 'group': None, 'argsrepr': '()', 'eta': None, 'retries': 0, 'delivery_info': {'exchange': '', 'redelivered': None, 'routing_key': 'celery', 'priority': 0}, 'root_id': 'a1d83951-0e24-488b-b400-0ddbd4161254', 'correlation_id': 'a1d83951-0e24-488b-b400-0ddbd4161254', 'origin': 'gen52076@foobar', 'timelimit': [None, None], 'parent_id': None}, b'[[], {}, {"chord": null, "callbacks": null, "chain": null, "errbacks": null}]', 'application/json', 'utf-8') kwargs:{})
[2017-01-30 16:23:48,095: DEBUG/MainProcess] Task accepted: app.foo[a1d83951-0e24-488b-b400-0ddbd4161254] pid:52112
^C
worker: Hitting Ctrl+C again will terminate all running tasks!

worker: Warm shutdown (MainProcess)
[2017-01-30 16:23:49,613: DEBUG/MainProcess] | Worker: Closing Hub...
[2017-01-30 16:23:49,613: DEBUG/MainProcess] | Worker: Closing Pool...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Worker: Closing Consumer...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Consumer: Closing Connection...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Consumer: Closing Events...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Consumer: Closing Mingle...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Consumer: Closing Gossip...
[2017-01-30 16:23:49,614: DEBUG/MainProcess] | Consumer: Closing Tasks...
[2017-01-30 16:23:49,615: DEBUG/MainProcess] | Consumer: Closing Control...
[2017-01-30 16:23:49,615: DEBUG/MainProcess] | Consumer: Closing Heart...
[2017-01-30 16:23:49,615: DEBUG/MainProcess] | Consumer: Closing event loop...
[2017-01-30 16:23:49,615: DEBUG/MainProcess] | Consumer: Stopping event loop...
[2017-01-30 16:23:49,615: DEBUG/MainProcess] | Consumer: Stopping Heart...
[2017-01-30 16:23:49,617: DEBUG/MainProcess] | Consumer: Stopping Control...
[2017-01-30 16:23:49,620: DEBUG/MainProcess] | Consumer: Stopping Tasks...
[2017-01-30 16:23:49,620: DEBUG/MainProcess] Canceling task consumer...
[2017-01-30 16:23:49,621: DEBUG/MainProcess] | Consumer: Stopping Gossip...
[2017-01-30 16:23:49,624: DEBUG/MainProcess] | Consumer: Stopping Mingle...
[2017-01-30 16:23:49,624: DEBUG/MainProcess] | Consumer: Stopping Events...
[2017-01-30 16:23:49,624: DEBUG/MainProcess] | Consumer: Stopping Connection...
[2017-01-30 16:23:49,625: DEBUG/MainProcess] | Worker: Stopping Pool...
[2017-01-30 16:23:53,099: INFO/PoolWorker-2] Task app.foo[a1d83951-0e24-488b-b400-0ddbd4161254] succeeded in 5.002987815998495s: 'foo'
[2017-01-30 16:23:54,103: DEBUG/MainProcess] | Worker: Stopping Hub...
[2017-01-30 16:23:54,103: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] Canceling task consumer...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] Closing consumer channel...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] | Consumer: Shutdown Gossip...
[2017-01-30 16:23:54,104: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2017-01-30 16:23:54,105: DEBUG/MainProcess] | Consumer: Shutdown Connection...
[2017-01-30 16:23:54,106: WARNING/MainProcess] Restoring 1 unacknowledged message(s)
[2017-01-30 16:23:54,111: DEBUG/MainProcess] removing tasks from inqueue until task handler finished

(Note the second to last line even after the log says the task succeded.)

Expected behavior

The task is acked before exit.

Actual behavior

The task is not acked and restored. The next time the worker starts up, the same task will be executed again.

Issue 2:
I want to use celery in my project and I need that the celery tasks (subprocesses) will run in a diffrent python virtualenv.

I noticed that there is an --executable flag but it will change the executable for all celery processes (main and tasks).

If this feature is not exeists I'm willing to put some work to add this but need some guidance on where to begin with.