cocos2d/cocos2d-x — pull request #6318 by sachingarg05 (2014-04-17 17:01:06+00:00)
Title: Fix in Renderer to support PingPong (multi step) RenderTexture usage.

The new renderer implementation with multiple queues gets VERY confused when trying to use two RenderTextures to do multi-step processing using shaders (PingPong RenderTextures).

The problem description is big, but the code changes needed to fix are simple and 'safe' :-)

Solution description:
1) Renderer::visitRenderQueue is now public (was protected).
2) Renderer::visitRenderQueue takes int renderQueueID as parameter (parameter was RenderQueue& queue).
3) RenderTexture::end() now calls Renderer::visitRenderQueue to execute its own queue.

Problem description:

Lets say we want to execute 4 shaders one by one in this order, output of shader1 is input of second2, and so on:
Shader1, Shader2, Shader3, Shader4.

When trying to do multipass shader processing using two RenderTexture objects (known as PingPong processing), the expected order of execution is:

Correct Order:
RenderTexture1 > Shader1 > RenderTexture2 (Queue1)
RenderTexture2 > Shader2 > RenderTexture1 (Queue2)
RenderTexture1 > Shader3 > RenderTexture2 (Queue1)
RenderTexture2 > Shader4 > RenderTexture1 (Queue2)
... and so on.

The new cocos2dx renderer creates a queue for each RenderTexture, commands are pushed to the queues, and then queues are executed one by one.

There are two problems here:
1) When same RenderTexture object is used twice, the commands go in same queue. But the CommandGroup is pushed into renderer twice.
2) Order of execution is different from expected order.

Result is this order of execution which is wrong: (The 'Repeated' commands are executed twice because commandGroup object was pushed each time, this is wrong)

Queue1 (wrong order):
RenderTexture1 > Shader1 > RenderTexture2
RenderTexture1 > Shader3 > RenderTexture2
RenderTexture1 > Shader1 > RenderTexture2 (Repeated)
RenderTexture1 > Shader3 > RenderTexture2 (Repeated)

Queue2 (wrong order):
RenderTexture2 > Shader2 > RenderTexture1
RenderTexture2 > Shader4 > RenderTexture1
RenderTexture2 > Shader2 > RenderTexture1 (Repeated)
RenderTexture2 > Shader4 > RenderTexture1 (Repeated)

Solution:
RenderTexture should execute its queue in RenderTexture::end().

Reasons:
1) Each RenderTexture step is independent of other render commands and its queue can be executed independently without any performance loss.
2) Each RenderTexture step must finish executing before second RenderTexture step can be executed. We can't wait till the end.
3) After each RenderTexture step, its queue must be emptied so that queue can be used in next RenderTexture step.