agnosticbin2: Remove target pad when a pad is added to queue

Change-Id: I0b2bf9057a51b2b077511f706e7fa344607701b5