filterelement: Remove input and output adapters

They are negotiated by the agnosticbin if it is needed

Change-Id: I7d081c488a9e43175c7b89cf2d79273e91122268