Clean up of removing extra libraries

Signed-off-by: Tony Stephens <tony.stephens@agilex.com>