Got it working by adding the libraries as catkin package exports