Move pkg-config and man page to the component dir

Signed-off-by: Timothy Gu <timothygu99@gmail.com>