Rework spec and buildsys a bit

Change-Id: I89e564f9f25da969a9723d02e9957da9bcda6659
Signed-off-by: Fabian Deutsch <fabiand@fedoraproject.org>