[#83051254] Remove all files not specified in gemspec

Signed-off-by: Chad Woolley <cwoolley@pivotal.io>