Remove unnecessary extra directory when building