Free allocator only of it exists