NuGet packages.config was outside the project so intalling and uninstalling packages was really wonky because of that it seems.