The general consensus is to have 35-40 Vigor by the time you reach Mountaintops of the Giants, to not die in one hit, then 55+ Vigor by the end of the game. Player skill and different playstyles can allow players to get by with a lower amount.