controlnet: Make it possible to load points from user dir