When you're looking for a cheap horse, keep your eye out for a free one. Finding someone who needs to give up their horse is easier than ever today with the help of social media sites. You could buy a grade or older horse in the range of $500 to $50000 but the cheaper the horse, the more you’d spend later on vet bills, training, etc.