The cost of a horse can greatly depend on many factors, but generally, some of the more affordable horse breeds include Quarter Horses and certain types of ponies. However, remember that the initial cost of buying a horse is only a small part of the overall expense.