Yes, you can place a bet on a horse not to win and this is called a lay bet.
Laying a horse is when you back a racehorse not to win the race.
Essentially, the punter becomes the bookie in a bid to cover any winnings if the horse was to get his head in front. It can cost a lot of money laying bets because you need to pay out the bet amount multiplied by the odds of the horse in the betting market.