The major difference is feeding behavior - sheep are grazing animals while goats are brousers. Goats, although very social animals, are not as flock-oriented as sheep.
Goats are more likely to seek shelter in wet weather than sheep.
Male goats will rear up on their hind legs and lunge downward to butt heads while male sheep will run at, or charge, each other to butt heads. If you mix male goats and sheep, the sheep will dominate because they don't play by the same rules and will charge the goats while they are rearing up. Goats have an erect tail, sheep have hangy-down tails. Most goats have beards. Sheep do not have beards.