It is through experience and reminders that they learn they can't just take things they want. That isn't yours. We need to pay for that before you take it. Our role as adults in a children's lives is to help them build impulse control. We as adults can know something is wrong yet we still do it.
Children are the same way.
For the most part, this refers back to the impulse control. We can help by walking through the decision-making process. Sometimes it is very hard to do the right thing. Perhaps a child feels they are not getting enough attention or love and getting caught stealing at least shows them adults care enough to get angry. Perhaps a child feels he has no control in his life and steals to show the world that they can't control everything he does.