Quite a few people say that they find it hard to meditate. When asked what they are trying to do when they meditate, they usually reply that they are trying to still their minds. That is not meditation. Meditation is not about keeping your mind in some sort of catatonic state. Nature loathes a vacuum.
Meditation is all about 'sati' - awareness.
Being aware of where the focus of your mind wanders and learning to take control of that focus. The observing of your breathing in meditation is merely a device used to catch your mind change focus. Nature loathes a vacuum, and keeping the mind still is like keeping a beautiful car in the garage. It was designed for movement. The idea is to get the car onto the road and move.