When I started designing speech interfaces, I was frustrated because I didn't know what to draw. Designing a speech interface was tough. I still haven't found any good tools to help design a speech interface, but I finally came to a technique of writing out dialogs as you would for a play. Then my team did readings. Just having things spoken out loud was amazing.
Consistency is another big difference.
You'd never design a graphical error dialog to be different every time it came up. But graphical designers moving into speech have some different issues to think about. Speech interface design needs to be a separate design effort. If you just tack it onto the graphical interface, your application is doomed to mediocrity, if not failure. Techniques that work in GUIs don't do so in speech applications.