Theoretically, you can swing a baby too fast. But in reality, it's hard to do. If you swing a baby too fast, they will likely get dizzy and sick. So while you can swing a baby too fast, it's not recommended.