Wheels are safe on a high chair, even with a baby in the chair. Think about a stroller, they have wheels. But you do need to keep the baby buckled in in the high chair before pushing them around. Also, most high chairs have caster wheels and they are not meant for large or a lot of movement. Moving around the kitchen is one thing, their older sibling pushing them at full speed around the house is another.