If she cries, pick her up and feed her, if necessary. Feel free to take baby for a walk to the foyer or outside if that makes you feel more comfortable. Pick out a toy in advance to be the special 'church toy' for that week. Ask one of the welcomers for a pre-schooler's activity bag. Don't expect to worship privately yourself.
Family worship means that parents worship through the involvement of their children.
Children learn from the care you take to worship with them. Encouraging them to be involved as much as possible for their age will help them grow into the Christian life. Children of this age appreciate the order and predictability of the service. They get a buzz out of knowing what is going to happen next. Help them follow the worship in the books and explain what is happening during the service.