Many baby bouncers come with a baby-sized insert to help your child feel more comfortable. You may also wish to attach toys or other baby gear that can provide entertainment and distraction from the discomfort of being in an unfamiliar place, such as their favorite pacifier, baby monitor, or something they may have been playing with during tummy time.
You could also try switching up how you position them inside the seat - some babies prefer sitting upright while others like lying down on their backs so they can look around. It might take a little bit of experimentation to find out what works best for them at this stage!