Yes, you definitely should use a toothpick or two to close the opening on the stuffed chicken before cooking it. If you don't, there's a good chance that a lot of the filling will be pushed out as the meat cooks.