For the most part, you can lift safely without collars. If you lift under control with a weight that you can handle and maintain proper form, the weights shouldn’t go anywhere. But if you want to push your strength limits and/or work with speed, the chances that your weight pates may move around or fall off will greatly increase.
Think of a set of barbell collars as a seat belt for your barbell. You hope you never need it, but you’ll be glad it’s there when you do.