You can use suction cups by attaching them to the bottom of the caddy. It will fix them to the shower wall and further increase their stability. You can also use adhesive strips, which you will find at various hardware stores.