In fact, storing a kayak vertically is the best way to store the watercraft without damaging any of its parts. The best way to do it is to turn your kayak in a vertical position and let it rest against the wall. Kayak’s cockpit has to be facing out when stored in this position. For your gear to not fall, make sure it is stored at a slight angle with the bow facing the ceiling.
Ideally, you should put some form of a cushion underneath the kayak in order to protect one of its ends from getting worn out. A few towels and a wooden crate work wonders for keeping your kayak in place.