This operation can be cached. Zsh completions support caching. You'll have to read `man zshcompsys` though.