I'm sure what I'm going to write are not new insights but, to add to your point, generally we expect to consume information much quicker through reading than through hearing. When reading, say English, we don't really sound through each word but recognize the shape as you say. More concretely, it's probably our brains' pattern matching of frequently occurring clusters/patterns of letters: "psy" in "psychology" for example, if you spelt that "phonetically" as "saikolojee" I doubt people would recognize that.

I don't know Korean, but I've been told that it has plenty of Chinese loanwords (or Chinese-derived character compound "words") like Japanese, yet because it is less homophonous than Japanese, Korean people nowadays are very able to recognize such words by the shape of them when written in Hangul.

As a native Chinese speaker, Cantonese to be precise, my opinion is probably biased but I find Japanese to be so homophonous (for Sino-Japanese words 漢語 but even for native words 和語 to a degree) that even if Japanese adopted a more "efficient" syllabary like Hangul, words would still be difficult to decipher, spaces included. Pitch accent only helps so much (and of course that isn't written down in kana), the typical examples being 紙 ("kamì": paper), 髪 ("kamì": hair) and 神 ("kàmi": god). I'm trying to indicate the pitch accent in an adhoc manner with the accented letters; note that according to my (native) Japanese dictionary, 紙 and 髪 have the same pitch accent. So I think, a more phonetic writing system for Japanese, that remains efficient for reading, would end up annotating words with semantic or etymological hints, like the idiosyncratic spellings of Latin and Greek root words in English.

Also, the texts we read are allowed to use more sophisticated words, more literary words, with more complicated sentence structures. So while phonetic spelling ought to work to represent ordinary speech, that's not necessarily the case for general written expression. In practice, I find 漢語 in Japanese speech to be surprisingly difficult to comprehend, in the sense that, words are often too indistinct for me to pick them up by ear if I have not heard them in speech before, even if I have seen them in text before and would know the characters already (including their readings 音読み in Japanese).

That's different for me as a Cantonese speaker where I can pick up new literary compound words if their constituent characters are ones I know from other words/compounds, even rather infrequent ones. I would say it's because the sound system of Cantonese has 6 tones, mapping nearly one-to-one with the tones + voiced/voiceless distinction from Middle Chinese, which in turn is a much more monosyllabic language where characters have quite distinct sound values compared to any of the modern Chinese languages.

Incidentally I have a harder time picking up new terms in Mandarin by ear where many characters that are distinct in Cantonese sound the same; and it's widely agreed that Mandarin Chinese has evolved more disyllabic words to compensate. Again, yes, even when I know the compounds already through writing, learning to pattern-match for them in hearing in real time, has been, unfortunately, for me at least, a different story.

(Edit: added below)

I believe also that both Chinese and Japanese got more homophonous because they could get away with it (i.e. people got lazy in pronouncing more intricate sounds) when there was 漢字 to distinguish characters/words when needed. So there's certainly a feedback loop in there. If for the 2 to 3 thousand years of history there was no writing in ideographic characters, there would have been evolutionary linguistic pressure against too many homophones in the languages.