I listened off and on for decades and it never really "clicked". Then at some point in time, well into my adult life, I realized that I was quietly obsessed with Aphex Twin tracks and listened to them all the time. Broadly speaking, I think it's just about paying attention to details.

For example in SAW2 trk1 "cliffs" there's kind of a "peak and drop" moment. It's implemented as a very small shift in phasing between two voices, one which was lagging the other begins leading the other. And maybe Although it's barely audible the feeling of the track changes from... maybe a wistful reverie on a solitary walk on an ocean clifftop in the breeze and sun of a late-summer afternoon... to something more "unblocked", like joyfully running down a hill to home.

That kind of detail -- "Tempo" as a feeling, not just a BPM, accomplished by changing other stuff besides pitch and average loudness of notes. It is the sort of thing that one attends to when rehearsing with a good symphony conductor or a chamber-group coach. Or with a music teacher, e.g. "play through the measure to here [drawn out slightly]". You might be instructed to play something "scherzando", playfully, or "andante", leisurely. And God help you if you play something "lightly" instead of "playfully", or "dragging" rather than "stately", or vice versa! Rendering subtle emotional nuance through soundmaking tools is certainly learnable, and you get to learn Italian/Russian cursing for free.

Or "The oboes here are cats, there's a cat in heat and the oboes are meowing around. Then the bassoon comes in and it's like a pregnant cat and it's not so excitable". That one came from a world-class Boston Symphony musician, and it's absolutely serious.

Really good musicians play with sound. Sometimes they do things for the lols. Music being about pattern matching, good musicians also play with the listener's sense of expectation and fulfullment. Good musicians do with sound what good lovers do with touch. Good musicians do with sound what good comedians do with concepts.

There are lots of good musicians in the world, and many don't have good recordings. Afx is one who managed to make a pretty good musical career out of his artistic vision, which seems rare. It's nice that it's easy to find recordings and people who've heard those recordings, and there's always something new to hear in them.

An example of the expectations thing: An absolutely beautiful melody buried under much louder boring/aversive voices. It's impossible to focus on the melody so it leaves you wanting more. Related to this is the "bass drop" and the idk "fake-out bass drop", maybe. I think "teasing" / "trolling" / "ruining or messing with expectations" is something that seems pretty characteristic of Aphex Twin tracks.

I think enjoying "challenging listens" is really just about exposure and mindfulness. Truly listening to all music helps. And playing music, only because it forces you to pay attention to what actually makes things sound the way they do. And paying attention to the sounds of the world around you, and how sounds are modified by the environment. And listening to the same recording over and over agin, to build familiarity.

Personally, I became a total afx fanboy over decades, and it was an incremental and slow process. I was thoroughly and extensively trained in classical music through high school; I mostly just listened to Drukqs for the piano and endured the other stuff. In college I ate acid more often than usual; it was general knowledge that "listening to Aphex Twin... enhances your trip, it's like hacking your brain or something" and then you got on with the rest of your life. And then one day I found myself dissecting the auditory nuances of why I liked something, or why something seemed like a deliberate choice to irritate the listener.

I think appreciating nuance is just something that comes with age, in that experience of the fullest range and detail of human experience comes with age. And also appreciation of a wider range of qualia, just like stinky cheese is something one usually does not appreciate immediately.

Also, it's instructive to watch musicians playing, and see how they communicate with visuals, body cues, and sound. Chamber groups are good. Some bush-league music festivals are good, after the main day is over sometimes the professional musicians jam with each other for the lols, or sit in on another group's late night set.