Okay, I've been banging my head against a wall about this problem for a long time and I don't have a great answer, but I have a decent answer that I've been honing in on. Let me know what you think, and please correct any of my misunderstandings.

PGP keys come in different flavors which have different capabilities. There are Authentication keys, Signing keys, Encryption keys, and Certification keys. A PGP key can have any or all of these capabilities at the same time.

PGP keys can also have subkeys, and these subkeys can have any of these capabilities as well.

A Certification key can be used to create and sign other keys.

A Signing key can be used to sign data to prove it comes from you.

An Authentication key can be used for things like SSH to authenticate yourself to servers.

An Encryption key can be used to decrypt data that people encrypt against your public key.

Yubikeys actually have multiple PGP slots in them, and can accommodate three keys, one Signing key, one Authentication key and one Encryption key. You can easily pick and choose subkeys from a PGP key to fill any or all of the slots on the Yubikey.

(The key has other applets on it as well, like U2F and other things but lets ignore that for now)

So you can actually use your Yubikey to log into servers via SSH, to sign messages you send and actions you do (emails, blog posts, files, git commits), and to decrypt messages that are sent to you.

Now in order for you to use different yubikeys to log into different things, or for multiple factors with each other, each yubikey needs to have a different authentication key on it. And if a yubikey is lost, that authentication key needs to be revoked.

If you were to create a new PGP key for each of these things, it becomes really painful to manage them all, and keep them all safe, like you said.

However, if you have a single "master" key, whose only role is to create subkeys, you can start to have a somewhat saner experience. Here's how it works.

On an airgapped computer you create a master PGP key and only give it the capability to Certify, aka create and sign other keys. This key will never be on a device that touches the internet for the rest of your life. You will only use this key offline to create subkeys. Full stop.

Now, on your airgapped computer you can create subkeys. You can create a different Authentication subkey for each yubikey you intend to use. Then you can move these subkeys to the different yubikeys. Now all the keys are based off the original key and can be easily revoked and new ones created.

Now in order for people to be able to send you encrypted messages, you need to publish your public Encryption key. This key doesn't generally change often because it's used long term to decrypt messages. You can create a single Encryption subkey and put the same one on all your different yubikeys so any of them can decrypt messages. This means no more looking for the right key in that scenario.

Finally you have signing keys, which I personally use to sign my commits and emails (occasionally). Again each yubikey should have its own signing key that they can easily be revoked.

When you publish your public key, people will be able to see that you have N signing keys (on N yubikeys), any of which can be used to sign messages from you.

Now you can think of yubikeys as device-specific, instead of account-specific.

You have a yubikey in your laptop, a yubikey in another laptop, a yubikey on your keychain, and a yubikey in a vault. They can all access your servers, sign your commits and decrypt your emails. They can all be U2F devices for various services.

This way you only need to keep a single private key safe. Now this is a whole different, very challenging issue but that's a topic for another post.