> So, it will have to learn, and LLMs aren't good at learning

LLMs are bad at human-like learning, but their zero-shot performance + semantic search more than make up for it.

If you give an LLM access to your Spotify account via an API, it has access to your playlists and access to details about each song like `BPM`, `vocality`, even `energy` :

https://developer.spotify.com/documentation/web-api/referenc...
https://developer.spotify.com/documentation/web-api/referenc...

An LLM with no prior explanation of either endpoint, can figure out that it should look at your favorites playlists, and find which songs in your favorite list are most suitable for a tired person.

-

But it can go even further and identify its own sorting criteria for different situations with chain of thought:

Bedroom at night: https://chat.openai.com/share/6b1787ef-cd84-4834-b582-5024f8...
Kitchen at 5pm: https://chat.openai.com/share/7ddaa047-0855-48c1-bcea-308083...

Rather than blindly selecting the most relaxing songs it understands nuance like:

> Room State: "lights on" and "garage door open" can imply either returning home from work or engaging in some evening activity. The environment is probably not yet set for relaxation completely.

And genuinely comes up with an intelligently adapted strategy based on the situation
-

And say it gets your favorite wrong, and you correct it: an LLM with no specialized training can classify your follow up as a correction vs an unrelated command. It can even use chain-of-thought to posit why it may have been wrong.

You can then store all messages it classified as corrections and fetch those using semantic similarity.

That addresses both the customization and determinism issues: You don't need to rely on the zero-shot performance getting it right every time, the model can use the same chain of thought to translate past corrections into future guidance without further training.

For example, if your last correction was from classical music to hard metal when you got back from work, it's able to understand that you prefer higher energy songs, but still able to understand that doesn't mean every time in perpetuity it should play hard metal

Kitchen w/ memory: https://chat.openai.com/share/43635427-55d5-4394-b282-46acae...
Bedroom w/ memory: https://chat.openai.com/share/8c146dd5-2233-4aba-8f6a-b97b7a...

-

I experimented heavily with things like this when GPT came out; part of me wants to go back to it since I've seen shockingly few projects do what I assumed everyone would do.

LLMs + well thought out memory access can do some incredible things as general assistants right now, but that seemed so obvious I moved on from the idea almost immediately.

In retrospect, there's an interesting irony at play: LLMs make simple products very attractive. But if you embed them in more thoroughly engineered solutions, you can do some incredible things that are far above what they otherwise seem capable of.

Yet a large number of the people most experienced in creating thoroughly engineered solutions view LLMs very cynically because of the simple (and shallow) solutions that are being churned out.

Eventually LLMs may just advanced far enough that they bridge the gap in implementation, but I think there's a lot of opportunity left on the table because of that catch-22