I'm extremely cynical and distrustful of AI-first. I don't accept the three "truths" FTA:

>>>>>

We developed the following truths as anchors for why it’s so important to take a human-centered approach to building products and systems powered by ML:

1.) Machine learning won’t figure out what problems to solve. If you aren’t aligned with a human need, you’re just going to build a very powerful system to address a very small—or perhaps nonexistent—problem.

2.) If the goals of an AI system are opaque, and the user’s understanding of their role in calibrating that system are unclear, they will develop a mental model that suits their folk theories about AI, and their trust will be affected.

3.) In order to thrive, machine learning must become multi-disciplinary. It's as much–if not more so—a social systems challenge as it's a technical one. Machine learning is the science of making predictions based on patterns and relationships that've been automatically discovered in data. The job of an ML model is to figure out just how wrong it can be about the importance of those patterns in order to be as right as possible as often as possible. But it doesn't perorm [sic] this task alone. Every facet of ML is fueled and mediated by human judgement; from the idea to develop a model in the first place, to the sources of data chosen to train from, to the sample data itself and the methods and labels used to describe it, all the way to the success criteria for the aforementioned wrongness and rightness. Suffice to say, the UX axiom “you are not the user” is more important than ever.

<<<<<

My take:

1. I like the "human-centric" AI goal, but in the end we're inevitably going to move slow, faulty humans out of the loop. No matter how "thoughtful" we are along the way, the inevitable result is that we always want to take humans out of the loop to save money and avoid hard work and instead automate it away, because of sheer economics. It's a mistake to think AI is not going to figure out what problems to solve, or that we are not going to let AI make its own goals (it already makes its own subgoals at multiple levels). We are continually incapable of figuring out what problems to work on. While the planet warms up, we waste energy on cryptocurrency bubbles. While the ocean fills up with trash we make chat apps and self-driving cars and better smartphones. These decisions are dumb. We'll eventually realize that we suck at focusing on the right problems and put AI into the decision making processes. Worse, we'll want to (stupidly, IMO) pursue artificial general intelligence to do that. Each step we get closer we'll just hope for more and better outcomes from the AIs, and cede more and more decisions to the AIs. And then one day we'll find it's terrifying to have an inscrutable AI develop its own goals at a superhuman pace.

2. AI's goals are always opaque. Especially when that AI
system is run by a massive opaque for-profit corporation. We all know what such corporation's ultimate goals are: make as much money for shareholders as possible. It's great that we tell ourselves stories about how awesome AI is for people, but make no mistake, AI is just the latest weapon in the ever-escalating pursuit of perpetual exponential growth. Who the hell knows what its goals will be in the end.

3. This is an unparseable smokescreen going in six directions at once. Translation: we have no idea what we are doing or why, or whether it is going to work. But we are committed to it, and we're full of a blind faith in technology. Maybe if we had more Sociologists it will work?