> Whatever the argument is, there's an argument that having real people instead of programs matters for some things. And the standard we can use is something like "the things where they aren't interchangeable" at least (although others may have other qualities to emphasize). So, a person who literally just does the exact same thing a program can do can be replaced by the program.

I agree. But my point is that most people who talk about these things don't do a very good job in thinking about which things should and should not be automated. They basically just look at whatever has been automated (which they love being automated, now that it has been) and say "Automating that stuff was great!", and look at the things that haven't been automated and say "Automating that stuff is bad! It is essential to the human experience!".

> For example, data entry. A person who cannot be duplicated nearly exactly by a program is where we want to be careful about considering keeping the human position around. For example, teachers.

Why? Why would automating teachers be bad? I'll give you two separate arguments why this would not be a bad thing:

1. Access: Teachers require human labor, which is in finite supply. This means that not every child on earth has access to teachers, and sometimes if they do, the teachers they do have access to are spread incredibly thin, making them effectively useless. Automating teaching has the potential to massively expand access to the extremely underprivileged. Teachers also operate on a set schedule. If the class starts at 8pm, and you have a job that requires you to be at work at that time, well, tough luck for you. If you automate teaching, this problem goes away.

2. Personalization, optimization, and equity. Human teachers can tailor their lessons a little bit to the individual, but high quality AI instruction promises to expand that greatly. Also, not all human teachers are good. Figuring out which ones are and are not good takes a lot of time, and wastes thousands of hours of child development, and sometimes doesn't happen at all. If we had artificial instructors, they would be the same for everyone. We would be able to optimize their methodology, and ensure they are teaching at a minimum standard of quality - however we want to define that. This also opens up massive opportunity for improving equity among races, classes and whatever other social divisions you want to impose. Now, that isn't a given, AI has the potential to encode our own biases. But unlike in human instruction, we actually have the opportunity to correct those biases if we're careful in designing the AI - we have no such opportunity for human instructors.

The problem, I think, is that when people think about AI replacing teachers, they think about all their favorite experiences with human teachers, and how those might be lost. And that may well be true. But what about all of the negative ones? What about all the kids who's teachers were lazy or having a bad day that day? AI doesn't have bad days. It doesn't get tired. It doesn't dislike children because they're queer. It doesn't get frustrated by insolence. It teaches at the same standard of quality every day.

So yes, maybe some of the great human experiences may be lost. But so too will the negative ones. And it's really not obvious which side of that equation is more important. But that's sort of the point: it's not obvious.