Analogical reasoning has strong theoretical foundation. It is logical. It isn't an accident that analogy shares a root with logic. They are fundamentally related. Something like syllogistic logic is itself an analogical reasoning method. If you can map to exactly the same or to similar enough logical structures for two different things then you can safely use the analogy that is the symbols as a proxy for reasoning about the thing you have made an analogy to and despite being different things you can have high confidence that doing so is not in error. Ditto for math. This isn't dangerous, but one of the most important and greatest advances that humans ever formalized.

Anthromorphism is an instance of thinking via proxy by analogy to another structure. The biggest issue with it is that it carries with it far more baggage. For something like mathematics, you are dropping units: three apples plus three apples to six apples is pretty easy to justify analogically as three unitless plus three unitless to six unitless. The analogical similarity is obvious. For agents, well, it isn't so clear whether analogies are justified. They could be, but there is a lot more that could go wrong because there are so many more assumptions that the analogy is making. As you get more complicated structures, you have more room for error, so you have more tendency to error. So even though analogy is fine, the greater potential for error makes the lazy detector just classify this analogical approach as fallacious. However, it might not be and it might not even be dangerous.

Typically when people disagree with anthropomorphism they do so because the transitional structure isn't similar enough to justify the analogy. For example, one of the more infamous dangers is wasting resources and time seeking intervention from a non-agentic being, like a statue made up of pieces of wood. Since an agent can respond to your requests, including to help, but the piece of wood can't, the analogy doesn't hold. So the proxy relationship that the analogy seeks to make use of isn't reasonable. So you can't trust your conclusions made through analogy to hold in the different decision context. The beliefs aren't generalizing or they don't have reach or they aren't universal or whatever you want to call it that lets you know your thinking isn't working.

In this case it is pretty obvious that the transitional structure has a lot of things that make the analogy valid. The most obvious is that this structure is related to the other structure is an optimization target of the machine learning model. We have mathematical optimization seeking to make these two structures similar. So analogy is going to have some limited applications where it is going to be valid. If you tried to propose something beyond that limited set, for example, that it would walk, because the proxy structure didn't have that as a part of its objective function, you wouldn't have strong reason to suspect congruence.

But that is only one level at which this analogical structure is appropriate or inappropriate or dangerous or non-dangerous. That is on the level of whether the map corresponds with the territory.

Agents are kind of awesome in a way that the rest of reality isn't, because the map ought to not correspond with the territory. So analogies can seem less valid than they really are. With anthropomorphism we are in a unique situation relative to other decision making contexts. We confront both undedicability and also intractability. The former is a regime where logic can create logical paradoxes. The latter is a realm where, because of the limitations imposed, a lot of arguments seem sound and valid, but aren't, because the analogy they imply doesn't correspond to the resource limitations that constraint correct thinking.