I understand what you are saying, but I think it's too broad a generalization and more applicable to places that had historic ethnic mixtures, such as Vilnius / Wilno or, on the other side of Germany, Strasbourg / Straßburg. And as I've tried to explain before, in the West, particularly in the U.S., people do think in ethnic terms, about Europe anyway. Königsberg a) always was ethnically German, and b) never was de jure part of Poland, i.e. Poland proper. Although the (German) dukes of Prussia were obliged to render homage to the Polish monach for 150 years or so, this was a result of the defeat of the Teutonic Order and its aftermath, not a symptom of Königsberg's ethnicity, which remained German until 1945-46, when Königsberg as such ceased to exist.

Thus, for the purposes of English speakers, Königsberg was a German city all its life.