Your make a good point.  Let me first address it from the perspective of Marshall Hodgson.

Civilization is a somewhat arbitrary term, often narrowly defined by language.  This obviously has its limits.  On the other hand, the more broadly you define a civilization, the more subgroupings it must include.  Eventually, we come to see civilizations as groupings useful for the purpose at hand.  And ultimately, civilizations, like religions, are self-defined.

Hodgson considers the lands of Islam to constitute a world civilization.  He calls Islam a world civilization in the title of his book The Venture of Islam: Conscience and History in a World Civilization.

For another perspective, I can turn to H. Patrick Glenn, in Legal Traditions of the World, speaking on tradition and identity.  Not all is arbitrary in the definition of a tradition, or civilization.

Glenn quotes Fernand Braudel, who speaks of an "underlying structure" of civilization; "those elements of a civilization which could not be changed without a fundamental change, or more probably disappearance, of the tradition itself."  Applying this criteria to Islamic civilization, one immediately understands that structure to be the religion of Islam.

To summarize: looking at the lands of Islam, with its concept of Umma, the shared traditions, the shared Arabic religious and often secular language, and indeed the shared religion of Islam, it seems logical to fall in with those who perceive the outline of a world Islamic civilization.