The amount of land of the "city proper" doesn't matter that much. Like any other large world city, NYC has plenty of rail or car commuter suburbs, and peripheral cities with their own urban cores which still owe their existence to the main metropole. By that measure we should mostly be comparing New York's characteristics to the rest of the world's large metropolitan areas, and its overall density seems pretty average over the entire metropolitan area.

The core reason that cities get big is that they are economically beneficial[1]. Companies have access to more employees, suppliers, and customers. Residents have access to more jobs, and more people with/companies marketing to their interests (hence the mythical "cultural amenities" of big cities) and so on. The city as a whole is connected to other cities worldwide by transport links. It follows that at least some people see themselves better off in a city, and thus we would expect to see housing demand in big cities. Hence Tokyo's population increases despite Japan's decreasing, or this website's industry concentrating in a few counties in California. Obviously, this is true of New York as well. The ideal distribution of population between cities should also be different from the past. Many towns/cities in the Western world were founded to either serve as a central place for agriculture, to extract some sort of other resource, or to do some sort of industrial production using those resources. Both are greatly mechanized now, or using cheaper labour (whether migrant or offshore), hence the declining economies of many such places. It follows that bigger cities are the recepients of those leaving such places.

Following from the housing demand is a need for homebuilding. There are really only two broad ways for new homes to be built; greenfield or infill. The problem with greenfield development is that there is no land close to many urban centres left. Hence, greenfield is usually found at the edge of existing urban areas, and as a result, they are almost always far less dense, more car dependent (even with good transport links), and considered to be ugly "sprawl". They are also usually not within the jurisdiction of the central city anymore. There are often greenbelts around cities imposed by higher governments to legally prevent "sprawling" greenfield development from happening, although the answer is often just to keep on building greenfield sites beyond the greenbelt. Of course, the further you are from the economic nucleus the less the economic benefits of that nucleus. Transportation costs ($$ and time) increase and agglomeration benefits decrease, not to mention environmental effects.

The alternative is building in existing cities; redevelopment of existing housing (i.e. infill). This comes with, of course, NIMBYs. People are afraid of change, which redevelopment is by definition. Of course, there is the aspect of the middle classes not liking to see the lower classes, and a big dash of racial bias thrown in, but at its core it is conservatism in its most literal sense: preserving the past. The political problem of housing construction is that while building housing is good on a national, or even regional level, it is opposed by local government, which is run by locals who value their current environment, and are often willing to pay (in the form of high property taxes) to keep it that way. In a democracy, the more local of a government land-use planning is being done at, the more political power NIMBYs have, and the US has relatively strong local planning powers (thanks to Euclid v. Amber[2]). Additionally, most suburbs in the US are not politically part of their metropole, may not even be in the same state, are often entirely residential (+amenities), and thus have even stronger political reasons for opposing redevelopment. Compound this with the fact that increasing property prices is an easy way to retire (sell the whole thing and move to Florida with your spouse at 65) for homeowners, along with a strong homeownership culture in the US. Consider also that these suburban dwellers are often swing voters in state or federal elections, and it's easy to see why most higher levels of government chose to close their eyes to the problem and take the economic hit in return for not rocking the political boat. Plus, there is a prisoners' dilemma going on between jurisdictions. No one county, one town, one neighborhood wants to take on "more than its share" of the development, but the powers that can force a more even division do not do so.

They choose the third alternative, which is "none of the above". This results in prices increasing, homelessness increasing, etc, the current complaint about cities. It results in slowed economic growth. But that is not the fault of just cities; it is the fault of a lack of housing in a market which includes everything from the high-rise apartment to the exurb two hours out, driven by a political system that incentivizes an inefficient equilibrium.

Fundamentally, we can look at cities worldwide as choosing 0-2 of these solutions. In the greenfield column we have the sprawling cities of the Sunbelt. In the infill column we have cities like Tokyo. Many choose a bit of both. In the 0 column we have the big cities of the Anglo west: NYC, California, the big Canadian, Aussie, and NZ cities, and London.

[1] https://en.wikipedia.org/wiki/Economies_of_agglomeration

[2] https://en.wikipedia.org/wiki/Village_of_Euclid_v._Ambler_Re....