Dark matter is more complicated than that. It would be nice if it were as simple as you claim it is, but it isn't.

From the '30s through to the '70s, evidence was piling up that what we saw and what our models predict were incompatible. Zwicky's application of the virial theorem to a galaxy cluster, and various astronomer's calculations of galaxy rotation curves implied that either there was a bunch of stuff we couldn't see, or general relativity was wrong, or both. MOND was born in this era to explain that general relativity was wrong. (note when I say "wrong" I mean in the same way F = Gmm/r^2 is wrong: it's correct in the limit, but wrong in the extremes. Newtonian gravity is wrong at very high acceleration, and MOND implies General Relativity is wrong at very low acceleration) If science had stopped in 1985, you'd be correct: we couldn't tell the difference between dark matter as particles, (CDM, cold dark matter) dark matter as heavy dark objects, (MACHOs: brown dwarfs, black holes) or dark matter as a new gravity model. (MOND: modified Newtonian dynamics)

But science did not stop in 1985.

First and most obvious is gravitational lensing and the bullet cluster. This is well trodden ground, so I won't get too much into it: but the bullet cluster shows us that whatever dark matter is, it has momentum. Some MOND theories do predict something like that, and are compatible with the bullet cluster, other models are not compatible with that, and are falsified by the bullet cluster.

Second is baryon acoustic oscillations. (BAOs) In the few minutes after the Big Bang, the universe was, to a first approximation, a roiling sea of photons. There were electrons, protons, and the odd helium/lithium nucleus, but because charged particles interact via the electromagnetic force, they were being tossed about on the sea of photons. Baryons could not form overdensities because they were charged, and if anything thought about clumping up, the photons would scatter them. But baryons were not the only objects with mass: there was also dark matter. Dark matter could form clumps, and clumps formed by clumping dark matter would be able to clump normal matter. Eventually, the universe cooled enough that normal matter could clump properly, and at the moment the universe cooled enough to be transparent, the cosmic microwave background (CMB) was born. In order for the CMB to look the way it does, there are very tight bounds on how dark matter has to behave, and wouldn't ya know it, these bounds are compatible with the bounds on CDM in order to explain galaxy clusters and galactic rotation curves.

Third is the detection of ultra diffuse galaxies. These galaxies are remnants from a galaxy merger, which spilled off some of its normal matter but none of its dark matter, creating a galaxy with negligible dark matter. In other cases, these collisions create massive blobs of dark matter, but with little to no normal matter. These galaxies falsify MOND. For MOND to be correct, these galaxies cannot exist.

Forth is LIGO/VIRGO and the neutron star-neutron star collision a few years ago and the associated gamma ray burst. Many MOND theories predict that gravity travels slower than light. However, GW170817 shows that gravity travels at the speed of light. Some MOND theories are compatible with this, others are not.

In general, theories of MOND comes in two flavors: those that are compatible with the bullet cluster, and those that are compatible with GW170817. None of them, AFAIK, are compatible with both.

So if you want a MOND theory with no CDM, that's fine, but you have a number of hurdles to jump. You need to create a theoretical framework which is compatible with both the bullet cluster and GW170817, which nobody's been able to do. You need to show that ultra diffuse galaxies are a sensor or interpretation error; those galaxies are significantly closer or farther than currently believed. You need to come up with an entirely new mechanism that explains BAOs. It's not impossible, it is just extraordinarily difficult.