I think the first lesson here is that the term "force" is pretty unhelpful. I think the term "interaction" is generally preferred to avoid confusion.

What I would say is that to bundle gravity in with electromagnetic, strong and weak interaction as "four fundamental interactions" – without explaining the rationale for doing so – is also pretty unhelpful.

The reason the other three fundamental interactions seem distinct from gravity is that, theoretically, we can "unify" them at a high enough energy scale. That is, at an energy scale that reflects conditions within a fraction of a second after the big bang, these three interactions can be understood as one interaction, and only as energy distributes and temperature decreases do the forces become distinct.

If I am not mistaken, we have experimentally demonstrated that the electromagnetic and weak interactions unify at high enough energy, and we have theories that make predictions towards unifying the strong interaction at higher energy scales.

Gravity doesn't slot into this picture quite so neatly at all because gravity resists definition at extremely high energies, and by extension resists any effort to understand it as a component of a unified interaction.

I think it is a mistake to dismiss unified interaction as impossible because 'gravity is not a force'. Whenever anyone says this, they make an analogy to the centrifugal force and local reference frames, and that's a fair point to highlight, but I think this indicates more of a linguistic problem in the context of gravity as a fundamental interaction.

Similarly, people are quick to dismiss the graviton because it is impossible to detect and philosophically counter-intuitive – how can the effects on matter by the curvature of spacetime be mediated by a particle?

But remember: there is no particle. The graviton is a mathematical abstraction, it's not some magic ball of gravity that you can fire from a gravity gun or surround your spaceship with. A graviton is just a... I don't want to say "position" so I'll say "probability density" of spacetime at a QM scale where uncertainty is part of the business. (Someone will correct me if that is wrong).

It makes sense that we should want to use that as way to describe gravity because it would use the same framework and methods that we use to successfully describe the rest of the universe.

Of course, there are deep seated problems with gravitons, and that's another thing that sets gravity apart, but I wouldn't say that these problems arise because the notion of the graviton is in any way absurd.

Gravity is different, and it perhaps doesn't help that at human-relevant scales, Newtonian or "classical" gravity is usually sufficient for our needs in application. It lets us believe gravity is intuitive, simple, and then when we look at galaxies and subatomic particles, and we encounter consequences of GR and QM – which can be rather surprising.

But all we have to bear in mind is that at relativistic scales, the uncertainty aspect of QM is still there, it's just incredibly improbable that any quantum weirdness will occur at the macro scale. Similarly, relativistic effects are accounted for at low energy scales, but their influence is so minute that we can remove them from our equations and leave ourselves with classical solutions that are faster to reason about.

As for why gravity is different to other interactions, I think any insightful answer to that question would probably require more ground than has yet been covered in efforts to develop quantum gravity.

At the moment, the difference is technical, it's different because our tools and methods don't account for it very well as long as we're trying to fit it into our current, well-tested models of physics, and to try and fit all of our physics into a model that makes gravity more sensible is even more daunting a task.

--—
[IANAP but I am keen to learn so please please please correct me if I am wrong]