When I used to clean my dad's floors at his office, I used straight ammonia with a little water. Let it sit a few minutes. The dirt will come out. Be sure and use plenty of ventilation. Your cleaning dilemma is one that faces many individuals, and you do not have to have an old floor to have a problem.
Many people find that keeping the newer floors clean is also difficult.
It seems as though you have used a battery of cleaners with little success. Sometimes the cleaning method in itself will be the downfall of a cleaner looking floor. If ample rinsing is not done, even in some of the cleaners that suggest no rinsing is necessary, it will attract dirt to the flooring. You are correct, the dirt particles appear to be lodge in the fabric of the linoleum. Dirt and dust will etch the floor.