You support my point pretty well. You have to be exactly a Tom Clancy or Michael Crichton to attract a publisher's attention. They will print millions of copies of your book and ship it out to the public, and your book will probably be salable trash like the 2 authors you cite as examples.
Publishers lower the lowest common denominator so they can sell.
They don't want risk, in the form of something they haven't seen before. It's this bunch of people for whom models like shareware, open software, etc. Originally, they did shareware. They probably made almost nothing on the shareware, but it leveraged them into a position of becoming mainstream.