Well, I oversimplified a bit, it's actually a bit more complicated than either description in terms of the incremental costs. The theoretical marginal cost of adding an additional Shuttle flight to the manifest is around $110 mil, but that doesn't include certain necessary operational costs (astronaut training and handling, etc.). It's a difficult thing to estimate accurately and sort of moot regardless because of the extreme limitations due to the small size of the Orbiter fleet and the amount of time it takes to refurbish them after each launch.

As far as the average annual cost you have to keep in mind that the total amount of money spent on Shuttle operations was more than just the spending on the Shuttle's standing army of maintenance workers. There was additional Shuttle related spending that in total added up to a figure of about a billion dollars per flight. Also keep in mind that near the end of the program the flight rate was quite low, and was often only 3 or 4 a year. Additionally, the Shuttle expenditures were higher back in the mid '80s than in the 2000s, when adjusted for inflation, so even though the flight rate was higher the per launch cost was similar (the best time being in 1985 when they managed 9 flights with a budget of $5.6 bil (in 2010 dollars)). Up through the 80s and 90s there were a few good years where the cost per flight was as low as $600 mil, but in mediocre year that cost could be as much as $1.2 bil, and in a bad couple of years that cost could be as much as $5 bil or even $7 bil (in the 3 year periods just after the Challenger and Columbia disasters, for example).

For SpaceX, much of what keeps their costs low is just bog standard common sense. The biggest problem in spaceflight over the last 4 decades has been a lack of common sense.

Consider a few salient points. The more components you add to a vehicle the less reliable and more costly it is to operate. One big problem that typical launch vehicles have had is being undersized. And this comes because nobody thinks to build any margin into the design. During the Apollo program Von Braun sandbagged the Saturn V's quoted payload capability, knowing that the Apollo spacecraft would go over its mass budget. Instead of having to go back to the drawing board, everything was fine because the Saturn V was scaled to take into account that overrun. These are the sorts of things that we know are common. But what has happened with so many vehicles is that they end up adding on solid fueled boosters to the vehicle to increase the payload capacity. Well, this dramatically changes the engineering of the whole beast, and makes the vehicle less safe and more expensive in the processes. It's better to scale the vehicle right from the get go and avoid using solid rockets at all. If you look at the Falcon 9 you can see where that has been a huge advantage to them, allowing them to do an on pad abort and a fast recycle without endangering the mission even when there was a problem with the rocket. And it also makes range safety a lot easier.

Then you have the fascination with using LOX/LH2 in lower stages. The earliest orbital launchers typically used LOX and Kerosene. This is a very straightforward design with a lot of advantages because the high density of Kerosene and the relatively mild cryogenic character of LOX allows you to achieve very high mass ratios on lower stages using little more than ordinary airplane grade engineering and a modicum of thermal shielding. LOX/LH2 has a higher Isp though, and it produces stages with a lower total weight so you can create a launcher with a lower "GLOW" (gross liftoff weight). Each of these factors has been enormously enticing to launch vehicle designers for decades and has got a hold of their minds regardless of common sense. The downsides of LH2 in a lower stage are extreme. It is super-cryogenic so it requires a lot of specialty construction and extra insulation in the fuel tanks. Moreover, it is very low density so it requires using exotic materials (such as Al-Li alloys) to achieve reasonable mass ratios. The thing is, rocket performance scales exponentially with respect to both mass ratio and Isp, so it's important to make a smart tradeoff between them. As far as GLOW, it's just a number, and it doesn't mean a whole lot since most of it is just propellant which is cheap compared to the total cost of the vehicle. If you were making a very low cost vehicle then GLOW might be a concern, but we are about 2 orders of magnitude away from that realm so focusing on it is actually an extreme anti-optimization. Worse yet, LOX/LH2 tends to generate less thrust than LOX/Kerosene for similar rocket engine sizes, so often (as is the case with Ariane V, the Shuttle, the Japanese H-IIA, and others) LH2 fueled rockets rely on the extra thrust of solid boosters to get the vehicle off the ground. This adds complexity, cost, and reduces reliability and safety.

Now, there is also the annoying idea that "simple" rocket designs such as an ordinary 2 stage LOX/Kerosene booster are old hat, they've been done, they're outmoded technology. New rockets need to make use of more cutting edge technology such as reusable space planes, or linear aerospike engines, or SSTOs, or helicopter/rocket hybrids. The fallacy here is that we hadn't explored simple rocket designs nearly enough. There's nothing wrong with taking a proven, traditional design and making it even simpler, more robust, more reliable, and reducing cost.

And that's exactly what SpaceX has been doing. Their rocket design is not the smartest or most innovative launch vehicle in history, it's actually closest to the dumbest possible idea that's workable. Why not build a 2-stage LOX/Kerosene rocket and put a capsule on top of it? It's not a new idea, it's not innovative, it's not high-tech. It's a design that would be intimately familiar to anyone who had worked on the Saturn IB rocket or the Gemini capsule or the Soyuz rocket / spacecraft going back nearly 50 years ago. Granted, they've put a lot of high-tech work into the vehicle, but it is ultimately just a very common sense design. The only reason nobody else has done the same sort of thing is because they haven't had the guts.

Once you start from there, with a foundation of making sound design and business decisions based on good, well rounded common sense then the benefits to the whole system flow out from there. Then you can avoid red tape and bureaucracy and make the sensible decision based on the full truth at each step, which tends to have a multiplicative effect. The end result being a company like SpaceX, which even when it does little more than launch boring rocket designs into orbit nevertheless revolutionizes the entire industry, because no one else had thought that boring could be so smart.