This article does a good job of covering the basics of bidding, but most of these things should be obvious. In performance marketing, the goal is to know how much money you are going to make for a given visitor before that visitor enters your site. Then, you can use any number of bidding models to reach maximum profit.

What is really lacking here, is that Adwords is not just a basic auction system. If I have a Max CPC of $1.00, and you have a Max CPC of $0.85, it is not certain that I am going to be at a higher ad space than you.

There are a number of variables that Google takes in to account to determine the order of placements in an auction, and it obfuscates all these variables in to something called a 'Quality Score', which publicly operates on a integer scale of 1-10. Google uses the quality score to manipulate auctions to perform better for google, not the advertiser. Here's some of the most important factors:

CTR: If you're holding down position one, but nobody is clicking your ad, google is not making money. Just like you as a performance marketer, Google is not trying to get the maximum profit per individual click. They are trying to get the maximum volume of profitable clicks. One key here is to not just dip your toe in to a keyword. If you overbid (usually to a loss) out of the gate, you will get substantial volume faster, thus giving you a better CTR and a higher quality score. Once you have solidified your position in the auction, you can then ratchet down your bid to a profitable level.

Page Relevancy: It is important that your landing page is relevant to both the keyword you are bidding on, as well as your ad copy that you displayed. Google AdsBot will index your landing page using algorithms similar to it's organic bot. Page load time is a major factor here as well. This is google playing the long tail. They want to make sure that whoever clicked on the ad is happy with the experience they get on the landing page, so that they will in the future click on more ads. If a user keeps ending up on junk pages, they are less likely to click on ads or use google in the future.

Account History: This is in my opinion, one of the most frustrating parts of adwords, and it's impact seems to vary wildly from vertical to vertical. The age of your account and ads can play a major role in how much you are paying. How much of an impact? I've seen ads that are 3-4 years old cost $1-2/click, and an entirely new account at a new company cost upwards of $10-15/click on the same keyword. The more volatile the space, the bigger the disparity is. There is very little information out there about how/why this is, but here is my theory: The majority of people using adwords at scale are using it entirely for performance marketing. As such, if you get knocked out of a top position on a high volume keyword, it can have a significant effect to your bottom line. If a company has consistently been on a keyword spending money every day for 2-3 years, you would assume that they are going to stay there if they keep their position. If you allow a newcomer to come in and spend to a loss for a while, disrupting the positions of established players, there is not a guarantee that the established player will ever recover on that keyword. And since this new player has been spending so much, they can't afford to keep operating and have closed up shop. The net effect is less money for google. Again, this comes more in to play in volatile markets, but is straight up market manipulation.

----

As far as questions regarding technology to optimize SEM is concerned, Google has mostly prevented this from happening by having one of the worst API's I have ever worked with. They will make marginal changes and sunset the old version after only a couple months. Many of their practices seem somewhat openly hostile to developers working with their API. While being good performance marketers, data is our friend. We want as much of it as we can get. Google, however, wants us to only have just enough information to keep spending money. It is not their goal for you to have a 100% optimized campaign. They know there are junk queries that are worth less, and want you to keep blending those in with those that are performing, so that they can make more money.

Anybody that's curious about their tech should take a look in to what it takes to build something that uses the Adwords API. The fact that they can have such an awful API and tools for something that makes 70%+ of their revenue is amazing.