[One (M7) General] bgdwiepp:
Actually, it depends highly upon the specifics of the battery, including specific cell chemistry, the dimensions of the electrodes, it's internal resistance, it's thermal conductivity, it's gross capacity and much much more. Almost all of these things you, or many other people have no specifications on so you can't make any guarantees about. For small Lithium chemistry cells, between 50mah and 2500mah, if not specifically designed for high discharge and charge currents, should be charged at a maximum of 0.5C for optimal lifespan and safety, simply due to thermal conductivity and electrode size. larger capacity cells generally have correspondingly larger electrodes thus can handle larger charge currents, in the order of 1C.

Actually, the issues to do with charging speed/temperature is evaporation of electrolyte, which is due to the peak temperatures, as the electrolyte boils off, it cannot redissolve and shortens the life of the cell. Whilst prolonged high temperatures are not beneficial to the cell, this is due to a different issue, the irreversible completion of chemical reactions.

Rate is not really adjusted on the fly, initially, the battery is charged at a fixed rate, eg, the rate set by the phone depending on USB or AC adaptor detection (and kernel). Once the battery is charged to it's maximum voltage, the charge mode is changed to constant voltage, this happens at around 80% charge, from this point onwards, no matter what the charge rate is set at, the charge controller will provide only the amount of current it is able to without increasing the voltage of the cell, until the cell stops accepting any more current.

There is a hard limit on the rate the battery can charge, that is the maximum rate of the charge controller, and in devices like this, it is around 1.5A, larger devices may be able to do more, because they are able to dissipate the extra heat the charge controller is specifically specified for a higher current rating - BUT that doesn't mean they will charge faster, because generally the batteries are correspondingly larger so the speed stays the same but the capacity increases.

Another issue with charging of the batteries is excess power usage by the charge controller, as you increase the charge current during the first stage of charging (which is the only stage you can increase charge speed of), the controller will have to dissipate more heat, you see, the way the first portion of the charge is done is that a fixed current is supplied to the battery (constant current/CC), to maintain this fixed current, the charge controller must limit the amount of voltage provided to the cell, and how does it limit this current? it burns it off as heat. the amount of energy it has to burn off is calculated at (Vbatt-Vusb)*ChargeCurrent, as the current goes up, so does the heat.

Whilst there is some legitimacy of the above "bigger battery takes longer to charge", the time is not directly proportional to the capacity divided by the charge rate, because the charging is done in two stages described above (CC/CV), say you have a "normal" charge rate, stage 1 takes 90mins and stage 2 takes 30mins, your battery is charged in 2 hours. Now you increase that to a fast charge, 2x the charge current/rate, stage 1 takes 45mins, stage 2 takes 30 mins, your battery is charged in 1 hour 15 mins, not a direct half in charge time.

The situation you suggest is a common reality; and can be seen in a similar light to the above described, but in reverse.

Say you're charging your phone in your computer, abiding by the USB spec to not draw more than 500ma (which all devices should) and your phone is using 400ma to do it's day to day stuff, you have 100ma left over to charge your phone, it will take 23 hours to charge. Increase that current available to the phone to 1A, double what it had before, and now, instead of it taking 23 hours to charge, it takes less than 4 hours!

TL:DR

Batteries are complicated and there is more you don't know that you do know, trust the engineers at the phone companies because hardware is a whole different ballgame to software - Internet Explorer may crash, but it won't set your pants on fire like a flaming battery.