Q:

Fluid Typography Linear Transition: How is the calc()-expression computed? 

Using the approach described by e.g. Mike Riethmuller of using calc() to achieve a fluid transition from a minimum font-size to a maximum font-size, the result returned from calc() and displayed in the browser is somewhat puzzling to me (and not as intended).  How does the following code return this result for h1 at a viewport size from 25rem up to 80rem? ( I've included the calc() for the html font-size as well, since it also scales with h1 proportionate to it)
Result:

h1 calc(0.450091rem + 3.38364vw)

Code:
html {
  font-size: 1.125rem;
}

@media screen and (min-width: 25rem) {
  html {
    font-size: calc(1.125rem + 0.25 * ((100vw - 25rem) / 55));
  }
}

@media screen and (min-width: 80rem) {
  html {
    font-size: 1.375rem;
  }
}

h1 {
  font-size: 1.296rem;
}

@media screen and (min-width: 25rem) {
  h1 {
    font-size: calc(1.296rem + 1.861 * ((100vw - 25rem) / 55));
  }
}

@media screen and (min-width: 80rem) {
  h1 {
    font-size: 3.157rem;
  }
}

The result is a "jumpy" non-fluid transition as we enter the viewport min-width of 80rem and reach the max font-size of 3.157rem. 

A:

Since you are changing the font-size of the html element you are then changing the rem value.  It's somehow tricky but let's follow the code and the calculation step by step. 
Initially we will have  a font-size equal to 1.125rem within html and we will have a font-size for h1 equal to 1.125 * 1.296 * 16px = 23.328px.  For the media query we will have 25rem = 25*16px = 400px and 80rem = 80*16px = 1280px1. 
After 400px (the first media query), it become tricky because we will have dynamic values of font-size with html thus the font-size for h1 will be something like:
 (1.296* P *16px + 1.861 * ((100vw - (25 * P * 16px) ) / 55))

Where 
 P = (1.125 * 16px + 0.25 * ((100vw - 25 * 16px) / 55))

Now when we reach the next media query (1280px), we will have 100vw equal to 1280px (not 80rem like we may think!)  and P = 1.375rem so the result considering the forumla before 1280px will be 1.296 * 1.375 * 16px + 1.861 *((1280px - 25 * 1.375 * 16px)/55) = 28,512px + 24.7px = 53.21px.  
And the result considering the formula after 1280px will be 3.157 * 1.375 * 16px = 69.454px.  This explain why you don't have the transition and the culprit was the 100vw that is equal to 1280px and not 80rem so you want have the intendend result of (80 - 25)/55 = 1
Following the formula you found we will have the same result:

h1 calc(0.450091rem + 3.38364vw)

when at 80rem 100vw will be 1280px and 3.38364vw will be 43.13px and 0.45rem will be 0.45 * 1.375 * 16px = 9.9px and the total 53.03px. 
The result is also logical because calc will reduce the formula to group all the same unit together:
calc(1.296rem + 1.861 * ((100vw - 25rem) / 55))
calc(1.296rem + 1.861 * (1.8181vw - 0.4545rem)
calc(1.296rem + 3.3834vw - 0.8459rem)
calc(0.4501rem + 3.3834vw)

1Why in the media query we don't multiply with the value used in html? 
Considering the below calculation, we may think that within the media query we should also consider the same logic thus 25rem and 80rem should also be dynamic.  This is not the case for media query as detailed in the specification:

Relative length units in media queries are based on the initial value, which means that units are never based on results of declarations. 

So whataver the the value of font-size used within html is, we will always consider the intial value of 16px for 1rem inside media query. 

There will be some slight difference between the values obtained using different method due to some rounding.  The purpose is not to provide exact calculation but to explain the calculation.