I rarely use rem since I only stopped needing to support IE8 recently. Instead, to deal with the pain of em calculations I use these sass mixins (feel free to use; license CC0 https://wiki.creativecommons.org/wiki/CC0) (below is my first attempt at "sassdoc," which I happened to discover while preparing this comment):

$current-font-size: 16;

// Remove the units from a value (pretty sure someone else wrote this originally)
@function strip-units($number) {
@if unit($number) == '' {
@return $number;
}
@return $number / ($number * 0 + 1);
}

/// Sets font-size and line-height using percentage and unitless values respectively, with support for nesting.
/// @param {Number} $new-font-size - Desired new font size in pixels (units ignored)
/// @param {Number} $new-line-height - New line height in pixels (units ignored)
/// @param {Number} $context-font-size - Current font size where you're including this mixin, if it isn't the global $current-font-size
@mixin font-scale($new-font-size, $new-line-height:null, $context-font-size:$current-font-size) {

// Set the font size
$context-font-size: strip-units($context-font-size);
$new-font-size: strip-units($new-font-size);
$font-ratio: $new-font-size / $context-font-size;

font-size:$font-ratio * 100%;

// Set the line height
@if $new-line-height != null {
$new-line-height: strip-units($new-line-height);
// Honestly don't remember why the "+ 0.0001" here:
$line-ratio: $new-line-height / $new-font-size + 0.0001;

line-height:$line-ratio;
}

// Include nested content with global $current-font-size set to the new value
$last-cfs: $current-font-size;
$current-font-size: $new-font-size !global;

@content;

// Reset the global $current-font-size to what it was before
$current-font-size: $last-cfs !global;
}

/// Convert a pixel length to ems at the current (or given) font size
/// @param {Number} $px-length - Length in pixels (units ignored)
/// @param {Number} $font-size - Current font size in px, if it isn't the global $current-font-size (units ignored)
@function ems($px-length, $font-size:$current-font-size) {
$px-length: strip-units($px-length);
$font-size: strip-units($font-size);
@return ($px-length / $font-size)*1em;
}

(Those are modified slightly and haven't been tested in this exact form.) This lets me do, for example:

.title {

// Make the text in .title 112.5% (18 ÷ 16) with a line-height of 1.22 (22 ÷ 18)
@include font-scale(18, 22) {

// Give it 15 pixels of padding using em units
padding:ems(15);

// Make the text of a .subtitle inside a .title 14px (112.5% × (14 ÷ 18) = 87.5%)
// (line-height remains 1.22 in this case, working out to 17.11px)
.subtitle {
@include font-scale(14);

// Give .subtitle 5 pixels of padding, but instead of nesting
// the mixin again, I'll just tell ems() that the current
// font size is 14 directly.
padding:ems(5, 14);
}

}
}

The nesting isn't attractive and `ems(15)` is a little more typing than `15em` (though not more than `(15/18)×1em`), but I think just using em for everything reduces some cognitive load.

It also lets you do some tricks, like you can set the root element of some component to `font-size:(16/1024)*100vw` and it'll have a "normal" size when the viewport is 1024px wide but scale up smoothly in every dimension when the viewport is larger (and down when smaller). And you can still scale the whole page, including that component, by changing the font-size of html/body.

Thus far I only use rem when I need to override something for an inner element, like if a piece of the aforementioned vw-sized component gets too small when the viewport is less than 640, I would set the font-size for just that piece in a media query using rem, allowing the rest of the component to continue to scale down.