Only set stem direction for notes that have stems.