[HTC One A9] djsubtronic:
With regards to skipping of music when opening apps/changing screens etc, here's what's happening in the logs. Hopefully this can help resolve the problem or at least point in the right direction. (Note I triggered the skip several times so the errors will be shown in the log several times)

03-23 14:32:40.574 4073 4073 D PhoneStatusBar: disable: < expand ICONS* alerts SYSTEM_INFO* back home recent clock search quick_settings >
03-23 14:32:40.625 7450 7465 W SearchService: Abort, client detached.
03-23 14:32:40.628 7450 8796 I DeviceStateChecker: DeviceStateChecker cancelled
03-23 14:32:40.628 7450 7450 I MicroDetector: Keeping mic open: false
03-23 14:32:40.628 7450 7450 I MicrophoneInputStream: mic_close com.google.android.apps.gsa.speech.audio.ae@2d797a7
03-23 14:32:40.629 7450 8798 E AudioRecord-JNI: Error -4 during AudioRecord native read
03-23 14:32:40.634 439 8800 D audio_hw_primary: in_set_parameters: enter: kvpairs=routing=0
03-23 14:32:40.634 439 8800 D audio_hw_primary: in_standby: enter: stream (0xf60db980) usecase(9: audio-record)
03-23 14:32:40.671 439 8800 D audio_hw_primary: disable_audio_route: reset and update mixer path: audio-record
03-23 14:32:40.673 439 8800 D hardware_info: hw_info_append_hw_type : device_name = headset
03-23 14:32:40.673 439 8800 D audio_hw_primary: disable_snd_device: snd_device(63: headset)
03-23 14:32:40.678 439 3897 D AudioPolicyManagerCustom: stopInput() setParameters rec_play_conc is setting to OFF
03-23 14:32:40.678 439 1804 D audio_hw_primary: adev_set_parameters: enter: rec_play_conc_on=false
03-23 14:32:40.678 439 1804 D msm8916_platform: setting record playback concurrency to false
03-23 14:32:40.678 439 1804 E bt_a2dp_hw: adev_set_parameters: ERROR: set param called even when stream out is null
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 1
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 2
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 3
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 4
03-23 14:32:40.679 8193 8210 W AudioTrack: dead IAudioTrack, PCM, creating a new one from obtainBuffer()
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 5
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 6
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 8
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 9
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 10
03-23 14:32:40.679 439 3897 D AudioPolicyManagerCustom: Invalidate on stopInput for stream :: 11
03-23 14:32:40.687 7450 7656 I MicroRecognitionRunner: Stopping hotword detection.
03-23 14:32:40.687 7450 8797 I MicroRecognitionRunner: Detection finished
03-23 14:32:40.689 439 439 D audio_hw_primary: adev_close_input_stream: enter:stream_handle(0xf60db980)
03-23 14:32:40.689 439 439 D audio_hw_primary: in_standby: enter: stream (0xf60db980) usecase(9: audio-record)
03-23 14:32:41.414 2178 4967 W InputMethodManagerService: Window already focused, ignoring focus gain of: com.android.internal.view.IInputMethodClient$Stub$Proxy@5535508 attribute=null, token = android.os.BinderProxy@43a53d2
03-23 14:32:41.418 7450 7450 I MicroDetectionWorker: Micro detection mode: [mDetectionMode: [1]].
03-23 14:32:41.451 4073 4073 D PhoneStatusBar: disable: < expand icons* alerts system_info* back home recent clock search quick_settings >
03-23 14:32:41.461 7450 8806 I MicroRecognitionRunner: Starting detection.
03-23 14:32:41.463 7450 8807 I MicrophoneInputStream: mic_starting com.google.android.apps.gsa.speech.audio.ae@c2dd49f
03-23 14:32:41.464 439 3897 D audio_hw_primary: adev_open_input_stream: enter: sample_rate(16000) channel_mask(0x10) devices(0x80000010) stream_handle(0xf06a1cc0) io_handle(73) source(6)
03-23 14:32:41.465 439 8809 I AudioFlinger: AudioFlinger's thread 0xf2780000 ready to run
03-23 14:32:41.466 439 8809 D audio_hw_primary: in_standby: enter: stream (0xf06a1cc0) usecase(9: audio-record)
03-23 14:32:41.469 439 8809 D audio_hw_primary: in_standby: enter: stream (0xf06a1cc0) usecase(9: audio-record)
03-23 14:32:41.473 439 3897 D AudioPolicyManagerCustom: startInput() setParameters rec_play_conc is setting to ON
03-23 14:32:41.474 439 1804 D audio_hw_primary: adev_set_parameters: enter: rec_play_conc_on=true
03-23 14:32:41.474 439 1804 D msm8916_platform: setting record playback concurrency to true
03-23 14:32:41.474 439 1804 E bt_a2dp_hw: adev_set_parameters: ERROR: set param called even when stream out is null
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 1
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 2
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 3
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 4
03-23 14:32:41.474 8193 8210 W AudioTrack: dead IAudioTrack, PCM, creating a new one from obtainBuffer()
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 5
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 6
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 8
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 9
03-23 14:32:41.474 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 10
03-23 14:32:41.475 439 3897 D AudioPolicyManagerCustom: Invalidate on releaseInput for stream :: 11
03-23 14:32:41.475 439 8809 D audio_hw_primary: in_set_parameters: enter: kvpairs=input_source=6;routing=-2147483632
03-23 14:32:41.476 8193 8205 W AudioSystem: ioConfigChanged() modifying unknown output! 73
03-23 14:32:41.478 439 1874 D AudioPolicyManagerCustom: voice_conc:Record mode is on forcing deep buffer output for non ULL... flags: 0
03-23 14:32:41.478 7450 8807 I MicrophoneInputStream: mic_started com.google.android.apps.gsa.speech.audio.ae@c2dd49f
03-23 14:32:41.486 439 8809 D audio_hw_primary: start_input_stream: enter: stream(0xf06a1cc0)usecase(9: audio-record)
03-23 14:32:41.487 439 8809 D audio_hw_primary: select_devices: out_snd_device(0: ) in_snd_device(63: headset)
03-23 14:32:41.487 439 8809 D hardware_info: hw_info_append_hw_type : device_name = headset
03-23 14:32:41.487 439 8809 D audio_hw_primary: enable_snd_device: snd_device(63: headset)
03-23 14:32:41.492 439 8809 E audio_hw_dolby: audio_extn_dolby_ds2_set_endpoint: dap_hal_set_hw_info is NULL
03-23 14:32:41.492 439 8809 D msm8916_platform: Resetting app type for Tx path to default
03-23 14:32:41.493 439 8809 D audio_hw_primary: enable_audio_route: apply mixer and update path: audio-record
03-23 14:32:41.494 439 8809 D audio_hw_primary: select_devices: done
03-23 14:32:41.550 439 8809 D audio_hw_primary: start_input_stream: exit
03-23 14:32:41.568 7450 7450 I MicroDetectionWorker: onReady

Edit: completely disable "Ok Google" detection as a workaround for this issue. It looks like Google polls the microphone at every window change, even if you set it to listen in the app only, and this is what causes the stutter.