I now talk to the three channels of the receiver that I'm interested in. This currently takes about 142ms to read all three channels which is completely unacceptable except... I'm going to accept for now and see if this will work regardless.

I also integrated the IMU into the project. Luckily because I2C is a synchronous protocol I can do all the processing I need in right around 2ms.