I think you're right - for efficiency, they should be constant references. 
On Monday, I'd like to chat more about passing by reference/pointer/value. It bugs me that I'm doing everything (input and output) by reference, but that seems to match all the OpenCV example code. Is there some technical reason that I'm missing for why this is a good idea? The google style guide, referenced by the roscpp style guide, suggests reference for input, and pointer for output.