Chances are they'll say yes.
Creating a logo for both Web and print can be a challenging task. But failing to ask this question up front could mean producing the logo all over again. You should also tell the client about the extra costs involved for digital production. Otherwise, you may end up designing a logo that's so costly to print your new client will be filing Chapter 13 before they even start. Knowing the differences between designing for the Web and creating for print will help you to walk the client smoothly through the digital production process.