Since color images are bigger in size than grayscale images, converting a PDF to B/W will always reduce its size, especially if the document has any photos.
Keep in mind that a grayscale PDF may not render finer details and transparency correctly, so if the main reason you want to convert PDF to black and white is to reduce its size, you can do so without the need to lose color from your document. Try PDF Squeezer.
PDF Squeezer is a simple, yet powerful tool that helps quickly compress PDF files while preserving their quality and original look. Featuring a minimalistic interface, PDF Squeezer has all the necessary options to work with your files. Drag and drop your document into the app and you’ll be able to batch resize, compress folders, work with password-protected files, and shrink document size without having to remove color from a PDF.