SPARK-3968: fix scala code style and add some more tests for filtering on optional columns