serial: dont restore cflag

Change-Id: I5ff91993f0cbb8afea244d52dea25d3df359fc0b