Fix some sign conversion warnings

Unfortunately, there are about 1000+ warnings of -Wsign-conversion in this
project. The task is not so trivial. We need to be very careful for fixing
the warnings without regressions nor memory errors.