He was beginning to be rather ashamed of the things he had said.
He had started feeling ashamed about what he said.