I know many other people have. It won't be easy, that's for sure, but it can be done. Be aware that living in China will be stressful for most kids. In fact, that was just his way of dealing with the attention and pressure. If your children start showing unusual or undesirable behavior, it could be their way of dealing with the stress and attention.
Some kids spit, hit, or just ignore the people.
Our son would chew on his sleeve to deal with the stress. Some deal with it fine. Your children may or may not be accepted by Chinese children. If they are school-aged they may have more problems being accepted. Chinese are very concious of differences in skin color.