It is  quite  constrained  for  us  to use  some  other  input  devices  to  communicate  with  computers. In this  paper, we combine human-computer interaction  technologies with  handwritten  Chinese character recognition strategies using depth image information provided by Kinect sensor to  realize an unconstrained handwritten character recognition system, which only uses our hand as input  device. We predefine several hand gestures as instructions, and for the recognition  of  these hand  gestures, we calculate the contour and fingertips of the hand used for writing using depth image taken  by Kinect. By mimicking the functionalities of the computer mouse only using our hands, we can  write freely and get the original character image. After Gaussian blurring and normalization, we adopt  handwritten character recognition schemes to accomplish the recognition work.