A tracking vehicle control system for controlling a tracking vehicle that tracks an operator as a tracking object, the tracking vehicle control system comprising: a drive section configured to drive the tracking vehicle; a drive control section configured to control the drive section; a recognition section configured to recognize the tracking object; a first judgement section configured to judge whether the tracking object has started access to the tracking vehicle; and a second judgement section configured to judge whether the tracking object has ended access to the tracking vehicle, characterized in that after the drive control section controls the drive section to get the tracking vehicle to track the tracking object according to a result of recognition of the tracking object performed by the recognition section, the drive control section controls the drive section to stop the tracking vehicle from tracking the tracking object when the first judgement section judges that the tracking object has started access to the tracking vehicle, and the drive control section controls the drive section to get the tracking vehicle to restart tracking the tracking object when the second judgement section judges that the tracking object has ended access to the tracking vehicle, the recognition section recognizes the tracking object even when tracking of the tracking object by the tracking vehicle is suspended. the first judgement section judges, according to a result of recognition of the tracking object performed by the recognition section, whether a distance between the tracking vehicle and the tracking object has been equal to or less than a specified value for suspension of tracking for a predetermined continuous time, the first judgement section judges that the tracking object has started access to the tracking vehicle when the first judgement section judges that the distance between the tracking vehicle and the tracking object has been equal to or less than the specified value for the predetermined continuous time, the second judgement section judges whether a specified time has passed since the tracking vehicle stopped tracking the tracking object, and the second judgement section judges that the tracking object has ended access to the tracking vehicle, when the second judgement section judges that the specified time has passed since the tracking vehicle stopped tracking the tracking object. a drive section configured to drive the tracking vehicle; a drive control section configured to control the drive section; a recognition section configured to recognize the tracking object; a first judgement section configured to judge whether the tracking object has started access to the tracking vehicle; and a second judgement section configured to judge whether the tracking object has ended access to the tracking vehicle, characterized in that after the drive control section controls the drive section to get the tracking vehicle to track the tracking object according to a result of recognition of the tracking object performed by the recognition section, the drive control section controls the drive section to stop the tracking vehicle from tracking the tracking object when the first judgement section judges that the tracking object has started access to the tracking vehicle, and the drive control section controls the drive section to get the tracking vehicle to restart tracking the tracking object when the second judgement section judges that the tracking object has ended access to the tracking vehicle, the recognition section recognizes the tracking object even when tracking of the tracking object by the tracking vehicle is suspended. the first judgement section judges, according to a result of recognition of the tracking object performed by the recognition section, whether a distance between the tracking vehicle and the tracking object has been equal to or less than a specified value for suspension of tracking for a predetermined continuous time, the first judgement section judges that the tracking object has started access to the tracking vehicle when the first judgement section judges that the distance between the tracking vehicle and the tracking object has been equal to or less than the specified value for the predetermined continuous time, the second judgement section judges whether a specified time has passed since the tracking vehicle stopped tracking the tracking object, and the second judgement section judges that the tracking object has ended access to the tracking vehicle, when the second judgement section judges that the specified time has passed since the tracking vehicle stopped tracking the tracking object.