An autonomous driving system comprising: a driver information acquisition device configured to acquire driver information indicating an action and a state of a driver of a vehicle; and a control device configured to perform autonomous driving control that controls autonomous driving of the vehicle, wherein the autonomous driving control includes: deactivating action detection processing that detects, based on the driver information, a deactivating action of the driver to deactivate the autonomous driving; ready state detection processing that detects, based on the driver information, a ready state indicating that the driver is ready for manual driving, after the deactivating action is detected; deactivation standby control that repeatedly executes the ready state detection processing without deactivating the autonomous driving, until the ready state is detected after the deactivating action is detected; and deactivation processing that deactivates the autonomous driving when the ready state is detected after the deactivating action is detected; characterized in that in the deactivation standby control, the control device repeatedly executes the ready state detection processing without executing the deactivation action detection processing. a driver information acquisition device configured to acquire driver information indicating an action and a state of a driver of a vehicle; and a control device configured to perform autonomous driving control that controls autonomous driving of the vehicle, wherein the autonomous driving control includes: deactivating action detection processing that detects, based on the driver information, a deactivating action of the driver to deactivate the autonomous driving; ready state detection processing that detects, based on the driver information, a ready state indicating that the driver is ready for manual driving, after the deactivating action is detected; deactivation standby control that repeatedly executes the ready state detection processing without deactivating the autonomous driving, until the ready state is detected after the deactivating action is detected; and deactivation processing that deactivates the autonomous driving when the ready state is detected after the deactivating action is detected; characterized in that in the deactivation standby control, the control device repeatedly executes the ready state detection processing without executing the deactivation action detection processing. deactivating action detection processing that detects, based on the driver information, a deactivating action of the driver to deactivate the autonomous driving; ready state detection processing that detects, based on the driver information, a ready state indicating that the driver is ready for manual driving, after the deactivating action is detected; deactivation standby control that repeatedly executes the ready state detection processing without deactivating the autonomous driving, until the ready state is detected after the deactivating action is detected; and deactivation processing that deactivates the autonomous driving when the ready state is detected after the deactivating action is detected; characterized in that in the deactivation standby control, the control device repeatedly executes the ready state detection processing without executing the deactivation action detection processing.