An autonomous driving system (100; 200; 300; 400; 500) that performs autonomous driving of a vehicle, the autonomous driving system comprising: a vehicle position acquisition device configured to acquire a position of the vehicle on a map; a vehicle state acquisition device configured to acquire a vehicle state of the vehicle, said vehicle state comprising vehicle speed, acceleration, yaw rate of the vehicle and opening and closing of a door of the vehicle; an occupant state acquisition device configured to acquire an occupant state of the vehicle, said occupant state comprising the number of occupants, a sitting position of the occupant and a pressure applied to a seat of the vehicle due to an occupant sitting on the seat; a destination candidate database configured to store destination candidate data including a destination candidate and the occupant state correlated with the destination candidate; and an electronic control unit (10; 30; 40; 50; 60) configured to determine whether or not the vehicle is in a boarding completion state based on the vehicle state and the occupant state, wherein the vehicle is in a boarding completion sate in a case where a door of the vehicle is opened and closed, and then a weight is detected in a seat of the vehicle, and set a destination of the vehicle in autonomous driving based on the position of the vehicle on the map, the occupant state, and the destination candidate data in a case where electronic control unit (10; 30; 40; 50; 60) determines that the vehicle is in the boarding completion state. a vehicle position acquisition device configured to acquire a position of the vehicle on a map; a vehicle state acquisition device configured to acquire a vehicle state of the vehicle, said vehicle state comprising vehicle speed, acceleration, yaw rate of the vehicle and opening and closing of a door of the vehicle; an occupant state acquisition device configured to acquire an occupant state of the vehicle, said occupant state comprising the number of occupants, a sitting position of the occupant and a pressure applied to a seat of the vehicle due to an occupant sitting on the seat; a destination candidate database configured to store destination candidate data including a destination candidate and the occupant state correlated with the destination candidate; and an electronic control unit (10; 30; 40; 50; 60) configured to determine whether or not the vehicle is in a boarding completion state based on the vehicle state and the occupant state, wherein the vehicle is in a boarding completion sate in a case where a door of the vehicle is opened and closed, and then a weight is detected in a seat of the vehicle, and set a destination of the vehicle in autonomous driving based on the position of the vehicle on the map, the occupant state, and the destination candidate data in a case where electronic control unit (10; 30; 40; 50; 60) determines that the vehicle is in the boarding completion state. determine whether or not the vehicle is in a boarding completion state based on the vehicle state and the occupant state, wherein the vehicle is in a boarding completion sate in a case where a door of the vehicle is opened and closed, and then a weight is detected in a seat of the vehicle, and set a destination of the vehicle in autonomous driving based on the position of the vehicle on the map, the occupant state, and the destination candidate data in a case where electronic control unit (10; 30; 40; 50; 60) determines that the vehicle is in the boarding completion state.