MemoryPilotNet

These models contain end-to-end imitation learning-based models for control of an autonomous ego vehicle in CARLA simulator.

• GPU: NVIDIA GeForce 3090
• CUDA: 11.6
• Driver version: 510.54
• Input shape (150,50,3)/(200,66,3)

Tensorflow

• Tensorflow version: 2.7.0 / 2.11.0

Model name	Visual memory	Kinematic input	MAE test	MSE test	Model size (MB)	Filename	Previous filename
PilotNet*	✗	✗	0.0507	0.0177	20	pilotnet_no_visual_no_kinematic.h5	20230125-120238_pilotnet_CARLA_14_12_dataset_bird_eye_300_epochs_no_flip_3_output_both_directions_all_towns_adam_AFFINE_PAPER.h5
PilotNet*	✗	✓	0.0295	0.0082	20	pilotnet_no_visual_kinematic.h5	20230127-180655_pilotnet_CARLA_17_10_dataset_bird_eye_300_epochs_no_flip_3_output_velocity_all_towns_vel_30_AFFINE.h5
DeepestLSTMTinyPilotNet*	✓	✗	0.0332	0.0086	2	deepestLSTMtinypilotnet_no_visual_no_kinematic.h5	20230126-183457_deepestLSTMTinyPilotNet_CARLA_14_12_dataset_bird_eye_400_epochs_no_flip_3_output_both_directions_all_towns__less_learning_rate_PAPER_cp.h5
DeepestLSTMTinyPilotNet*	✓	✓	0.0074	0.0079	2	deepestLSTMtinypilotnet_no_visual_kinematic.h5	20230127-180856_deepestLSTMTinyPilotNet_CARLA_17_10_dataset_bird_eye_300_epochs_no_flip_3_output_velocity_all_towns_vel_30_AFFINE_PAPER.h5
PilotNetx3* (Conv3D)	✓	✗	0.0456	0.0094	127	pilotnet_conv3d_visual_no_kinematic.h5	20230222-110851_memDCCP_small_LSTM_2_CARLA_14_12_dataset_bird_eye_300_epochs_no_flip_3_output_both_directions_all_towns_sequences_more_more_extreme_AFFINE_PAPER_cp.h5
PilotNetx3* (Conv3D)	✓	✓	0.0069	0.0086	127	pilotnet_conv3d_visual_kinematic.h5	20230221-175351_memDCCP_small_CARLA_14_12_dataset_bird_eye_300_epochs_no_flip_3_output_velocity_both_directions_all_towns_sequences_more_extreme_previous_V_AFFINE_PAPER_cp.h5
PilotNetx3* (TimeDistributed)	✓	✗	0.0662	0.0196	16	pilotnet_time_distributed_visual_no_kinematic.h5	20230220-105422_pilotnet_small_more_more_LSTM_x3_CARLA_14_12_dataset_bird_eye_300_epochs_no_flip_3_output_both_directions_all_towns_sequences_more_more_extreme_t_5_t_10_dataset_AFFINE_PAPER_cp.h5
PilotNetx3* (TimeDistributed)	✓	✓	0.0289	0.0077	16	pilotnet_time_distributed_visual_kinematic.h5	20230221-162112_pilotnet_x3_small_CARLA_14_12_dataset_bird_eye_300_epochs_no_flip_3_output_velocity_both_directions_all_towns_sequences_more_extreme_previous_V_AFFINE_PAPER_cp.h5

visual_memory_density models

Model name	Visual memory	Kinematic input	MAE test	MSE test	Model size (MB)	Filename
PilotNetx3* t, t-1, t-2 (Conv3D)	✓	✗	0.036	0.0107	127	pilotnet_conv3d_visual_no_kinematic_t_t_1_t_2.h5
PilotNetx3* t, t-10, t-20 (Conv3D)	✓	✗	0.027	0.0074	127	pilotnet_conv3d_visual_no_kinematic_t_t_10_t_20.h5
PilotNetx3* t, t-20, t-40 (Conv3D)	✓	✗	0.024	0.0060	127	pilotnet_conv3d_visual_no_kinematic_t_t_20_t_40.h5
PilotNetx3* t, t-1, t-2 (TimeDistributed)	✓	✗	0.030	0.0073	16	pilotnet_time_distributed_visual_no_kinematic_t_t_1_t_2.h5
PilotNetx3* t, t-10, t-20 (TimeDistributed)	✓	✗	0.035	0.0105	16	pilotnet_time_distributed_visual_no_kinematic_t_t_10_t_20.h5
PilotNetx3* t, t-20, t-40 (TimeDistributed)	✓	✗	0.032	0.0090	16	pilotnet_time_distributed_visual_no_kinematic_t_t_20_t_40.h5

visual_memory_length models

Model name	Visual memory	Kinematic input	MAE test	MSE test	Model size (MB)	Filename
PilotNetx3* 5 frames (Conv3D)	✓	✗	0.022	0.0062	127	pilotnet_conv3d_visual_no_kinematic_5_frames.h5
PilotNetx3* 9 frames (Conv3D)	✓	✗	0.015	0.0040	246	pilotnet_conv3d_visual_no_kinematic_9_frames.h5
PilotNetx3* 5 frames (TimeDistributed)	✓	✗	0.026	0.0062	16	pilotnet_time_distributed_visual_no_kinematic_5_frames.h5
PilotNetx3* 9 frames (TimeDistributed)	✓	✗	0.020	0.0042	16	pilotnet_time_distributed_visual_no_kinematic_9_frames.h5