A 3D surface measurement system (10, 400, 500, 700, 800, 3000) comprising: an engineered illumination systen (30, 430, 530, 3130) configured to provide at least one ray bundle sequentially at N illumination directions incident a surface of a sample being imaged, wherein each ray bundle comprises illumination beams of various intensities, the illumination beams of each ray bundle configured to be at different incidence angles to a surface point at the surface of the sample; a first camera (60, 460, 660, 760, 860, 3060, 3160) with an imaging lens and at least one sensor configured to acquire intensity images based on light received from the illuminated sample, wherein the at least one sensor comprises a plurality of sensor pixels, each intensity image acquired while the at least one ray bundle is directed at one of the N illumination directions; and a controller configured to execute instructions to: determine a captured sensor response at each sensor pixel from the acquired intensity images at the N illumination directions; calculate pre-determined sensor responses for different surface normals based on a known distribution of the engineered illumination at the different illumination directions; match the captured sensor response at each sensor pixel to one of the plurality of predetermined sensor responses to determine a surface normal at each sensor pixel; and construct a map of surface normals of the surface of the sample by combining the determined surface normals of all the sensor pixels of the at least one sensor. an engineered illumination systen (30, 430, 530, 3130) configured to provide at least one ray bundle sequentially at N illumination directions incident a surface of a sample being imaged, wherein each ray bundle comprises illumination beams of various intensities, the illumination beams of each ray bundle configured to be at different incidence angles to a surface point at the surface of the sample; a first camera (60, 460, 660, 760, 860, 3060, 3160) with an imaging lens and at least one sensor configured to acquire intensity images based on light received from the illuminated sample, wherein the at least one sensor comprises a plurality of sensor pixels, each intensity image acquired while the at least one ray bundle is directed at one of the N illumination directions; and a controller configured to execute instructions to: determine a captured sensor response at each sensor pixel from the acquired intensity images at the N illumination directions; calculate pre-determined sensor responses for different surface normals based on a known distribution of the engineered illumination at the different illumination directions; match the captured sensor response at each sensor pixel to one of the plurality of predetermined sensor responses to determine a surface normal at each sensor pixel; and construct a map of surface normals of the surface of the sample by combining the determined surface normals of all the sensor pixels of the at least one sensor. determine a captured sensor response at each sensor pixel from the acquired intensity images at the N illumination directions; calculate pre-determined sensor responses for different surface normals based on a known distribution of the engineered illumination at the different illumination directions; match the captured sensor response at each sensor pixel to one of the plurality of predetermined sensor responses to determine a surface normal at each sensor pixel; and construct a map of surface normals of the surface of the sample by combining the determined surface normals of all the sensor pixels of the at least one sensor.