  In recent years, many learning based approaches have been studied to realize
robotic manipulation and assembly tasks, often including vision and
force/tactile feedback. However, it remains frequently unclear what is the
baseline state-of-the-art performance and what are the bottleneck problems. In
this work, we evaluate some off-the-shelf (OTS) industrial solutions on a
recently introduced benchmark, the National Institute of Standards and
Technology (NIST) Assembly Task Boards. A set of assembly tasks are introduced
and baseline methods are provided to understand their intrinsic difficulty.
Multiple sensor-based robotic solutions are then evaluated, including hybrid
force/motion control and 2D/3D pattern matching algorithms. An end-to-end
integrated solution that accomplishes the tasks is also provided. The results
and findings throughout the study reveal a few noticeable factors that impede
the adoptions of the OTS solutions: expertise dependent, limited applicability,
lack of interoperability, no scene awareness or error recovery mechanisms, and
high cost. This paper also provides a first attempt of an objective benchmark
performance on the NIST Assembly Task Boards as a reference comparison for
future works on this problem.
