Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems  . In this position paper, we present the current landscape of TinyML and discuss the challenges and direction towards developing a fair and useful hardware benchmark for TinyML workloads  . Our viewpoints reflect the collective thoughts of the TinyMLPerf working group that is comprised of URLanizations.
Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems and is therefore fundamental to a field reaching maturity . In this position paper, we present the current landscape of TinyML and discuss the challenges and direction towards developing a fair and useful hardware benchmark for TinyML workloads . Furthermore, we present our three preliminary benchmarks and discuss our selection methodology . Our viewpoints reflect the collective thoughts of the TinyMLPerf working group that is comprised of URLanizations.
A
and is therefore fundamental to a field reaching maturity
A
. Furthermore, we present our three preliminary benchmarks and discuss our selection methodology