Software can be equally variable.
In some cases, software and hardware used in testing processes can obscure how a system is behaving, skewing benchmark results in an unforeseen fashion. Because there are usually no standards or generally accepted practices for embedded devices as is more often the case for desktop systems, this initialization state can vary based on which software debugger is used to initialize the board. This can result in variable performance results from identical binary executable files based strictly on the debugger that was used to download file to the board. Because of this sort of nuance, it is imperative that those who undertake a benchmarking effort be sure that the test procedure is clearly documented and that all inputs to the test can be duplicated by independent third parties. Differences Between Desktop and Embedded System Benchmarking Some systems, such as desktop PCs and workstations, are easier to benchmark than other systems.