We present AutoPerf, a generalized software performance regression diagnosis system. AutoPerf uses autoencoders, an unsupervised learning technique, and hardware performance counters to learn the performance signatures of a program. It then uses this knowledge to identify when newer versions of the program suffer from performance regressions, while simultaneously providing root cause analysis to help programmers debug the program's performance. AutoPerf is the first zero-positive learning performance regression diagnosis system. It trains entirely in the negative (non-anomalous) space to learn positive (anomalous) behaviors . We demonstrate AutoPerf's generality against three different types of performance regressions : (i) true sharing cache contention, (ii) false sharing cache contention, and (iii) NUMA latencies across 15 real world performance regressions and 7 open source programs. On average, AutoPerf exhibits only 3.7 \% profiling overhead and diagnoses more regressions than prior state-of-the-art approaches.
A performance regression is a degradation of software performance due to a change in code. A common way to identify such regressions is to manually create tests for them. This is known as regression testing. Although widely used, regression testing can be slow and error-prone due to its historically manual nature. Moreover, such tests' utility is usually constrained by the expertise of the developer creating them. Due to these limitations, researchers have turned their attention to the automatic generation of regression tests. In this paper, we present AutoPerf - a novel approach to automate regression testing that utilizes three core techniques: (i) zero-positive learning , (ii) autoencoders, and (iii) hardware telemetry . We demonstrate AutoPerf's generality and efficacy against 3 types of performance regressions across 10 real performance bugs in 7 benchmark and open-source programs. On average, AutoPerf exhibits 4 \% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches.
R
We present AutoPerf, a generalized software performance regression diagnosis system. AutoPerf uses autoencoders, an unsupervised learning technique, and hardware performance counters to learn the performance signatures of a program. It then uses this knowledge to identify when newer versions of the program suffer from performance regressions, while simultaneously providing root cause analysis to help programmers debug the program's performance. AutoPerf is the first
A performance regression is a degradation of software performance due to a change in code. A common way to identify such regressions is to manually create tests for them. This is known as regression testing. Although widely used, regression testing can be slow and error-prone due to its historically manual nature. Moreover, such tests' utility is usually constrained by the expertise of the developer creating them. Due to these limitations, researchers have turned their attention to the automatic generation of regression tests. In this paper, we present AutoPerf - a novel approach to automate regression testing that utilizes three core techniques: (i)
R
performance regression diagnosis system. It trains entirely in the negative (non-anomalous) space to learn positive (anomalous) behaviors
, (ii) autoencoders, and (iii) hardware telemetry
R
against three different
and efficacy against 3
R
: (i) true sharing cache contention, (ii) false sharing cache contention, and (iii) NUMA latencies across 15 real world performance regressions and
across 10 real performance bugs in
R
open source
benchmark and open-source
R
only 3.7
4
R
diagnoses more regressions
accurately diagnoses more performance bugs