  Automatic differentiation is a set of techniques to efficiently and
accurately compute the derivative of a function represented by a computer
program. Existing C++ libraries for automatic differentiation (e.g. Adept, Stan
Math Library), however, exhibit large memory consumptions and runtime
performance issues. This paper introduces FastAD, a new C++ template library
for automatic differentiation, that overcomes all of these challenges in
existing libraries by using vectorization, simpler memory management using a
fully expression-template-based design, and other compile-time optimizations to
remove some run-time overhead. Benchmarks show that FastAD performs 2-10 times
faster than Adept and 2-19 times faster than Stan across various test cases
including a few real-world examples.
