  We discuss the realization of a universal set of ultrafast single- and
two-qubit operations with superconducting quantum circuits and investigate the
most relevant physical and technical limitations that arise when pushing for
faster and faster gates. With the help of numerical optimization techniques, we
establish a fundamental bound on the minimal gate time, which is determined
independently of the qubit design solely by its nonlinearity. In addition,
important practical restrictions arise from the finite qubit transition
frequency and the limited bandwidth of the control pulses. We show that for
highly anharmonic flux qubits and commercially available control electronics,
elementary single- and two-qubit operations can be implemented in about 100
picoseconds with residual gate errors below $10^{-4}$. Under the same
conditions, we simulate the complete execution of a compressed version of
Shor's algorithm for factoring the number 15 in about one nanosecond. These
results demonstrate that compared to state-of-the-art implementations with
transmon qubits, a hundredfold increase in the speed of gate operations with
superconducting circuits is still feasible.
