  The Swampland de Sitter conjecture in combination with upper limits on the
tensor-to-scalar ratio $r$ derived from observations of the cosmic microwave
background endangers the paradigm of slow-roll single field inflation. This
conjecture constrains the first and the second derivatives of the inflationary
potential in terms of two ${\cal O} (1)$ constants $c$ and $c'$. In view of
these restrictions we reexamine single-field inflationary potentials with
$S$-duality symmetry, which ameliorate the unlikeliness problem of the initial
condition. We compute $r$ at next-to-leading order in slow-roll parameters for
the most general form of $S$-dual potentials and confront model predictions to
constraints imposed by the de Sitter conjecture. We find that $c \sim {\cal O}
(10^{-1})$ and $c' \sim {\cal O} (10^{-2})$ can accommodate the 95\% CL upper
limit on $r$. By imposing at least 50 $e$-folds of inflation with the effective
field theory description only valid over a field displacement ${\cal O} (1)$
when measured as a distance in the target space geometry, we further restrict
$c \sim {\cal O} (10^{-2})$, while the constraint on $c'$ remains unchanged. We
comment on how to accommodate the required small values of $c$ and $c'$.
