  We study the problem of optimizing a non-convex loss function (with saddle
points) in a distributed framework in the presence of Byzantine machines. We
consider a standard distributed setting with one central machine (parameter
server) communicating with many worker machines. Our proposed algorithm is a
variant of the celebrated cubic-regularized Newton method of Nesterov and
Polyak \cite{nest}, which avoids saddle points efficiently and converges to
local minima. Furthermore, our algorithm resists the presence of Byzantine
machines, which may create \emph{fake local minima} near the saddle points of
the loss function, also known as saddle-point attack. We robustify the
cubic-regularized Newton algorithm such that it avoids the saddle points and
the fake local minimas efficiently. Furthermore, being a second order
algorithm, the iteration complexity is much lower than its first order
counterparts, and thus our algorithm communicates little with the parameter
server. We obtain theoretical guarantees for our proposed scheme under several
settings including approximate (sub-sampled) gradients and Hessians. Moreover,
we validate our theoretical findings with experiments using standard datasets
and several types of Byzantine attacks.
