Problem: Let $a, b, c$ be nonnegative real numbers
such that $a^2+b^2+c^2 + abc = 4$.
Show that \[ 0 \le ab + bc + ca - abc \le 2. \]

Solution: The left-hand side of the inequality is trivial;
just note that $\min \left\{ a,b,c \right\} \le 1$.
Hence, we focus on the right side.
We use Lagrange Multipliers.

Define \[ U = \left\{ (a,b,c) \mid a,b,c > 0
  \text{ and } a^2+b^2+c^2 < 1000 \right\}. \]
This is an intersection of open sets, so it is open.
Its closure is
\[ \ol U = \left\{ (a,b,c) \mid a,b,c \ge 0
\text{ and } a^2+b^2+c^2 \le 1000 \right\}. \]
Hence the constraint set
\[ \ol S = \left\{ \mathbf{x} \in \ol U \mid g(\mathbf{x}) = 4 \right\} \]
is compact, where $g(a,b,c) = a^2+b^2+c^2+abc$.

Define \[ f(a,b,c) = a^2+b^2+c^2+ab+bc+ca. \]
It's equivalent to show that $f \le 6$ subject to $g$.
Over $\ol S$, it must achieve a global maximum.
Now we consider two cases.

If $\mathbf{x}$ lies on the boundary,
that means one of the components is zero
(since $a^2+b^2+c^2=1000$ is clearly impossible).
WLOG $c=0$, then we wish to show $a^2+b^2+ab \le 6$
for $a^2+b^2=4$, which is trivial.

Now for the interior $U$, we may use the method of Lagrange Multipliers.
Consider a local maximum $\mathbf{x} \in U$.
Compute \[ \nabla f = \left<2a+b+c, 2b+c+a, 2c+a+b \right> \]
and \[ \nabla g = \left<2a+bc, 2b+ca, 2c+ab\right>. \]
Of course, $\nabla g \neq \mathbf 0$ everywhere,
so introducing our multiplier yields
\[ \left<2a+b+c,a+2b+c,a+b+2c\right> = \lambda \left<2a+bc,2b+ca,2c+ab\right>. \]
Note that $\lambda \neq 0$ since $a,b,c > 0$.
Subtracting $2a+b+c = \lambda(2a+bc)$ from $a+2b+c = \lambda(2b+ca)$
implies that \[ (a-b)(\left[ 2\lambda - 1 \right] - \lambda c) = 0. \]
We can derive similar equations for the others.
Hence, we have three cases.
\begin{enumerate}
  \ii If $a=b=c$, then $a=b=c=1$, and this satisfies $f(1,1,1) \le 6$.
  \ii If $a$, $b$, $c$ are pairwise distinct,
  then we derive $a = b = c = 2 - \lambda^{-1}$, contradiction.
  \ii Now suppose that $a=b \neq c$.

  Meanwhile, the constraint (with $a=b$) reads
  \begin{align*}
    a^2+b^2+c^2+abc=4
    &\iff c^2 + a^2c + (2a^2-4) = 0 \\
    &\iff (c+2)(c-(2-a^2)) = 0
  \end{align*}
  which since $c > 0$ gives $c = 2-a^2$.

  Noah Walsh points out that at this point,
  we don't need to calculate the critical point;
  we just directly substitute $a=b$ and $c=2-a^2$ into
  the desired inequality:
  \[ a^2+2a(2-a^2)-a^2(2-a^2) = (a-1)^2(a^2-2) + 2 \le 2 \]
  since $a \le \sqrt 2$.
  So any point here satisfies the inequality anyways.
\end{enumerate}
\begin{remark*}
  It can actually be shown that the critical point
  in the third case we skipped is pretty close: it is given by
  \[ a = b = \frac{1+\sqrt{17}}{4}
    \quad c = \frac{1}{8} \left( 7 - \sqrt{17} \right). \]
  This satisfies
  \[ f(a,b,c) = 3a^2+2ac+c^2
    = \frac{1}{32} \left( 121 + 17\sqrt{17} \right)
    \approx 5.97165 \]
  which is just a bit less than $6$.
\end{remark*}
\begin{remark*}
  Equality holds for the upper bound if $(a,b,c)=(1,1,1)$
  or $(a,b,c)=(\sqrt2,\sqrt2,0)$ and permutations.
  The lower bound is achieved if $(a,b,c)=(2,0,0)$ and permutations.
\end{remark*}