  We study the secure stochastic convex optimization problem. A learner aims to
learn the optimal point of a convex function through sequentially querying a
(stochastic) gradient oracle. In the meantime, there exists an adversary who
aims to free-ride and infer the learning outcome of the learner from observing
the learner's queries. The adversary observes only the points of the queries
but not the feedback from the oracle. The goal of the learner is to optimize
the accuracy, i.e., obtaining an accurate estimate of the optimal point, while
securing her privacy, i.e., making it difficult for the adversary to infer the
optimal point. We formally quantify this tradeoff between learner's accuracy
and privacy and characterize the lower and upper bounds on the learner's query
complexity as a function of desired levels of accuracy and privacy. For the
analysis of lower bounds, we provide a general template based on information
theoretical analysis and then tailor the template to several families of
problems, including stochastic convex optimization and (noisy) binary search.
We also present a generic secure learning protocol that achieves the matching
upper bound up to logarithmic factors.
