Convex analysis
Convex analysis is the branch of mathematics devoted to the study of properties of convex functions and convex sets, often with applications in convex minimization, a subdomain of optimization theory. Convex sets. A subset formula_1 of some vector space formula_2 is called if it satisfies any of the following equivalent conditions: Throughout, formula_10 will be a map valued in the extended real numbers formula_11 with a domain formula_12 that is a convex subset of some vector space. The map formula_10 is if holds for any real formula_6 and any formula_15 with formula_16 If this remains true of formula_17  when the defining inequality () is replaced by the strict inequality then formula_17 is called . Convex functions are related to convex sets. Specifically, the function formula_17 is convex if and only if its is a convex set. The epigraphs of extended real-valued functions play a role in convex analysis that is analogous to the role played by graphs of real-valued function in real analysis. Specifically, the epigraph of an extended real-valued function provides geometric intuition that can be used to help formula or prove conjectures. The domain of a function formula_10 is denoted by formula_21 while its is  the set The function formula_10 is called if formula_23 and formula_24 for formula_25 Alternatively, this means that there exists some formula_26 in the domain of formula_17 at which formula_28 and formula_17 is also equal to formula_30 In words, a function is if its domain is not empty, it never takes on the value formula_31 and it also is not identically equal to formula_32 If formula_33 is a proper convex function then there exist some vector formula_34 and some formula_35 such that where formula_38 denotes the dot product of these vectors. Convex conjugate. The of an extended real-valued function formula_10  (not necessarily convex) is the function formula_40 from the (continuous) dual space formula_41 of formula_42 and where the brackets formula_44 denote the canonical duality formula_45 The of formula_17 is the map formula_47 defined by formula_48 for every formula_49 If formula_50 denotes the set of formula_51-valued functions on formula_42 then the map formula_53 defined by formula_54 is called the . Subdifferential set and the Fenchel-Young inequality. If formula_10 and formula_56 then the is For example, in the important special case where formula_58 is a norm on formula_2, it can be shown that if formula_60 then this definition reduces down to : For any formula_56 and formula_64 formula_65 which is called the . This inequality is an equality (i.e. formula_66) if and only if formula_67 It is in this way that the subdifferential set formula_68 is directly related to the convex conjugate formula_69 Biconjugate. The of a function formula_10 is the conjugate of the conjugate, typically written as formula_71 The biconjugate is useful for showing when strong or weak duality hold (via the perturbation function). For any formula_72 the inequality formula_73 follows from the . For proper functions, formula_74 if and only if formula_17 is convex and lower semi-continuous by Fenchel–Moreau theorem . Convex minimization. A () is one of the form Dual problem. In optimization theory, the states that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. In general given two dual pairs separated locally convex spaces formula_79 and formula_80 Then given the function formula_81 we can define the primal problem as finding formula_26 such that If there are constraint conditions, these can be built into the function formula_17 by letting formula_85 where formula_86 is the indicator function. Then let formula_87 be a perturbation function such that formula_88 The with respect to the chosen  perturbation function is given by where formula_90 is the convex conjugate in both variables of formula_91 The duality gap is the difference of the right and left hand sides of the inequality This principle is the same as weak duality. If the two sides are equal to each other, then the problem is said to satisfy strong duality. There are many conditions for strong duality to hold such as: Lagrange duality. For a convex minimization problem with inequality constraints, the Lagrangian dual problem is where the objective function formula_103 is the Lagrange dual function defined as follows: Convex analysis Convex