exponentiated gradient algorithms for conditional random fields and max margin markov networks log linear and maximum margin models are two commonly used methods in supervised machine learning and are frequently used in structured prediction problems efficient learning of parameters in these models is therefore an important problem and becomes a key factor when learning from very large data sets this paper describes exponentiated gradient eg algorithms for training such models where eg updates are applied to the convex dual of either the log linear or max margin objective function the dual in both the log linear and max margin cases corresponds to minimizing a convex function with simplex constraints we study both batch and online variants of the algorithm and provide rates of convergence for both cases in the max margin case o 1 e eg updates are required to reach a given accuracy e in the dual in contrast for log linear models only o log 1 e updates are required for both the max margin and log linear cases our bounds suggest that the online eg algorithm requires a factor of n less computation to reach a desired accuracy than the batch eg algorithm where n is the number of training examples our experiments confirm that the online algorithms are much faster than the batch algorithms in practice we describe how the eg updates factor in a convenient way for structured prediction problems allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing we perform extensive evaluation of the algorithms comparing them to l bfgs and stochastic gradient descent for log linear models and to svm struct for max margin models the algorithms are applied to a multi class problem as well as to a more complex large scale parsing task in all these settings the eg algorithms presented here outperform the other methods

pinning and wetting transition for 1 1 dimensional fields with laplacian interaction we consider a random field 1 n as a model for a linear chain attracted to the defect line 0 that is the x axis the free law of the field is specified by the density exp iv i with respect to the lebesgue measure on n where is the discrete laplacian and we allow for a very large class of potentials v the interaction with the defect line is introduced by giving the field a reward 0 each time it touches the x axis we call this model the pinning model we consider a second model the wetting model in which in addition to the pinning reward the field is also constrained to stay nonnegative we show that both models undergo a phase transition as the intensity of the pinning reward varies both in the pinning a p and in the wetting a w case there exists a critical value ca such that when ca the field touches the defect line a positive fraction of times localization while this does not happen for ca delocalization the two critical values are nontrivial and distinct 0 cp cw and they are the only nonanalyticity points of the respective free energies for the pinning model the transition is of second order hence the field at cp is delocalized on the other hand the transition in the wetting model is of first order and for cw the field is localized the core of our approach is a markov renewal theory description of the field

dynamics of cubic siegel polynomials we study the one dimensional parameter space of cubic polynomials in the complex plane which have a fixed siegel disk of rotation number where is a given irrational number of brjuno type the main result of this work is that when is of bounded type the boundary of the siegel disk is a quasicircle which contains one or both critical points of the cubic polynomial we also show that these boundaries vary continuously as one moves in the parameter space this is most nontrivial near the set of cubics with both critical points on the boundary of their siegel disk we prove that this locus is a jordan curve in the parameter space most of the techniques and results can be generalized to polynomials of higher degrees

herman s condition and siegel disks of bi critical polynomials we extend a theorem of herman from the case of unicritical polynomials to the case of polynomials with two finite critical values this theorem states that siegel disks of such polynomials under a diophantine condition called herman s condition on the rotation number must have a critical point on their boundaries

structured prediction dual extragradient and bregman projections we present a simple and scalable algorithm for maximum margin estimation of structured output models including an important class of markov networks and combinatorial models we formulate the estimation problem as a convex concave saddle point problem that allows us to use simple projection methods based on the dual extragradient algorithm nesterov 2003 the projection step can be solved using dynamic programming or combinatorial algorithms for min cost convex flow depending on the structure of the problem we show that this approach provides a memory efficient alternative to formulations based on reductions to a quadratic program qp we analyze the convergence of the method and present experiments on two very different structured prediction tasks 3d image segmentation and word alignment illustrating the favorable scaling properties of our algorithm

accelerated training of max margin markov networks with kernels structured output prediction is an important machine learning problem both in theory and practice and the max margin markov network m 3n is an effective approach all state of the art algorithms for optimizing m 3n objectives take at least o 1 e number of iterations to find an e accurate solution nesterov 1 broke this barrier by proposing an excessive gap reduction technique egr which converges in o 1 e iterations however it is restricted to euclidean projections which consequently requires an intractable amount of computation for each iteration when applied to solve m 3n in this paper we show that by extending egr to bregman projection this faster rate of convergence can be retained and more importantly the updates can be performed efficiently by exploiting graphical model factorization further we design a kernelized procedure which allows all computations per iteration to be performed at the same cost as the state of the art approaches

one sided local large deviation and renewal theorems in the case of infinite mean if s n n 0 is an integer valued random walk such that s n a n converges in distribution to a stable law of index 0 1 as n then gnedenko s local limit theorem provides a useful estimate for p s n r for values of r such that r a n is bounded the main point of this paper is to show that under certain circumstances there is another estimate which is valid when r a n in other words to establish a large deviation local limit theorem we also give an asymptotic bound for p s n r which is valid under weaker assumptions this last result is then used in establishing some local versions of generalized renewal theorems

the range of stable random walks limit theorems are proved for the range of d dimensional random walks in the domain of attraction of a stable process of index p3 the range rn is the number of distinct sites of zd visited by the random walk before time n our results depend on the value of the ratio 3 d the most interesting results are obtained for 2 3 1 which can only occur if d 1 we prove the convergence in distribution of rn e rn toward some constant times the lebesgue measure of the range of the limiting stable process some of our results require regularity assumptions on the characteristic function of x

excessive gap technique in nonsmooth convex minimization in this paper we introduce a new primal dual technique for convergence analysis of gradient schemes for nonsmooth convex optimization as an example of its application we derive a primal dual gradient method for a special class of structured nonsmooth optimization problems which ensures a rate of convergence of order o 1 k where k is the iteration count another example is a gradient scheme which minimizes a nonsmooth strongly convex function with known structure with rate of convergence o 1 k2 in both cases the efficiency of the methods is higher than the corresponding black box lower complexity bounds by an order of magnitude

intersections and limits of regenerative sets a regenerative set m is a random set of real numbers that has a type of markov property not necessarily at arbitrary times like the set of instances following independent exponentially distributed waiting times but only at those times belonging to m early uses of the term regenerative set appear in meyer 1970 and maisonneuve and morando 1970 and earlier tackas 1956 used regeneration point for a member of a certain type of regenerative set as clarification for some readers but possibly prematurely for others i mention that feller 1968 in chapter xiii used the term recurrent event but this term has been largely discarded because a regenerative set is not what is usually called an event hoffman jorgensen 1969 uses the term markov set whereas krylov and yushkevich 1965 use markov random set for a stochastic process related to a regenerative set breiman 1968 and kingman 1972 focus on the indicator function of m using the terms renewal process and regenerative phenomenon respectively we will stick to regenerative set

estimates on path delocalization for copolymers at selective interfaces starting from the simple symmetric random walk s n n we introduce a new process whose path measure is weighted by a factor exp open image in new window with h 0 w n n a typical realization of an iid process and n a positive integer we are looking for results in the large n limit this factor favors s n 0 if w n h 0 and s n 0 if w n h 0 the process can be interpreted as a model for a random heterogeneous polymer in the proximity of an interface separating two selective solvents it has been shown 6 that this model undergoes a de localization transition more precisely there exists a continuous increasing function h c such that if h h c then the model is localized while it is delocalized if h h c however localization and delocalization were not given in terms of path properties but in a free energy sense later on it has been shown that free energy localization does indeed correspond to a strong form of path localization 3 on the other hand only weak results on the delocalized regime have been known so far

all bounded type siegel disks of rational maps are quasi disks we prove that every bounded type siegel disk of a rational map must be a quasi disk with at least one critical point on its boundary this verifies douady sullivan s conjecture in the case of bounded type rotation numbers

a numerical approach to copolymers at selective interfaces we consider a model of a random copolymer at a selective interface which undergoes a localization delocalization transition in spite of the several rigorous results available for this model the theoretical characterization of the phase transition has re mained elusive and there is still no agreement about several important issues for example the behavior of the polymer near the phase transition line from a rigorous viewpoint non coinciding upper and lower bounds on the critical line are known in this paper we combine numerical computations with rigorous arguments to get to a better understanding of the phase diagram our main results include various numerical observations that suggest that the critical line lies strictly in between the two bounds a rigorous statistical test based on concentration inequalities and super additivity for determining whether a given point of the phase diagram is in the localized phase this is applied in particular to show that with a very low level of error the lower bound does not coincide with the critical line an analysis of the precise asymptotic behavior of the partition function in the de localized phase with particular attention to the effect of rare atypical stretches in the disorder sequence and on whether or not in the delocalized regime the polymer path has a brownian scaling a new proof of the lower bound on the critical line this proof relies on a characteri zation of the localized regime which is more appealing for interpreting the numerical data

bessel spdes and renormalised local times in this article we prove integration by parts formulae ibpfs for the laws of bessel bridges from 0 to 0 over the interval 0 1 of dimension smaller than 3 as an application we construct a weak version of a stochastic pde having the law of a one dimensional bessel bridge i e the law of a reflected brownian bridge as reversible measure the dimension 1 being particularly relevant in view of applications to scaling limits of dynamical critical pinning models we also exploit the ibpfs to conjecture the structure of the stochastic pdes associated with bessel bridges of all dimensions smaller than 3

a decomposition theorem for herman maps abstract in 1980s thurston established a topological characterization theorem for postcritically finite rational maps in this paper a decomposition theorem for a class of postcritically infinite branched covering termed herman map is developed it s shown that every herman map can be decomposed along a stable multicurve into finitely many siegel maps and thurston maps such that the combinations and rational realizations of these resulting maps essentially dominate the original one this result is motivated by a non expanding version of mcmullen s problem and thurston s theory on characterization of rational maps it enables us to prove a thurston type theorem for rational maps with herman rings

bundle methods for regularized risk minimization a wide variety of machine learning problems can be described as minimizing a regularized risk functional with different algorithms using different notions of risk and different regularizers examples include linear support vector machines svms gaussian processes logistic regression conditional random fields crfs and lasso amongst others this paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems it can be parallelized on a cluster of workstations allows for data locality and can deal with regularizers such as l1 and l2 penalties in addition to the unified framework we present tight convergence bounds which show that our algorithm converges in o 1 e steps to e precision for general convex problems and in o log 1 e steps for continuously differentiable problems we demonstrate the performance of our general purpose solver on a variety of publicly available data sets

random walk versus random line abstract we consider random walks x n in z obeying a detailed balance condition with a weak drift towards the origin when x n we reconsider the equivalence in law between a random walk bridge and a 1 1 dimensional solid on solid bridge with a corresponding hamiltonian phase diagrams are discussed in terms of recurrence versus wetting a drift x n 1 o x n 2 of the random walk yields a solid on solid potential with an attractive well at the origin and a repulsive tail 2 8 x n 2 o x n 3 at infinity showing complete wetting for 1 and critical partial wetting for 1

are there critical points on the boundaries of singular domains generalizing a result of e ghys we prove a general theorem that implies that if a rational functionf of the riemann sphere of degree 2 leaves invariant a singular domainc a disk or a ring on which the rotation number off satisfies a diophantine condition provided that on bar c f is injective then each boundary component ofc contains critical point off the injectivity condition is always satisfied for singular disks associated to linearizable periodic elliptic points off z z n a withne n 2 andae we also show that the singular disks associated to periodic elliptic points off z e az that satisfy a diophantine condition are unbounded in in the end of the paper we give a survey of the theory of iteration of entire functions of

concentration under scaling limits for weakly pinned gaussian random walks we study scaling limits for d dimensional gaussian random walks perturbed by an attractive force toward a certain subspace of mathbb r d especially under the critical situation that the rate functional of the corresponding large deviation principle admits two minimizers we obtain different type of limits in a positive recurrent regime depending on the co dimension of the subspace and the conditions imposed at the final time under the presence or absence of a wall the motivation comes from the study of polymers or 1 1 dimensional interfaces with pinning

bounded type siegel disks of a family of sine families abstract we prove that for any positive integer n there is a polynomial p z with degree n such that the entire function g z p sin z has a bounded type siegel disk bounded by a quasi circle in the plane which passes through at least one critical point of g in addition let 0 1 be an irrational number of bounded type we prove that for any integer k 0 the boundary of the siegel disk of g z 0 sin z e 2 i 1 s 2 k d s centered at the origin is a quasi circle passing through exactly two critical points 2 and 2 with multiplicity k 1 respectively

scaling limits of equilibrium wetting models in 1 1 dimension we study the path properties for the pinning wetting model in 1 1 dimension in other terms we consider a random walk model with fairly general continuous increments conditioned to stay in the upper half plane and with a measure reward for touching zero that is the boundary of the forbidden region it is well known that such a model displays a localization delocalization transition according to the size of the reward our focus is on getting a precise pathwise description of the system in both the delocalized phase that includes the critical case and in the localized one from this we extract the brownian scaling limits of the model

upper bound for the size of quadratic siegel disks if is an irrational number we let p n q n n 0 be the approximants given by its continued fraction expansion the bruno series b is defined as b alpha sum n geq 0 frac log q n 1 q n the quadratic polynomial p z e 2i z z 2 has an indifferent fixed point at the origin if p is linearizable we let r be the conformal radius of the siegel disk and we set r 0 otherwise yoccoz proved that if b then r 0 and p is not linearizable in this article we present a different proof and we show that there exists a constant c such that for all irrational number with b we have b alpha log r alpha c together with former results of yoccoz see y this proves the conjectured boundedness of b logr

wetting of gradient fields pathwise estimates we consider the wetting transition in the framework of an effective interface model of gradient type in dimension 2 and higher we prove pathwise estimates showing that the interface is localized in the whole thermodynamically defined partial wetting regime considered in earlier works moreover we study how the interface delocalizes as the wetting transition is approached our main tool is reflection positivity in the form of the chessboard estimate