the separating capacity of a multithreshold threshold element in answer to what represents the intrinsic information processing capability of the pattern classification system cover 1 has defined the separating capacity and has derived it for the linear machine and the so called machine in this paper the separating capacity of a multithreshold classification element is obtained it is shown that the capacity of a multithreshold threshold element with k thresholds k threshold element in n dimensional space is 2 n k a linear machine is a special case in the k threshold element with k 1 therefore its capacity becomes 2 n 1 from the above result further although it is intuitively apparent that the larger the number of thresholds the more powerful the information processing capability of the k threshold element using the capacity as a measure of this capability we may now state that the separating power of the k threshold element increases linearly with respect to k

a small world of weak ties provides optimal global integration of self similar modules in functional brain networks the human brain is organized in functional modules such an organization presents a basic conundrum modules ought to be sufficiently independent to guarantee functional specialization and sufficiently connected to bind multiple processors for efficient information transfer it is commonly accepted that small world architecture of short paths and large local clustering may solve this problem however there is intrinsic tension between shortcuts generating small worlds and the persistence of modularity a global property unrelated to local clustering here we present a possible solution to this puzzle we first show that a modified percolation theory can define a set of hierarchically organized modules made of strong links in functional brain networks these modules are large world self similar structures and therefore are far from being small world however incorporating weaker ties to the network converts it into a small world preserving an underlying backbone of well defined modules remarkably weak ties are precisely organized as predicted by theory maximizing information transfer with minimal wiring cost this trade off architecture is reminiscent of the strength of weak ties crucial concept of social networks such a design suggests a natural solution to the paradox of efficient information flow in the highly modular structure of the brain

generalization error bounds for threshold decision lists in this paper we consider the generalization accuracy of classification methods based on the iterative use of linear classifiers the resulting classifiers which we call threshold decision lists act as follows some points of the data set to be classified are given a particular classification according to a linear threshold function or hyperplane these are then removed from consideration and the procedure is iterated until all points are classified geometrically we can imagine that at each stage points of the same classification are successively chopped off from the data set by a hyperplane we analyse theoretically the generalization properties of data classification techniques that are based on the use of threshold decision lists and on the special subclass of multilevel threshold functions we present bounds on the generalization error in a standard probabilistic learning framework the primary focus in this paper is on obtaining generalization error bounds that depend on the levels of separation or margins achieved by the successive linear classifiers we also improve and extend previously published theoretical bounds on the generalization ability of perceptron decision trees

scale sensitive dimensions uniform convergence and learnability learnability in valiant s pac learning model has been shown to be strongly related to the existence of uniform laws of large numbers these laws define a distribution free convergence property of means to expectations uniformly over classes of random variables classes of real valued functions enjoying such a property are also known as uniform glivenko cantelli classes in this paper we prove through a generalization of sauer s lemma that may be interesting in its own right a new characterization of uniform glivenko cantelli classes our characterization yields dudley gine and zinn s previous characterization as a corollary furthermore it is the first based on a gine and zinn s previous characterization as a corollary furthermore it is the first based on a simple combinatorial quantity generalizing the vapnik chervonenkis dimension we apply this result to obtain the weakest combinatorial condition known to imply pac learnability in the statistical regression or agnostic framework furthermore we find a characterization of learnability in the probabilistic concept model solving an open problem posed by kearns and schapire these results show that the accuracy parameter plays a crucial role in determining the effective complexity of the learner s hypothesis class

growth model for fractal scale free networks generated by a random walk abstract the diversity of networked systems with fractal structures suggests that knowing the underlying mechanism that generates the fractality is necessary for building a model of the development of complex networks in the present paper we propose a growth model of a network generated by a random walk and show that the evolving graph forms a fractal structure with various properties including the scale free property if the graph which provides a space where a random walk occurs by itself is formed by the random walk the proposed model is regulated by two parameters p v and p e which define the probability of either a roundabout path via a new vertex or a shortcut being formed by the random walk respectively the power law exponent describing the vertex degree distribution is determined by the ratio p e p v and is related to an internal factor f i via the relation 1 f i 1 where f i is a parameter that describes the local structure generated by the random walk a sufficiently small p v provides the small world property to the model network the small world property is usually considered to be incompatible with the fractal scaling property open m c l c d c where open m c is the average number of vertices which can be reached from a randomly chosen vertex in at most l c steps however we demonstrate that fractality can be reconciled with the small world property by introducing a size dependent fractal cluster dimension d c

a tight characterization of the performance of static solutions in two stage adjustable robust linear optimization in this paper we study the performance of static solutions for two stage adjustable robust linear optimization problems with uncertain constraint and objective coefficients and give a tight characterization of the adaptivity gap computing an optimal adjustable robust optimization problem is often intractable since it requires to compute a solution for all possible realizations of uncertain parameters feige et al in lect notes comput sci 4513 439 453 2007 on the other hand a static solution is a single here and now solution that is feasible for all possible realizations of the uncertain parameters and can be computed efficiently for most dynamic optimization problems we show that for a fairly general class of uncertainty sets a static solution is optimal for the two stage adjustable robust linear packing problems this is highly surprising in view of the usual perception about the conservativeness of static solutions furthermore when a static solution is not optimal for the adjustable robust problem we give a tight approximation bound on the performance of the static solution that is related to a measure of non convexity of a transformation of the uncertainty set we also show that our bound is at least as good and in many case significantly better as the bound given by the symmetry of the uncertainty set bertsimas and goyal in math methods oper res 77 3 323 343 2013 bertsimas et al in math oper res 36 1 24 54 2011

a threshold of ln n for approximating set cover given a collection f of subsets of s 1 n setcover is the problem of selecting as few as possiblesubsets from f such that their union covers s and maxk cover is the problem of selecting k subsets from f such that their union has maximum cardinality both these problems arenp hard we prove that 1 o 1 ln n is a threshold below which setcover cannot be approximated efficiently unless np has slightlysuperpolynomial time algorithms this closes the gap up to low orderterms between the ratio of approximation achievable by the greedyalogorithm which is 1 o 1 lnn and provious results of lund and yanakakis that showed hardness ofapproximation within a ratio of log 2 n 2s0 72 ln n for max k cover we show an approximationthreshold of 1 1 e up tolow order terms under assumption that p np

stochastic programming approach to optimization under uncertainty in this paper we discuss computational complexity and risk averse approaches to two and multistage stochastic programming problems we argue that two stage say linear stochastic programming problems can be solved with a reasonable accuracy by monte carlo sampling techniques while there are indications that complexity of multistage programs grows fast with increase of the number of stages we discuss an extension of coherent risk measures to a multistage setting and in particular dynamic programming equations for such problems

the price of robustness a robust approach to solving linear optimization problems with uncertain data was proposed in the early 1970s and has recently been extensively studied and extended under this approach we are willing to accept a suboptimal solution for the nominal values of the data in order to ensure that the solution remains feasible and near optimal when the data changes a concern with such an approach is that it might be too conservative in this paper we propose an approach that attempts to make this trade off more attractive that is we investigate ways to decrease what we call the price of robustness in particular we flexibly adjust the level of conservatism of the robust solutions in terms of probabilistic bounds of constraint violations an attractive aspect of our method is that the new robust formulation is also a linear optimization problem thus we naturally extend our methods to discrete optimization problems in a tractable way we report numerical results for a portfolio optimization problem a knapsack problem and a problem from the net lib library

the capacity of multilevel threshold functions lower and upper bounds for the capacity of multilevel threshold elements are estimated using two essentially different enumeration techniques it is demonstrated that the exact number of multilevel threshold functions depends strongly on the relative topology of the input set the results correct a previously published estimate and indicate that adding threshold levels enhances the capacity more than adding variables

fractality and the small world effect in sierpinski graphs although some real networks exhibit self similarity there is no standard definition of fractality in graphs on the other hand the small world phenomenon is one of the most important common properties of real interconnection networks in this paper we relate these two properties in order to do so we focus on the family of sierpinski networks for the sierpinski gasket the sierpinski carpet and the sierpinski tetra we give the basic properties and we calculate the box counting dimension as a measure of their fractality we also define a deterministic family of graphs which we call small world sierpinski graphs we show that our construction preserves the structure of sierpinski graphs including its box counting dimension while the small world phenomenon arises thus in this family of graphs fractality and small world effect are simultaneously present

robust portfolio selection problems in this paper we show how to formulate and solve robust portfolio selection problems the objective of these robust formulations is to systematically combat the sensitivity of the optimal portfolio to statistical and modeling errors in the estimates of the relevant market parameters we introduce uncertainty structures for the market parameters and show that the robust portfolio selection problems corresponding to these uncertainty structures can be reformulated as second order cone programs and therefore the computational effort required to solve them is comparable to that required for solving convex quadratic programs moreover we show that these uncertainty structures correspond to confidence regions associated with the statistical procedures employed to estimate the market parameters finally we demonstrate a simple recipe for efficiently computing robust portfolios given raw market data and a desired level of confidence

robust convex optimization we study convex optimization problems for which the data is not specified exactly and it is only known to belong to a given uncertainty set u yet the constraints must hold for all possible values of the data from u the ensuing optimization problem is called robust optimization in this paper we lay the foundation of robust convex optimization in the main part of the paper we show that if u is an ellipsoidal uncertainty set then for some of the most important generic convex optimization problems linear programming quadratically constrained programming semidefinite programming and others the corresponding robust convex program is either exactly or approximately a tractable problem which lends itself to efficientalgorithms such as polynomial time interior point methods

a general lower bound on the number of examples needed for learning abstract we prove a lower bound of 1 ln 1 vcdim c on the number of random examples required for distribution free learning of a concept class c where vcdim c is the vapnik chervonenkis dimension and and are the accuracy and confidence parameters this improves the previous best lower bound of 1 ln 1 vcdim c and comes close to the known general upper bound of o 1 ln 1 vcdim c ln 1 for consistent algorithms we show that for many interesting concept classes including k cnf and k dnf our bound is actually tight to within a constant factor

learnability and the vapnik chervonenkis dimension valiant s learnability model is extended to learning classes of concepts defined by regions in euclidean space e n the methods in this paper lead to a unified treatment of some of valiant s results along with previous results on distribution free convergence of certain pattern recognition algorithms it is shown that the essential condition for distribution free learnability is finiteness of the vapnik chervonenkis dimension a simple combinatorial parameter of the class of concepts to be learned using this parameter the complexity and closure properties of learnable classes are analyzed and the necessary and sufficient conditions are provided for feasible learnability

on the adaptivity gap in two stage robust linear optimization under uncertain packing constraints in this paper we study the performance of static solutions in two stage adjustable robust packing linear optimization problem with uncertain constraint coefficients such problems arise in many important applications such as revenue management and resource allocation problems where demand requests have uncertain resource requirements the goal is to find a two stage solution that maximizes the worst case objective value over all possible realizations of the second stage constraints from a given uncertainty set we consider the case where the uncertainty set is column wise and constraint wise any constraint describing the set involve entries of only a single column or a single row this is a fairly general class of uncertainty sets to model constraint coefficient uncertainty we show that the two stage adjustable robust problem is varomega log n hard to approximate on the positive side we show that a static solution is an o big log n cdot min log vargamma log m n big approximation for the two stage adjustable robust problem where m and n denote the numbers of rows and columns of the constraint matrix and vargamma is the maximum possible ratio of upper bounds of the uncertain constraint coefficients therefore for constant vargamma surprisingly the performance bound for static solutions and therefore the adaptivity gap matches the hardness of approximation for the adjustable problems furthermore in general the static solution provides nearly the best efficient approximation for the two stage adjustable robust problem

fractal small world dichotomy in real world networks we draw attention to a clear dichotomy between small world networks exhibiting exponential neighborhood growth and fractal like networks where neighborhoods grow according to a power law this distinction is observed in a number of real world networks and is related to the degree correlations and geographical constraints we conclude by pointing out that the status of human social networks in this dichotomy is far from clear

fractal networks induced by movements of random walkers on a tree graph abstract we show that a fractal scale free and small world network can grow on a non fractal cayley tree through the simple random process of shortcut creation by movements of random walkers the cayley tree provides a stage for the walkers to diffuse into a locally one dimensional structure with the small world property self organization of fractal graph by adding shortcut edges to the locally one dimensional structure generated by walkers flow from the root vertex to the outermost shell of the cayley tree is a novel mechanism for generating fractal graphs also we show numerically that the fractal box counting dimension and a power law exponent describing the degree distribution of the resulting graph are mainly determined by the bifurcation number of the original cayley tree in our model the fractal cluster dimension must change with the size of the evolving graph this is a general result that resolves an apparent contradiction between the fractality in the context of the cluster growing method and the small world property of the graph

reciprocal relation between the fractal and the small world properties of complex networks the fractal and the small world properties of complex networks are systematically studied both in the box covering bc and the cluster growing cg measurements we elucidate that complex networks possessing the fractal small world nature in the bc measurement are always fractal small world even in the cg measurement and vice versa while the fractal dimensions d text b by the bc measurement and d text c by the cg measurement are generally different this implies that two structural properties of networks fractality and small worldness cannot coexist in the same length scale these properties can however crossover from one to the other by varying the length scale we show that the crossover behavior in a network near the percolation transition appears both in the bc and cg measurements and is scaled by a unique characteristic length ensuremath xi

on the number of multilinear partitions and the computing capacity of multiple valued multiple threshold perceptrons we introduce the concept of multilinear partition of a point set v spl sub r sup n and the concept of multilinear separability of a function f v spl rarr k 0 k 1 based on well known relationships between linear partitions and minimal pairs we derive formulae for the number of multilinear partitions of a point set in general position and of the set k sup 2 the n k s perceptrons partition the input space v into s 1 regions with s parallel hyperplanes we obtain results on the capacity of a single n k s perceptron respectively for v spl sub r sup n in general position and for v k sup 2 finally we describe a fast polynomial time algorithm for counting the multilinear partitions of k sup 2

on the power and limitations of affine policies in two stage adaptive optimization we consider a two stage adaptive linear optimization problem under right hand side uncertainty with a min max objective and give a sharp characterization of the power and limitations of affine policies where the second stage solution is an affine function of the right hand side uncertainty in particular we show that the worst case cost of an optimal affine policy can be omega m 1 2 delta times the worst case cost of an optimal fully adaptable solution for any 0 where m is the number of linear constraints we also show that the worst case cost of the best affine policy is o sqrt m times the optimal cost when the first stage constraint matrix has non negative coefficients moreover if there are only k m uncertain parameters we generalize the performance bound for affine policies to o sqrt k which is particularly useful if only a few parameters are uncertain we also provide an o sqrt k approximation algorithm for the general case without any restriction on the constraint matrix but the solution is not an affine function of the uncertain parameters we also give a tight characterization of the conditions under which an affine policy is optimal for the above model in particular we show that if the uncertainty set mathcal u subseteq mathbb r m is a simplex then an affine policy is optimal however an affine policy is suboptimal even if mathcal u is a convex combination of only m 3 extreme points only two more extreme points than a simplex and the worst case cost of an optimal affine policy can be a factor 2 worse than the worst case cost of an optimal fully adaptable solution for any 0

robust optimization methodology and applications robust optimization ro is a modeling methodology combined with computational tools to process optimization problems in which the data are uncertain and is only known to belong to some uncertainty set the paper surveys the main results of ro as applied to uncertain linear conic quadratic and semidefinite programming for these cases computationally tractable robust counterparts of uncertain problems are explicitly obtained or good approximations of these counterparts are proposed making ro a useful tool for real world applications we discuss some of these applications specifically antenna design truss topology design and stability analysis synthesis in uncertain dynamic systems we also describe a case study of 90 lps from the netlib collection the study reveals that the feasibility properties of the usual solutions of real world lps can be severely affected by small perturbations of the data and that the ro methodology can be successfully used to overcome this phenomenon

fractality in complex networks critical and supercritical skeletons fractal scaling char22 a power law behavior of the number of boxes needed to tile a given network with respect to the lateral size of the box char22 is studied we introduce a box covering algorithm that is a modified version of the original algorithm introduced by song et al nature london 433 392 2005 this algorithm enables easy implementation fractal networks are viewed as comprising a skeleton and shortcuts the skeleton embedded underneath the original network is a special type of spanning tree based on the edge betweenness centrality it provides a scaffold for the fractality of the network when the skeleton is regarded as a branching tree it exhibits a plateau in the mean branching number as a function of the distance from a root for nonfractal networks on the other hand the mean branching number decays to zero without forming a plateau based on these observations we construct a fractal network model by combining a random branching tree and local shortcuts the scaffold branching tree can be either critical or supercritical depending on the small worldness of a given network for the network constructed from the critical supercritical branching tree the average number of vertices within a given box grows with the lateral size of the box according to a power law an exponential form in the cluster growing method the critical and supercritical skeletons are observed in protein interaction networks and the world wide web respectively the distribution of box masses i e the number of vertices within each box follows a power law p m m ensuremath sim m ensuremath ensuremath eta the exponent ensuremath eta depends on the box lateral size ensuremath ell b for small values of ensuremath ell b ensuremath eta is equal to the degree exponent ensuremath gamma of a given scale free network whereas ensuremath eta approaches the exponent ensuremath tau ensuremath gamma ensuremath gamma ensuremath 1 as ensuremath ell b increases which is the exponent of the cluster size distribution of the random branching tree finally we study the perimeter h ensuremath alpha of a given box ensuremath alpha i e the number of edges connected to different boxes from a given box ensuremath alpha as a function of the box mass m b ensuremath alpha it is obtained that the average perimeter over the boxes with box mass m b is likely to scale as h m b ensuremath sim m b irrespective of the box size ensuremath ell b

impact of initial lattice structures on networks generated by traces of random walks abstract we show that the platform stage of network evolution plays a principal role in the topology of resulting networks generated by short cuts stimulated by the movements of a random walker the mechanism of which tends to produce power law degree distributions to examine the numerical results we have developed a statistical method which relates the power law exponent to random properties of the subgraph developed in the platform stage as a result we find that an important exponent in the network evolution is which characterizes the size of the subgraph in the form v t where v and t denote the number of vertices in the subgraph and the time variable respectively 2d lattices can impose specific limitations on the walker s diffusion which keeps the value of within a moderate range and provides typical properties of complex networks 1d and 3d cases correspond to different ends of the spectrum for with 2d cases in between especially for 2d square lattices a discontinuous change of the network structure is observed which varies according to whether is greater or less than 2 for 1d cases we show that emergence of nearly complete subgraphs is guaranteed by 1 2 although the transient power law is permitted at low increase rates of edges additionally the model exhibits a spontaneous emergence of highly clustered structures regardless of its initial structure

inferring decision trees using the minimum description length principle abstract we explore the use of rissanen s minimum description length principle for the construction of decision trees empirical results comparing this approach to other methods are given

a review of fractality and self similarity in complex networks we review recent findings of self similarity in complex networks using the box covering technique it was shown that many networks present a fractal behavior which is seemingly in contrast to their small world property moreover even non fractal networks have been shown to present a self similar picture under renormalization of the length scale these results have an important effect in our understanding of the evolution and behavior of such systems a large number of network properties can now be described through a set of simple scaling exponents in analogy with traditional fractal theory

covering number bounds of certain regularized linear function classes recently sample complexity bounds have been derived for problems involving linear functions such as neural networks and support vector machines in many of these theoretical studies the concept of covering numbers played an important role it is thus useful to study covering numbers for linear function classes in this paper we investigate two closely related methods to derive upper bounds on these covering numbers the first method already employed in some earlier studies relies on the so called maurey s lemma the second method uses techniques from the mistake bound framework in online learning we compare results from these two methods as well as their consequences in some learning formulations