on the fundamental limits of adaptive sensing suppose we can sequentially acquire arbitrary linear measurements of an n dimensional vector x resulting in the linear model y a x z where z represents measurement noise if the signal is known to be sparse one would expect the following folk theorem to be true choosing an adaptive strategy which cleverly selects the next row of a based on what has been previously observed should do far better than a nonadaptive strategy which sets the rows of a ahead of time thus not trying to learn anything about the signal in between observations this paper shows that the folk theorem is false we prove that the advantages offered by clever adaptive strategies and sophisticated estimation procedures no matter how intractable over classical compressed acquisition recovery schemes are in general minimal

active learning of multiple source multiple destination topologies we consider the problem of inferring the topology of a network with m sources and n receivers an m by n network by sending probes between the sources and receivers prior work has shown that this problem can be decomposed into two parts first infer smaller subnetwork components 1 by n s or 2 by 2 s and then merge them to identify the m by n topology we focus on the second part which had previously received less attention in the literature we assume that a 1 by n topology is given and that all 2 by 2 components can be queried and learned using end to end probes the problem is which 2 by 2 s to query and how to merge them with the given 1 by n so as to exactly identify the 2 by n topology and optimize a number of performance metrics including the number of queries which directly translates into measurement bandwidth time complexity and memory usage we provide a lower bound n 2 on the number of 2 by 2 s required by any active learning algorithm and propose two greedy algorithms the first algorithm follows the framework of multiple hypothesis testing in particular generalized binary search gbs the second algorithm is called the receiver elimination algorithm rea and follows a bottom up approach it requires exactly n 1 steps which is much less than all 2n possible 2 by 2 s simulation results demonstrate that both algorithms correctly identify the 2 by n topology and are near optimal but rea is more efficient in practice

a boundary element method for solving 2 d and 3 d static gradient elastic problems part i integral formulation abstract a boundary element formulation is developed for the static analysis of two and three dimensional solids and structures characterized by a linear elastic material behavior taking into account microstructural effects the simple gradient elastic theory of aifantis expressed in the framework of mindlin s general theory is used to model this material behaviour a variational statement is established to determine all possible classical and non classical due to gradient terms boundary conditions of the general boundary value problem the gradient elastic fundamental solution for both two and three dimensional cases is explicitly derived and used to construct the boundary integral representation of the solution with the aid of the reciprocal integral identity especially established for the gradient elasticity considered here it is found that for a well posed boundary value problem in addition to a boundary integral representation for the displacement a second boundary integral representation for its normal derivative is also necessary explicit expressions for interior displacements and stresses in integral form are also presented all the kernels in the integral equations are explicitly provided

minimax analysis of active learning this work establishes distribution free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes under various noise models the results reveal a number of surprising facts in particular under the noise model of tsybakov 2004 the minimax label complexity of active learning with a vc class is always asymptotically smaller than that of passive learning and is typically signi cantly smaller than the best previously published upper bounds in the active learning literature in high noise regimes it turns out that all active learning problems of a given vc dimension have roughly the same minimax label complexity which contrasts with well known results for bounded noise in low noise regimes we find that the label complexity is well characterized by a simple combinatorial complexity measure we call the star number interestingly we find that almost all of the complexity measures previously explored in the active learning literature have worst case values exactly equal to the star number we also propose new active learning strategies that nearly achieve these minimax label complexities

a reciprocity theorem in linear gradient elasticity and the corresponding saint venant principle in many practical applications of nanotechnology and in microelectromechanical devices typical structural components are in the form of beams plates shells and membranes when the scale of such components is very small the material microstructural lengths become important and strain gradient elasticity can provide useful material modelling in addition small scale beams and bars can be used as test specimens for measuring the lengths that enter the constitutive equations of gradient elasticity it is then useful to be able to apply approximate solutions for the extension shear and flexure of slender bodies such approach requires the existence of some form of the saint venant principle the present work presents a statement of the saint venant principle in the context of linear strain gradient elasticity a reciprocity theorem analogous to betti s theorem in classic elasticity is provided first together with necessary restrictions on the constitutive equations and the body forces it is shown that the order of magnitude of displacements are in accord with the sternberg s statement of the saint venant principle the cases of stretching shearing and bending of a beam were examined in detail using two dimensional finite elements the numerical examples confirmed the theoretical results 2005 elsevier ltd all rights reserved

the condition number of real vandermonde krylov and positive definite hankel matrices we show that the euclidean condition number of any positive definite hankel matrix of order n geq 3 may be bounded from below by gamma n 1 16n with gamma exp 4 cdot it catalan pi approx 3 210 and that this bound may be improved at most by a factor 8 gamma n similar estimates are given for the class of real vandermonde matrices the class of row scaled real vandermonde matrices and the class of krylov matrices with hermitian argument improved bounds are derived for the case where the abscissae or eigenvalues are included in a given real interval our findings confirm that all such matrices including for instance the famous hilbert matrix are ill conditioned already for moderate order as application we describe implications of our results for the numerical condition of various tasks in numerical analysis such as polynomial and rational i nterpolation at real nodes determination of real roots of polynomials computation of coefficients of orthogonal polynomials or the iterative solution of linear systems of equations

generalized density clustering we study generalized density based clustering in which sharply defined clusters such as clusters on lower dimensional manifolds are allowed we show that accurate clustering is possible even in high dimensions we propose two data based methods for choosing the bandwidth and we study the stability properties of density clusters we show that a simple graph based algorithm successfully approximates the high density clusters

information based objective functions for active data selection learning can be made more efficient if we can actively select particularly salient data points within a bayesian learning framework objective functions are discussed that measure the expected informativeness of candidate measurements three alternative specifications of what we want to gain information about lead to three different criteria for data selection all these criteria depend on the assumption that the hypothesis space is correct which may prove to be their main weakness

mixed finite element formulations of strain gradient elasticity problems theories on intrinsic or material length scales find applications in the modeling of size dependent phenomena in elasticity length scales enter the constitutive equations through the elastic strain energy function which in this case depends not only on the strain tensor but also on gradients of the rotation and strain tensors in the present paper the strain gradient elasticity theories developed by mindlin and co workers in the 1960s are treated in detail in such theories when the problem is formulated in terms of displacements the governing partial differential equation is of fourth order if traditional finite elements are used for the numerical solution of such problems then c 1 displacement continuity is required an alternative mixed finite element formulation is developed in which both the displacement and the displacement gradients are used as independent unknowns and their relationship is enforced in an integralsense a variational formulation is developed which can be used for both linear and non linear strain gradient elasticity theories the resulting finite elements require only c 0 continuity and are simple to formulate the proposed technique is applied to a number of problems and comparisons with available exact solutions are made 2002 elsevier science b v all rights reserved

hierarchical testing designs for pattern recognition we explore the theoretical foundations of a twenty questions approach to pattern recognition the object of the analysis is the computational process itself rather than probability distributions bayesian inference or decision boundaries statistical learning our formulation is motivated by applications to scene interpretation in which there are a great many possible explanations for the data one background is statistically dominant and it is imperative to restrict intensive computation to genuinely ambiguous regions the focus here is then on pattern filtering given a large set mathcal y of possible patterns or explanations narrow down the true one y to a small random subset widehat y subset mathcal y of detected patterns to be subjected to further more intense processing to this end we consider a family of hypothesis tests for y a versus the nonspecific alternatives y ac each test has null type i error and the candidate sets a subset mathcal y are arranged in a hierarchy of nested partitions these tests are then characterized by scope a power or type ii error and algorithmic cost we consider sequential testing strategies in which decisions are made iteratively based on past outcomes about which test to perform next and when to stop testing the set y is then taken to be the set of patterns that have not been ruled out by the tests performed the total cost of a strategy is the sum of the testing cost and the postprocessing cost proportional to y and the corresponding optimization problem is analyzed as might be expected under mild assumptions good designs for sequential testing strategies exhibit a steady progression from broad scope coupled with low power to high power coupled with dedication to specific explanations in the assumptions ensuring this property a key role is played by the ratio cost power these ideas are illustrated in the context of detecting rectangles amidst clutter

optimal aggregation of classifiers in statistical learning classification can be considered as nonparametric estimation of sets where the risk is defined by means of a specific distance between sets associated with misclassification error it is shown that the rates of convergence of classifiers depend on two parameters the complexity of the class of candidate sets and the margin parameter the dependence is explicitly given indicating that optimal fast rates approaching o n 1 can be attained where n is the sample size and that the proposed classifiers have the property of robustness to the margin the main result of the paper concerns optimal aggregation of classifiers we suggest a classifier that automatically adapts both to the complexity and to the margin and attains the optimal fast rates up to a logarithmic factor

wedgelets nearly minimax estimation of edges we study a simple horizon model for the problem of recovering an image from noisy data in this model the image has an edge with fi holder regularity adopting the viewpoint of computational harmonic analysis we develop an overcomplete collection of atoms called wedgelets dyadically organized indicator functions with a variety of locations scales and orientations the wedgelet representation provides nearly optimal representations of objects in the horizon model as measured by minimax description length we show how to rapidly compute a wedgelet approximation to noisy data by flnding a special edgelet decorated recursive partition which minimizes a complexity penalized sum of squares this estimate using su cient sub pixel resolution achieves nearly the minimax mean squared error in the horizon model in fact the method is adaptive in the sense that it achieves nearly the minimax risk for any value of the unknown degree of regularity of the horizon 1 fi 2 wedgelet analysis and de noising may be used successfully outside the horizon model we study images modelled as indicators of star shaped sets with smooth bound aries and show that complexity penalized wedgelet partitioning achieves nearly the minimax risk in that setting also

rates of convergence in active learning we study the rates of convergence in generalization error achievable by active learning under various types of label noise additionally we study the general problem of model selection for active learning with a nested hierarchy of hypothesis classes and propose an algorithm whose error rate provably converges to the best achievable error among classifiers in the hierarchy at a rate adaptive to both the complexity of the optimal classifier and the noise conditions in particular we state sufficient conditions for these rates to be dramatically faster than those achievable by passive learning

rademacher complexities and bounding the excess risk in active learning sequential algorithms of active learning based on the estimation of the level sets of the empirical risk are discussed in the paper localized rademacher complexities are used in the algorithms to estimate the sample sizes needed to achieve the required accuracy of learning in an adaptive way probabilistic bounds on the number of active examples have been proved and several applications to binary classification problems are considered

information based complexity feedback and dynamics in convex programming we study the intrinsic limitations of sequential convex optimization through the lens of feedback information theory in the oracle model of optimization an algorithm queries an oracle for noisy information about the unknown objective function and the goal is to approximately minimize every function in a given class using as few queries as possible we show that in order for a function to be optimized the algorithm must be able to accumulate enough information about the objective this in turn puts limits on the speed of optimization under specific assumptions on the oracle and the type of feedback our techniques are akin to the ones used in statistical literature to obtain minimax lower bounds on the risks of estimation procedures the notable difference is that unlike in the case of i i d data a sequential optimization algorithm can gather observations in a controlled manner so that the amount of information at each step is allowed to change in time in particular we show that optimization algorithms often obey the law of diminishing returns the signal to noise ratio drops as the optimization algorithm approaches the optimum to underscore the generality of the tools we use our approach to derive fundamental lower bounds for a certain active learning problem overall the present work connects the intuitive notions of information in optimization experimental design estimation and active learning to the quantitative notion of shannon information

bisection search with noisy responses bisection search is the most efficient algorithm for locating a unique point x 0 1 when we are able to query an oracle only about whether x lies to the left or right of a point x of our choosing we study a noisy version of this classic problem where the oracle s response is correct only with probability p the probabilistic bisection algorithm pba introduced by horstein ieee trans inform theory 9 1963 pp 136 143 can be used to locate x in this setting while the method works extremely well in practice very little is known about its theoretical properties in this paper we provide several key findings about the pba which lead to the main conclusion that the expected absolute residuals of successive search results i e e x xn converge to 0 at a geometric rate

sequential methods for design adaptive estimation of discontinuities in regression curves and surfaces in fault line estimation in spatial problems it is sometimes possible to choose design points sequentially by working one s way gradually through the response plane rather than distributing design points across the plane prior to conducting statistical analysis for example when estimating a change line in the concentration of resources on or under the sea bed individual measurements can be particularly expensive to make in such cases equential design adaptive methods are attractive appropriate methodology is largely lacking however and the potential advantages of taking a sequential approach are unclear in the present paper we address both these problems we suggest a methodology based on sequential refinement with reassessment that relies upon assessing the correctness of each sequential result and reappraising previous results if significance tests show that there is reason for concern we focus part of our attention on univariate problems and we show how methods for the spatial case can be constructed from univariate ones

minimax bounds for active learning this paper analyzes the potential advantages and theoretical challenges of active learning algorithms active learning involves sequential sampling procedures that use information gleaned from previous samples in order to focus the sampling and accelerate the learning process relative to passive learning algorithms which are based on nonadaptive usually random samples there are a number of empirical and theoretical results suggesting that in certain situations active learning can be significantly more effective than passive learning however the fact that active learning algorithms are feedback systems makes their theoretical analysis very challenging this paper aims to shed light on achievable limits in active learning using minimax analysis techniques we study the achievable rates of classification error convergence for broad classes of distributions characterized by decision boundary regularity and noise conditions the results clearly indicate the conditions under which one can expect significant gains through active learning furthermore we show that the learning rates derived are tight for boundary fragment classes in d dimensional feature spaces when the feature marginal density is bounded from above and below

bending and stability analysis of gradient elastic beams abstract the problems of bending and stability of bernoulli euler beams are solved analytically on the basis of a simple linear theory of gradient elasticity with surface energy the governing equations of equilibrium are obtained by both a combination of the basic equations and a variational statement the additional boundary conditions are obtained by both variational and weighted residual approaches two boundary value problems one for bending and one for stability are solved and the gradient elasticity effect on the beam bending response and its critical buckling load is assessed for both cases it is found that beam deflections decrease and buckling load increases for increasing values of the gradient coefficient while the surface energy effect is small and insignificant for bending and buckling respectively

on nonparametric estimation of density level sets let x 1 x n be independent identically distributed observations from an unknown probability density f consider the problem of estimating the level set g g f x r 2 f x from the sample x 1 x n under the assumption that the boundary of g has a certain smoothness we propose piecewise polynomial estimators of g based on the maximization of local empirical excess masses we show that the estimators have optimal rates of convergence in the asymptotically minimax sense within the studied classes of densities we find also the optimal convergence rates for estimation of convex level sets a generalization to the n dimensional case where n 2 is given

bayesian minimax estimation of the normal model with incomplete prior covariance matrix specification this work addresses the issue of bayesian robustness in the multivariate normal model when the prior covariance matrix is not completely specified but rather is described in terms of positive semi definite bounds this occurs in situations where for example the only prior information available is the bound on the diagonal of the covariance matrix derived from some physical constraints and that the covariance matrix is positive semi definite but otherwise arbitrary under the conditional gamma minimax principle previous work by dasgupta and studden shows that an analytically exact solution is readily available for a special case where the bound difference is a scaled identity the goal in this work is to consider this problem for general positive definite matrices the contribution in this paper is a theoretical study of the geometry of the minimax problem extension of previous results to a more general case is shown and a practical algorithm that relies on semi definite programming and the convexity of the minimax formulation is derived although the algorithm is numerically exact for up to the bivariate case its exactness for other cases remains open numerical studies demonstrate the accuracy of the proposed algorithm and the robustness of the minimax solution relative to standard and recently proposed methods

active learning using smooth relative regret approximations with applications the disagreement coeffcient of hanneke has become a central data independent invariant in proving active learning rates it has been shown in various ways that a concept class with low complexity together with a bound on the disagreement coeffcient at an optimal solution allows active learning rates that are superior to passive learning ones we present a different tool for pool based active learning which follows from the existence of a certain uniform version of low disagreement coeffcient but is not equivalent to it in fact we present two fundamental active learning problems of significant interest for which our approach allows nontrivial active learning bounds however any general purpose method relying on the disagreement coeffcient bounds only fails to guarantee any useful bounds for these problems the applications of interest are learning to rank from pairwise preferences and clustering with side information a k a semi supervised clustering the tool we use is based on the learner s ability to compute an estimator of the difference between the loss of any hypothesis and some fixed pivotal hypothesis to within an absolute error of at most e times the disagreement measure l1 distance between the two hypotheses we prove that such an estimator implies the existence of a learning algorithm which at each iteration reduces its in class excess risk to within a constant factor each iteration replaces the current pivotal hypothesis with the minimizer of the estimated loss difference function with respect to the previous pivotal hypothesis the label complexity essentially becomes that of computing this estimator

the geometry of generalized binary search this paper investigates the problem of determining a binary valued function through a sequence of strategically selected queries the focus is an algorithm called generalized binary search gbs gbs is a well known greedy algorithm for determining a binary valued function through a sequence of strategically selected queries at each step a query is selected that most evenly splits the hypotheses under consideration into two disjoint subsets a natural generalization of the idea underlying classic binary search this paper develops novel incoherence and geometric conditions under which gbs achieves the information theoretically optimal query complexity i e given a collection of n hypotheses gbs terminates with the correct function after no more than a constant times logn queries furthermore a noise tolerant version of gbs is developed that also achieves the optimal query complexity these results are applied to learning halfspaces a problem arising routinely in image processing and machine learning

smoothness disagreement coefficient and the label complexity of agnostic active learning we study pool based active learning in the presence of noise that is the agnostic setting it is known that the effectiveness of agnostic active learning depends on the learning problem and the hypothesis space although there are many cases on which active learning is very useful it is also easy to construct examples that no active learning algorithm can have an advantage previous works have shown that the label complexity of active learning relies on the disagreement coefficient which often characterizes the intrinsic difficulty of the learning problem in this paper we study the disagreement coefficient of classification problems for which the classification boundary is smooth and the data distribution has a density that can be bounded by a smooth function we prove upper and lower bounds for the disagreement coefficients of both finitely and infinitely smooth problems combining with existing results it shows that active learning is superior to passive supervised learning for smooth problems