do argue is happening in in a deep neural network do you not believe that well as soon as you have high dimensionality settings then it does not happen almost surely and i mean you could argue so for example let's say you take a gun or any big generative network right in the latin space you have hundreds of dimensions and you sample those guys from a gaussian distribution so even if you were saying okay your training set is sampled from a generative model and your test set is sampled from the same generative model and in that latent space you have interpolation well it's wrong from the beginning because in that latin space alone you will never have a gaussian sample that lies in the interpolation regime as soon as the dimensionality is greater than let's say NUMBER_TOKEN or NUMBER_TOKEN. so it does not happen because of the dimensionality unless you have very degenerate things of course if your generative network just spits out the constant then in pixel space you will have interpolation but this is degenerate by definition so i'd like to i'd like to pick up on that if you don't mind yeah so what i want to pick up on is again all of this hinges on definition one which is in the paper which is membership within this within this convex hall and there's a there's a sense in which that's an extremely rigid definition of interpolation right and and i think i heard i think i heard you earlier say that what we need to do is redefine what we mean by interpolation and higher dimension because the intuition of interpolation is still correct but we need to redefine what we mean by interpolation so if you if you took a shot at redefining interpolation how would you define it what do you think is a better definition of interpolation so i think it's a very task dependent so let's say you want to look at a task which is pure object classification in that case i think going back to what we were saying earlier on first trying to compress some all your data so that you can explain it with as little factors of variation as possible and then you can use the current definition just on a compression of your data then it could make sense because you don't really mind about the finicky details of your objects if you just want to differentiate between different classes of objects but in another setting where you might be trying i don't know today noise samples or things like that then you might want to have a very more specific definition based on what you try to denoise in your data and so on so i don't think there will be a general definition that works across the bone it will be really dependent on the type of task you are looking down downstream it's it's really it's really the key let me throw something out there because i've been thinking about this and i tried to ask uh professor lacoon about this but i didn't quite i didn't communicate it clearly which is that for something to be in the convex hall so in in dimensions you know for something to be within you know a convex hull it's a necessary condition that on every single dimension uh the sample data point lies within the range that was sampled so that's a necessary but not sufficient condition because that that's
NO_SEGMENT_TOKEN