Voronoi tiles and we'll get into these as we go so one of the yeah key ideas is linking these modern deep net ideas back to more classical signal processing ideas so let's just do a toy example so that you can visualize what goes on in one of these vector quantization partitions of the input spaces so let's just consider a toy example a three-layer net we go from an input space it's two-dimensional we're gonna four classes in the two dimensional input 2d just so we can visualize can't visualize really anything beyond 3d we


go to a layer of 45 units then three units and then we're gonna do a four class classification problem so we have four units on the output okay makes sense to everybody so this is the inputs this is what goes on in the input space we have four different classes with these four colors our goal is to build a classifier to to tell these apart this is the first axis of the input space the second axis and this is what happens after we go through the first layer right the we go to 45 this the layer the Maslow layer that map's the

Mad Max: Affine Spline Insights into Deep Learning


input to the output of this first layer this is the vector quantization partition or the spline partition that you that that that you obtain importantly we're going through a single layer so the tiling is convex right these are convex regions right makes sense okay moreover let's remember that these are splines after all so we can ask what what is the mapping from the input of the first layer to the output of the first layer well it's just a very simple affine map because it's an affine spline after all okay that once you know


a signal right a particular ax lands in this particular tile right there is a that that gives you a particular matrix a right and a offset vector B right that are different for every VQ tile but then the mapping for the input to the output of that layer is just simply this affine map so you can think of a the mapping from the input to the output of one deep knit layer is just a VQ dependent affine transformation so this is one layer so now if we go through two layers and we think of the partition induced on the


input space we now see that we start picking up non-convex or we start having non convex regions because the non-convex operator however we still have the same concept right that if a signal falls in this particular tile right this particular partition region the mapping from the input to the output of the second layer remains just simply in affine map right where the a and the B are indexed by this particular tile and just to be SuperDuper clear about it one more time every signal that lives in this tile that falls in this tile on the


input space has the exact same f-fine mapping okay and this is what happens when you learn just to see when you if you initialize with random random weights zero biases you just get a set of cones and as we go through learning epochs you see that we end up with these cones pulling away from the origin and then cones being cut up by other cones and we result again and for at least layers wanted to this particular mapping it at convergence okay and I'm gonna III think that it's really thinking of this geometrical


picture is really very useful to think about the inner workings of what's going on in a in a deep network in particular a deep net is a VQ mer machine right it's computing a vector quantization so was their question yeah we said oh let me just think if I got this right we set all the biases to zero in the whole in the whole network yeah so we'll just still it's still there's no beat there's no box set so it's just gonna remain yeah calling the corners of Collins is just calm that makes sense okay good so let's talk a little bit about some of


the geometrical properties you can actually delve deeper into the what the structure of these VQ tiles and show that that the part of the partition of each of a single layer right a single layers input space in terms of the output is something called it's actually not a Voronoi diagram it's something called power diagram question anybody here heard of power diagrams okay fantastic so it's a generalization of a Voronoi diagram now instead of just having a centroid it has a centroid and a radius all right so it's a it's a mild


generalization of a Voronoi tiling but the basically you just compute a Voronoi tiling but with something called a genre distance instead of the standard Euclidean distance but the tiles remain convex convex polytopes right and in high dimensional space moreover given these affine maps given the the entries in these a matrices and these B bias vectors we have there they're close form formulas for the centroids and the radii that determine all of these polytopes so you can understand you can study the the geometry of these the the eccentricity


the size etc by thanks to the in closed form thanks to these formulas moreover it should be pretty clear that since you're piling layers upon layers that the the pop the power diagram formed from let's say two Mazal layers applied in composition is going to be formed by a subdivision process because the cutting of the cuts from the second layer input of the second layer to the output will basically cut the vq tiling from the first layer right and so this is just an example of an input space tiling first layer will just be a set of


straight line cuts the second layer is going to be a subdivision of those cuts we colored them gray here subdivision but now the important thing is that the cuts are going to be bent right they're going to be bent at the gray bond arees which are the boundaries defined by the first layer cuts and these by these bends you can actually compute bounds for example on the dihedral angles and and these bends are precisely two main continent to maintain continuity of the mapping from the input to the output of this operator if you didn't have these


bending then you could have the spline become non continuous okay but again these bends are very important and they have a lot to do with weight sharing in deep deep networks so one of the conclusions you can just take away from this part partway through is that deep networks are really a very practical very efficient way of doing something that is extremely difficult which is free not spline approximation and Heidi all right that's that's that's really what deep deep networks are doing you could carry this all the way to the last layer in a


classification problem say it study the decision boundary of the deep net it is again just going to be one of these basically just one of these cuts and you can understand for example the the smoothness of the boundary by the fact that the you can you can only have so much bending between the cuts when you cut through the power diagram partition that you obtained from the previous regions there's lots of things that can be done to understand for example smoothness of the decision boundaries in different kinds


of deep Nets this is one direction that we've been exploring the other is looking in particular at these affine mappings so again when you're in in a VQ tile you know that there's for all signals that live in that tile there's just a fine map that goes from the input to the output what what what what properties can we do for me glean from these okay so in particular if we think let's just study the the simple the the case of input to the output of the entire network okay which we'll call this Z big L you can


ignore the softmax that doesn't really enter into any of the the discussions I'm gonna bring up but we're interested in the mapping through all the layers of the network the this affine mapping formula applies no matter where you are in the network you'll just have different A's and B's but we're interested in the one from the input to the very output okay from the input to the very output well you can develop closed-form formulas for this map particularly for a continent this is what the a the a matrix looks like this


is what this offset B looks like we can know all of these matrices here in close form so you can do different kinds of analyses for example look at the Lipschitz stability two different points in the network based on different inputs but the thing I'm most interested in talking about here is what what are the rows of this a matrix look like because if you think about this what is the output of the deep net right everything up until the the softmax well it's basically just a matrix a multiplied just ignore this typo it's


this matrix a multiplied by my signal ax plus a bias well that then that this is just a vector how big is this vector is one output for every class right I and what how do I determine which class the input is it's whichever of these outputs is largest right okay so let's think we have a matrix a that we're multiplying by our X each entry in this output is what just a inner product of a row of a with X so what is that right if we think about this matrix a well the C throw right course buying a Class C is dist we're just going to take


the inner product of the C throw with X in order to get the C output of Z we want to find the biggest what's the what's the the what do we call this in signal processing nomenclature we call this a match filter bank right because basically what we're doing is we're applying to our signal a set of filters by inner products cauchy-schwarz tells us that the more the filter looks like the input the larger the output is going to be okay and the optimal filter is what where the row is exactly the input right standard standard stuff that's


done in you know radar signal processing sonar walk communication systems etc and you can actually visualize this in a real network so this is just see far ten here's an input of an airplane here's the row this that this is the the row of the corresponding a matrix for that input vector unvectorized so that it looks like an image see it looks if you squint it looks a lot like an airplane okay I have a large inner product if you look at these other rows corresponding ship class dog class you see they don't actually look like a ship or a dog but


more like a anti-air plane all right in order to push down the inner product largest inner product smallest even smaller inner product in fact I yes sir I didn't talk about the bias but the way to think about the bias is if if you're a Bayesian then the B's would be related to the prior probabilities of the different classes so if you knew that that planes were very very likely you would put you would load B with a large number in the beef the beef entry that make sense yeah yeah Renee yeah so it's subtle it's subtle


but you could think of this as like a dictionary learning machine that's basically given an input is is is defining then a bit of Bayesian classifier does that help a little bit okay so and of course if you if you think what these rows of these a matrix matrices are and you think of the fact that we're decomposing the deep net input output relationship in terms of affine maps there's a just a direct link between the rows of this a matrix and what are called saliency maps by by the the community so it gives new intuition


behind what goes on or what happens when we when we think about salience email moreover if you if you if you you can prove a simple result that says if you have a high capacity deep net that's capable of producing basically any and arbitrary a matrix if you will from a from a given input then you can show that the Seath role of the a matrix when you input a piece of training data xn is going to become exactly xn when you're on the true class right X is late X ends label and essentially minus a constant times xn when you are not in when you're


in a different class and so this is this will tell us a little bit both about again reinforcing this match filter interpretation but also helping us understand a little bit about this memorization memorization process okay a couple more a couple more points so another thing we can do is we can think now because we under we can have formulas for these affine Maps we can characterize the prediction function f that map's the input to the output and we can think of different kind of complexity measures that we can derive


out of these affine mapping formulas so there's a lot of applications for complexity measures for example you might want to compare two deep networks one which has a very complicated prediction function the other that solves the same task but has a much simpler prediction function Occam's razor type idea we might also want to apply a complexity measure as a penalty directly to our to our learning process right so there's a large literature of deriving different complexity measures and complexity penalties for deep nets all this point


to you know two of them as examples one is there's a very nice recent paper that that links the the ubiquitous to norm to norm of the weights penalty for learning to a particular measure of the second derivative of the prediction function all right so that it really does say that for at least a very very simple kind of network there's a link between the weight values of the weights and the wiggliness of s and then there's another school of approaches that looks at well we have a VQ tiling of the input space let's count the number of tiles because


presumably the more codes there are in your code book the more tiles there are the more complicated the function that you're trying to approximate so these are two approaches well I'm going to get one that really expands upon these two and it it is leveraging the fact that we can was leveraging really the the fact that lots of data sets of particular image type data sets we have a reasonable reasonably true property that the the the training data live on low lowered that doesn't that be low dimensional but lower dimensional


manifold sub manifolds of the high dimensional space okay so let's assume that our data lives not filling up the entire input space but living on some lower dimensional sub manifold or sub manifolds and in this case we can we can look into the manifold learning literature and there's a beautiful paper by Donna Hahn crimes that defines what's called the Hessian eigen map manifold learning technique which is is basically trying to flatten a curvy manifold using the tangent a CN along the manifold so we can just a dot this this same measure


and we can define a complexity measure C as the integral of the of the tangent hessian along the manifold so you can just think of it roughly speaking is the low look you have F it's a peacefull at the continuous piecewise defined function and what we're looking at is the local deviation of F from flat so f if f of X was a completely flat function this measure would be 0 if F was very very jagged II meaning locally when you look over a few regions it's jumping up and down wildly this will be a large this will be a large number yes well


simply by integrating along this basically we're just integrating along the tangent manifold part yes yes yeah and you could also you know just integrate over the entire space but then you lose some of the nice some of the nice properties did that help oh yeah and we could talk we can talk about after so the nice thing about this measure is that you can develop a marker a Monte Carlo approximation in terms of the training data points the X ends that are your training data point and the affine mapping parameter so it's


actually extremely easy to actually compute the value of C given a set of trained training points and giving the affine mapping parameters no I won't left it just think of P is true for right now so it's all the ideally you will choose the P depending on particular for example the manifold dimension the ambient dimension week yeah let's talk about it at the break this is a date let's the data manifolds assume the training data or samples from some sub manifold in the in the ambient space because there are two factors that


can increase C one is f there is the many other you mean the smoothness of the manifold say yeah absolutely but but if if we just assume let's just say you have to to prediction functions and their domain in both cases is the same manifold and that would be normalized out right yes yeah are you no longer working with values here this is where rail is absolutely or piecewise peaceful a convex piecewise a fine nonlinear fashion then zero everywhere I know yeah let's talk about let's talk about it offline I think otherwise I'll run out of time


yeah well it won't be yeah okay so let's let's look at an application of this to something that is I would say somewhat still mysterious in the deep learning world and that is data the data augmentation so if we think of how deep networks are trained today they're typically trained with this technique called data augmentation that we don't just feed in images of been right we feed in images of translates right of been rotations have been etc and if we have a hypothesis that the images have been somehow came from a lower


dimensional manifold in high dimensional space where points on that manifold were translates and rotations of each other then the it's very convenient that these augmented data points that are generated from just your raw initial training data will live on the same data manifold okay so there's a in this particular setting you can you can prove a result that says that just starting with date writing out the the cost of say cross-entropy cost function with data augmentation terms you can actually develop that those data


augmentation terms and pull them out of the first part of the cost function and show that they basically form a this Hessian penalty this Hessian complexity regularization penalty okay so what that's saying is that data augmentation implicitly implements a hessian complexity regularization on the optimization okay so that's like the theorem here too or just a simple experiment with the CFR 100 data that so this is training epochs in the x-axis this complexity measure in the vertical axis and all we're doing here


is we are as we're training the network trained trained without data augmentation in black and with data augmentation in blue what we're looking at are based on the A's and the B's that we have the that we have learned with the network we're plugging that into our complexity measure that was on the previous previous slide and we're seeing that the measure is showing that the network that has learned using data augmentation has far lower complexity than the network that has learned with data augmentation this is both on the


training and on the test data yeah we're doing the station depends on the loss and is these anticipation arising due to two square lows oh yeah good question well in this case we it was it was cross entropy loss it wasn't that wasn't weird work with rotations of two similar images have the same label yeah so the loss in comparative labels and so that has to change the regularization has to change to the same yeah let's yeah so so for just for now the interest of time let's just assume cross-entropy loss for a classification


problem rather than l2 l2 loss for a regression problem and then let me think and maybe I'll have a better answer by the time that we get to question yeah complex measure is only a function of the model but not a lot is that right absolutely yes - okay so let's one last quick quick note what can we do beyond piecewise defined deep nets because sigmoid hyperbolic tanh these are still very useful in certain applications in particular in recurrent deep nets and it turns out that you can bring those under the same umbrella that of this max


affine spline framework and the way to do that is to switch from a deterministic hard vector quantization approach or way of thinking where if X lives in a in this particular vector quantization tile it definitively lives in that vector quantization tile to a soft VQ approach where now we just we have a probability that X will fall in a given vector quantization tile where for this particular signal maybe there's a high probability in this tile somewhat smaller in the local local region a burring tiles and then decreasing


probability as you move away so if you just set up a very simple Gaussian mixture model where the means and covariances are based on these A's and B's that we derive you can you can basically derive nonlinearities like the sigmoid like the softmax directly from Ray Lu absolute value and other piecewise defined convex nonlin non-linearity's and in particular if you if you do a look at a hybrid approach it's this between a hard vq and a soft vq alright with where you're basically blending between the two you can


generate infinite classes of interesting and potentially useful nonlinearities and I'll just point out one how many people here have heard of the swish non-linearity a few so this was a non-linearity that was discovered a few years ago through an empirical search that's the empirical search for is there a normal in the area that works better than Ray Lu right for large-scale classification problems and it turned out there there there was and it was a fairly sizable you know non-trivial gain in a lot of cases and it's this black


dashed line here and the interesting thing it's hard to know if it's a coincidence or not but if you look at the in some sense the midway point between hard VQ and South vq i based on the the Ray Lu function at the hard VQ side and the sigmoid gated linear unit at the soft vq the swish is precisely halfway in between it's quite quite ok you could also pull out sigmoid hyperbolic tangent by adopting a probabilistic viewpoint of the output of a layer no longer being just a deterministic output of the input but


instead the probabilities that you fall in the different VQ regions in the input that's what we can do beyond piecewise so I better wrap up so what I hope to get across is that this spline in particular max affine splined viewpoint can provide a useful language to talk about the things that we're learning about deep networks but also frame the kind of questions that we would like to move forward with my talked a bit about the the basic you know framework of max affine splines and deep nets I talked about the relationships with with vector


coin sation and really that a deep net is you could think of it as a vector quantization machine or you could think of it as a free not spline machine there's there really I think interesting links between power diagrams from computational geometry and the the subdivision that's generated by this layer upon layer max affine spline process this the affine transformations that we derive based on that these difficulties different vq regions allow us to link deep nets back to old-school signal processing ideas like match


filter banks and they allow us to define new kinds of complexity measures in particular this Hessian measure that we talked about it's all in there and say there's some some papers that people would like to take a peek and I'd be happy to answer any additional questions [Applause] it's all really a question the second derivative of that one so yeah so the basically the the way the way that we think about it is that you have a it's okay there's there's let's stop it to hear a heuristic way of thinking about it is if you had a piecewise-defined


function it's gonna be undefined that obviously kinks right but if you thought if you think of basically here some heuristic way to smooth any heuristic way you can think of to smooth out that kink then the second derivative is going to be related to the slope parameters on one side and the soul parameters on the other side with yeah exactly and the bandwidth of this movie that's the epsilon that was in that in that formula there's a there there details that we could you know we could talk about the brain yes


[Music] you mean the bat like how large the measure yeah so that was what this yeah that the point of this experiment was really to look at ads as we are training so as we're going through training affects what is happening you know what is the value of in this case the value of this complexity measure as we train the network through the various training training cycles both with in this case with data augmentation and in this case without data augmentation does that make sense no you just so it just in a nutshell


think of think of a Gaussian mixture model a Gaussian mixture model defined in terms of covariance means and covariances we're now the means and covariances are defined in terms of these different tiles pardon me so okay what's the best way to describe it so start from start from a hard VQ where we have a tiling now you have because of the power diagram you have a radius and you have a centroid now use that radius in that centroid as the per to develop a Gaussian mixture model for example a you know circle a symmetric Gaussian mixture


with that particularly the radius being the the variance now under that if you think in terms of now under that model you'll have a given an input you can think about the probability that it falls into each of these individual tiles will be determined by the probability under that particular each of those mixtures does that make sense and if you put if you now look at these probabilities these probabilities behave like in the case of say you start with the tiling derive from array Lu you will you will end up with set of


probabilities that are that that follow the functional form of a sigmoid gated linear unit in that case does that help that is not funny it's already trained you can mean there is no time right oh okay okay you know you I my question is I assume there is a procedure to arrive so can we go backwards you're saying I need to think about that yeah we were thinking only in the one in one direction going from a hard v q to a soft vq presumably you could reverse that process by if you have a certain kind of but it would have


to be not all possible nonlinearities are reachable via this saw this this soft our relaxation of this hard v q right you can't reach you can't reach arbitrary you can't reach arbitrary nonlinearities right only certain kind of nonlinearities if you wanted to reach arbitrary ones you would have to there's no way you could do that with just a ghost standard kind of Gaussian mixture framework that help yeah yes go back to the contacts to make sure it was sure or are we here it's a very good question um so it sounds like a


blood test sorry for the low D a after one and fifty ipok there's no number it is just because of a proton so that's something happen like oh oh sorry sorry yeah no we jet this is an artifact of the plotting we shall probably should have stopped a plot here yeah yes like a total number of examples yeah so the that's a really good question because in fact we're we're only the way that we compute okay there's the complexity measure and then there's the computation of the approximation of the complexity measure the more training


samples you have the more densely you will sample the manifold the the signal manifold and that closer your approximation will be to the true the true measure and so the more training data that you have the closer you'll get to the true measure yeah that's a really good question so there are methods that attempt to do adaptive yawn selection movies like Trent oh yeah and you can apply it with some on occasions to multivariate data and get adaptive finally yeah is there some sense of so it would give you similar results it's right hierarchical


VQ would be another yeah yeah but is there some sense about the properties of these quantizations and these timings are that will differentiate you from something that's that more directly tries to penalize is that to me this is this is the this is this is a really key key unanswered question right what the question was there that there are you know there are a number of different ways to try to find different free not spline approximations in higher dimensions and why are deepening of the Oh Howard deep nets bet you know


different or better and this this is a big question I answered question there there's no question that the methods that we use the you know current training method our optimization approaches are enabling us to find these free not spline approximations in there truly ridiculously high dimensions right where a lot of these other techniques you wouldn't even attempt to them right but still it does not mean that this is the that we that we've stumbled on the best way of doing this so I think that as we think of new kinds of optimization


[Music] all right thank you very much for the introduction and I hope you had a nice lunch and welcome to this talk about self supervised deep learning towards autonomously learning machines as you already heard my name is Simon T ballina I'm head of yard craft works and also lectured a couple of universities and craft works well you might have heard about us we are a vienna based artificial intelligence and big data company specializing in solving hard industrial problems using artificial intelligence and of course data also

Self-Supervised Learning - Towards Autonomously Learning Machines—Simon Stiebellehner


most of our clients come from the industry ranging from the automotive sector all the way to the energy sector and pretty much everything in between the topic of this talk self supervised deep learning is also primarily motivated by the work we do with our clients Before we jump right into the topic we first need to lose a few words about the current state of artificial intelligence though probably as most of you are aware of AI has come a pretty long way in the last couple of years in fact we have made massive progress for


example when you think of popular voice assistance such as Syria or Cortana they have become incredibly good at understanding human natural language just in the last couple of years similarly in a completely different area of artificial intelligence in computer vision we can now highly accurately segment even complex images into all of its part parts and not only that we can even automatically generate text that describes what's happening in an image so called scene understanding so obviously great great progress has been


made when it comes to artificial intelligence just in the last year's and it turns out that many of these breakthroughs many of these things you read about in the news are actually based on something that's called supervised deep learning and this brings us to the first part of this talk supervised deep learning the good and the ugly we will find - I supervise deep learning is so great and so many breakthroughs are based on it but also we will get to know it's very ugly side one of its major downsides one of its major weaknesses


subsequently we will find out how taking a self supervised approach to deep learning can help us overcome or at least mitigate that core weakness and finally we will look into an industry case that hopefully gives you a good idea of how you can use self supervision in practice to make your models better and more robust but first let's look into supervised learning probably some of you or maybe if many of you know this data set it's an incredibly popular famous data set even containing images of obviously cats and dogs and this data


set is used by data scientists around the world to do the first steps in image classification using especially deep learning so usually the task at hand is building a classified a differentiates between cats and dogs based on images theoretically you could approach this problem from two sides you could approach it from an unsupervised side unsupervised learning we do not use labels right and in supervised learning this would be the other side we do use labels so we do use textual or numeric information that tells us if there


really is a cat or if there really is a dog in that image just a quick recap to bring us on the same page in unsupervised learning as I mentioned we don't use labels we just tried to detect similarities and dissimilarities in these images and based on that form homogenous clusters that hopefully so we end up with a cat and the dog cluster of course you're not using labels here you lack supervision so usually results will not be optimal actually the web more choice for this kind of task really is supervised learning especially


supervised deep learning because in supervised deep learning by using different different deep learning techniques such as convolutional neural networks we can actually teach them what makes a cat a cat and what makes a dog a dog and they learn that in a fully automated fashion by providing them labels by providing them ground truth if the relays or if there really is a dog in that image and based on that they then make the classification and results can actually be astonishing astonishing good right we can use we can achieve human


and even super human performance especially for these types of tasks which are very specific and also based on images you can achieve astonishingly great results using supervised deep learning but well that great performance that you often read about in the news for reaching that for reaching human level and superhuman level performance very often you're gonna need just tons of labeled data and really tons we are speaking about tens of thousands hundreds of thousands millions of labeled images and that is a lot and


this is also a big big problem because obviously label data is so important for achieving this great performance but at the same time label data is scarce right data we have a lot of it we have data lakes and data warehouse is full of data but actually getting the labels that you need for solving your specific problem that's something that's usually not existent of course at that point you could argue well you know if I don't have these labels I can just label it by myself I can sit down on my desk look look through these images of cats and


dogs and note if there is a cat or a dog on that image and I fully agree yes you can do that absolutely but you can imagine that this is quite some effort and that effort Rises very very quickly with the complexity of the image for example this image is an image we took from a real practical use case we did together with me burr moebus a large manufacturer of industrial parts and that image actually shows one of these parts well on that image in theory you can recognize defects in the underlying part well it's not an easy dog because


these images they are really large they are highly noisy and defects they can be incredibly subtle incredibly small so it even takes a human expert quite some time to see me reliably find and label mark all these defects on such an image so assume you want to automate that by trading some supervised deep learning model and for that you need to build up a large label data set of let's say a hundred thousand images labeled images you can imagine that that's going to be quite some effort and you know time is money so it's also going to be


expensive how expensive well it's actually not a too hard calculation to to compute this assume you use some publicly available labeling tool such as Amazon Sage make a ground truth for example they charge you I think if I remember correct around four US dollars per labeled image then as we said before you can do 30 images per hour because you can do you need two minutes per image and then of course it's not going to be you labeling but probably you're going to employ some working student for example it does the labeling and that


working student charges you 15 US dollars per hour well and now this is critical highly critical but most people forget about that you do not only need one person labeling your images why well because a person can have a bad day a person you know might just not be too accurate on that task and and this actually biggest problem these images you saw a highly complex these defects are so difficult to spot it's really really easy to oversee one so you need multiple people labeling the same image and then you need to aggregate these


labels to actually end up with a really robustly labeled data set if you don't do that you're gonna end up with a garbage dataset and you know how it is in machine learning garbage in garbage out your model is not gonna learn accurately what you wanted to learn and if you do the math behind this you will find out that building up that label data set cost you ten thousand hours work time and that amounts to more than a quarter of a million US dollars just labeling cost and this is significant right of course if you're big if your business


case is big quarter of a million might be nothing you might not even lose a single thought on that but for most companies in most departments spending quarter of a million I'm just building up a label dataset it's a significant challenge because at that point you haven't built a single classifier at that point you haven't deployed anything in production you basically haven't shown any value and this is a problem and this is why we argue that supervised learning and especially supervised deep learning is just very often not feasible because


first labels the labels you need to solve your problems probably don't exist and second if you want to create these labels well that's gonna cost you a lot of money and this is also by thought leaders of the fields argue that Dai Revolution is not going to be based on supervised learning and I agree how can Dai Revolution be based on supervised learning if we don't have labels right if labels are not ubiquitous how can a I be ubiquitous it can't at least not when it's based on supervised learning and when I thought about this the first time


I still found it a bit strange because well deep learning is based on artificial neural networks and neural networks as the name already says are inspired at least inspired by the human brain but we humans we need so much less labelled data to learn a task highly accurately so where did we go wrong here what makes the difference and this brings us to the core of this talk self supervise deep learning when we talk about self supervise deep learning we first need to take a step back and think about the human brain we need to ask


ourselves how do we humans actually learn do we do supervised learning well absolutely yes sometimes for example in school or at university you have an explicit supervisor telling you what's right and what's wrong what is a cat and what is a dog so yes we do but of course not always you don't always have a supervisor standing next to you telling you what's good and what's bad and even if we have a supervisor for example at school you don't need to be taught something 10,000 times before you understand to do a task actually just a


few example as a fish so again we are we humans are such efficient and effective learners our supervised learning seems to be highly different from the supervised learning between machine learning obviously well how about trial and error learning or reinforcement learning just to bring us on the same page what is reinforcement learning basically it's a subfield of machine learning where we try to teach an agent to learn a policy a behavior to solve a highly complex mostly sequential task such as driving a car in a simulation or playing chess and


that agent learns that by let's say smart trial-and-error basically we humans we also do trial and error learning absolutely sometimes we try something we fail we try again and then do better hopefully but of course we don't trial and error everything for example you don't trial and error learning how to drive a car you don't just drive around randomly until you manage to stay on the road right this is not a good strategy how to how you should learn how to drive actually what we do is quite different we get into a


car the first time and after a couple of hours we are actually reasonable drivers we can at least follow traffic in a basic way and this is fascinating because if you want to teach a machine to follow a real-world traffic in a reliable way but this is a big big big problem you need a lot not only of examples but also engineering power behind this before as humans it's so easy it's oh it's so it's because we are such effective and efficient learners again so obviously also our trial and error learning that we humans do is fundamentally different


from the trial and error learning we see in machine learning so what makes us humans such effective learners what is this magic ingredient that allows us to only require very little label data to learn a task incredibly well well this magic ingredient is something that we call having a general understanding of the world it's something that we all some also called common sense we humans we just know how things work right we just know how things work and we obtained this general understanding through observe raishin from the day that we are born


until the day we die we humans we observe we observe what's happening around us with all our senses we smell we touch we see we hear and this continuous observation this taking the world around us as our supervisor this is actually what forms our general understanding and all this observation lets us understand the true meaning of things what does it mean if something is heavy what does it mean if something is hot what are the implications what are the consequences also we start to understand abstract concepts and concept


relationships such as friendship for example and all these forms our general understanding and this general understanding makes us such effective and efficient learners because this means everything we learn we do not start from zero we always have a head start we always base everything we learn upon our understanding of how the world works and all our previously acquired knowledge and this is what makes us human such effective and efficient learners well that's great for us humans right but this talk is not primarily


about human intelligence but of course it's about artificial intelligence so the question remains how do we inject this general understanding into machines how do we make machines effective and efficient learners how do we allow machines not to need tens of thousand hundreds of thousands millions of label data points anymore and still achieve great great maybe even human level performance well the answer is quite simple actually we just need to let them observe the world we just need to make the data and their supervisor well but


how do you do that how do you force a machine to observe the world doesn't sound too easy well imagine you put up a video camera at this traffic junction and that video camera just continuously records what's happening in that traffic situation so you basically end up with an endless and an endless sequence of images and now imagine you have never seen a car before you have never seen a motorcycle before you have never seen traffic before and somebody gives you this video and you watch that video I promise after


a sufficiently long amount of time you will have figured out what a car is what a motorcyclist and how traffic works when it's a car allowed to go right when it's a car allowed to go left when it needs to stops and so on you have understood the concept of traffic by simple observation well that's how we humans do it but again how do we frame this a machine learning problem now the good thing is the good news is that supervised machine learning problems always somewhat the same they always have this very very basic structure that


you can see here and I would say 95% of all supervised machine learning problems are framed just like that you have some kind of input which in our case of course is our video our sequence of images that goes into a model could be anything in our case that lets say some type of neural network that model spits out a prediction and we then compare the prediction to the label to the truth based on a difference based on the error that our model makes we're going to take a step in our optimization procedure for example using gradient descent and the


next time our model is going to perform better that's how supervised learning works at the very foundations and it's quite clear about to use this input and also the model part is clear but what is our model gonna predict actually and what are the labels we don't have labels right we only have a video only a sequence of images so what should we actually teach our model to output and this is actually myself supervision comes in and this is also where your creativity comes in you need to think about how do I shape and frame the data


the unlabeled data that I have to form a supervised learning problem from it one example would be well you could simply chop up your endless video your endless sequence of images into smaller sub sequences and always take the last image of such a sub sequence and use it as label and then teach your network to predict the future from the past you teach it to predict the end of that sequence from the beginning or the previous elements of that sequence of these images your model is going to learn to predict the


future from the past it's going to learn when a car is going to move when it can drive forward when it has to stop when it's allowed to go left and went allowed to go allowed to go right and we can do this in a fully automated fashion you know you can simply create a small Python program for example that chops up your video into small sequences always takes the last frame of such a sequence using it as label and then you train your network and you're gonna have an expert network when it comes to traffic basically and you can use this concept


of self supervision of automatically creating labels from unlabeled data also in other types of data it's very flexible for example you can use it on images you can just randomly crop out images of let's say faces and teach your model to predict these rectangles or you can also use it on text you randomly remove words from text and use the surrounding words the context words to predict the target word the missing word that you removed well that's that's awesome but so what right what what are you gonna do with the model I predict


the missing rectangle in an image that's usually not what you want to solve and that's true so what we actually gain from this what do we gain from a model that knows how to complete an incomplete face well what we gain from it is understanding that model by performing this task by learning this task builds up an understanding of the concept of a face it learns very low-level representations about the face and also high-level representations such as it's gonna learn that usually areas one knows in a face there are two ears in a face


these ears sits on the sides of your head and so on it will learn a general understanding of the concept of a face and you can use this understanding you can use this knowledge for any other task you really want to solve that is somewhat related what does it mean well first let's say you do exactly what did you randomly remove rectangles for images of faces and if if you have little images of faces you can remove rectangles in a variety of different ways right so you can actually end up with a large data set you train your


model to predict these missing rectangles your model will then have an understanding of the concept of a face then you take the same model you take the model with all of its knowledge and use it for whatever task you really want to solve for example for predicting the age based on images you just fine-tune it a bit maybe modify the architecture bit no big modifications and your model is not going to give you the missing rectangle anymore but it's going to give you the age of the person that's visible on that image and all that by simply


using a model that has acquired general understanding through self supervision and then you just fine tune it with little label data for the tasks you really want to solve what you gain from that is overall well first of all since your first building up general understanding and then fine-tuning your model to the task you want to solve and that means your model will need less label data to achieve better performance your model will converge faster because its weight already in a somewhat optimal position and also your model is going to


be more robust because the features it learned these important characteristics will learn on a very large data set on yourself supervised data set and you were just fine-tuning on your downstream tasks where you're potentially a very little labeled data this is actually really a win-win situation here but how does this work in practice instead really usable in practice well and this is why I brought this industry case with me which we did together with upstream ability and our task at hand was estimating mode of transport from GPS


movement data well GPS movement data is quite easily described you just have sequences of GPS coordinates and then associated timestamp and basically what we what we got was a data set of these trips across Vienna and that person here started his trip at the Green Dot ended it at the red dot and every dot in between was a GPS signal our task was for each of these GPS signals estimate which mode of transport he was taking was it going by bus was it going the Metro was he walking cycling going the car and so on actually


quite hard quite complex given the little information so the first thing we asked our self here was so how would we humans solve this problem well what I would do is I would go on Google Maps and check out where transport lines are running right because if a GPS signal is nowhere close to a metro line probably he was not going by Metro and that's what we did we enriched this map of Vienna by adding image channels and each image channel represents a different transport network one represents the Metro network one the bus network and so


on this is awful for humans right this produces an image you cannot interpret but it is great when it comes to deep learning well and then we used that information to embark on building a super lies deep learning model with no self supervision at this point the way we framed this problem was well we framed it as a sequence learning problem what does this mean so basically we thought our hypothesis was the movement history of a person provides us with important information also for estimating his current mode of transport


because obviously if a person has been on the Metro for the last five GPS signals because his GPS signals were just on top of that metro line and his current trip ethical is again on top of that metro line it is quite likely that he is still on that Metro because changing transport modes of transport is actually associated with costs in terms of time effort and maybe even money so our hypothesis was abused the previous movement history to estimate this current mode of transport so again how do we frame this a machine learning


problem we have a big big image with dots on it and then we want to incorporate the sequence of dots to make our estimation so you can do this in a variety of ways of course the way we did it was for each of these dots we want to incorporate in our for making our prediction we cropped out a small rectangle of the image around it so we end up with a sequence of image tiles each of these image tiles representing the path in total then goes into a convolutional neural network a so-called CNN which is a type of neural network


that is very good at extracting important information from images that CNN then produces what that CNN thinks is important for making the modality estimation so called feature vectors it learns feature vectors which is simply a numeric dense vector representation of that image and what is important in that small image title in that sequence of image a feature vector s then goes into a recurrent neural network a so called R and n an errand n again is a type of neural network just a very different type which is great for modeling


sequences from modeling temporal dependencies between elements of a sequence and since we are dealing with a sequence of GPS coordinates here or a sequence of image tiles are actually now a sequence of learned image features an RNN is an obvious choice here and there are an N then finally it gives you in the end which modality the person was likely to be taking was he walking going by bus taking the metro and so on and that what you can see here all that was basically one single deep learning model and it worked out reasonably well


actually surprisingly well however you can imagine that this was not a small model actually we were dealing with I think more a bit more than hundreds of thousands of parameters in that model which is ok if you have sufficient labeled data and you can guess well we didn't of course we were dealing with rather little real-world data here which is a problem given if you use a large network because the network is likely to overfit and so on so we thought about well how about self supervised deployer this is actually a


very good very good case for that how about first letting the model develop general understanding and then taking this model with all of its understanding and using it for solving our very specific modality estimation tasks and that's how we came up with that concept you could for example teach your model the entire Vienna transport network in a self supervised pre training stage just as we did before then take the not take that model with all of its knowledge and find unit to make modality estimation happen sounds


good but how do you teach a model EBN a transport network and this is again where it comes to self supervision where it comes to creativity how can you frame a problem from the data you have to make this happen one way is to for example randomly sample 32 by 32 pixels again these small image tiles from a map of Vienna which is publicly available of course and then you just automatically check which line of transport is running on that tile it's all information is publicly available again you take that as labels


and you teach a model to predict which line of transport is running on a given image tile your model is going to learn the Vienna transport network it's going to be an absolute expert and you can then take this model you can take this CNN that knows everything about the Vienna transport network and put it back into our original architecture replacing the CNN from before the CNN that had to be trained from scratch our model now does need to be trained from scratch it has a big big head start it has a quiet general understanding so it just needs


to be fine-tuned a bit and that's an end and passes on high-quality feature vectors again to RNN which learns the temporal dependencies and outputs the predictions what you gain from this is you have a significantly smaller model because you do not need to train the CNN from scratch this means you can have many of the parameters frozen and you end up with a more with a model with trainable parameters going from a hundred thousand before to ten or tens of thousands now which is a lot better suited for little labelled data so again


what you're going to get is your model will be able to deal with little label data better it's gonna be more robust it's gonna converge faster because you simply have to optimize less parameters here and overall it's gonna make more accurate predictions given the same labelled data we have come a pretty long way now all the way from supervised deep learning why it's so great so powerful but also we got to know its ugly side we got to know that it just did tons of label data to achieve this famous performance subsequently we found out


how taking a self supervised approach can help us overcome that key problem of supervised learning and finally we're looking through the industry case that hopefully gave you a pretty good practical idea of how you can apply self supervision to your problems to make your models work better on little label data I hope you enjoyed the talk feel free to ask any question you might have in case we are out of time for questions I will be hereafter talk as well so feel free to approach me and ask any question in private that may have been


left unanswered also of course drop me an email and add me on LinkedIn I'm always happy for for method for interesting message as I get and most of all visit our website craft works dot AI thank you very much okay um let's see I think we maybe have time for one question would you like to choose which one you think would be the best one the unites answer okay I think the second one what is the main difference between supervisors and self supervised learning this is a important question because it's very fundamental


so self supervised learning is supervised learning the concept behind self supervision is that you do not have a label dataset but you come up with a smart task how you can create labels from a previously unlabeled data set not needing any human labeling this is the concept of self supervision you use the data that you have that video that we had before you do not have any labels you cannot just apply supervised learning because you don't have labels but you need to implement a self supervised task that where you first


well I'm honored to be here and my hat is my favorite hat I think it makes me look rather handsome this hat is made from a mushroom called Amadou Amadou is a birch polypore and a Madhu is a hardwood conch and this mushroom is responsible for human survival not too long ago there's no doubt that we all came from Africa we went north we discovered something new called winter oops this mushroom allowed for the portability of fire moreover you can hollow this mushroom out put embers of fire inside and carry fire for days and

Mushrooms as Medicine with Paul Stamets at Exponential Medicine


the fire keeper of our clans thousands of years ago were absolutely critical for the clans survival well this mushroom has other properties and when you boil this mushroom it delaminates and becomes mycelium a fabric and since some ladies in Transylvania have kept this tradition alive so this threat of knowledge has carried forth over thousands of years and so many threads of knowledge have been interrupted because of famine disease and war well this mushroom is first described by Hippocrates in 450 BC II as an


anti-inflammatory as well as for Carter izing wounds another mushroom I brought mushroom friend of mine also is a polypore wood conch and this is a Garak on a gerakan is the longest living mushroom in the world rose exclusively in the old-growth forests now presently only known from Northern California Oregon Washington and British Columbia in a sky island or two in Central Europe it was described by ascribe ease in the very first materia medica as Alexei reom at Langham vitam the elixir of long life and it was suggested thousands of years


ago as a treatment against consumption later to be known as tuberculosis so I'm going to take you on a journey and I'm going to take a radical left turn halfway through this talk and I'm going to present some data that has never been shown any to anyone else outside of my research team so I am honored to be representing triple a s on June 9th this year I was awarded as the invention ambassador this is great it's like a first audience that knows what Triple A s is so I don't have to explain that but it's a huge honor and I grew up


in a small town in Ohio and my brother John with my got me into science and he went on to Yale my brother Bill one on the Cornell and we had this incredible laboratory in the basement which they would not let me have access to but they went off to college and I suddenly had this fully equipped laboratory including the the radio from the aircraft carrier the intrepid my father was on it and after world war ii he got the radio so I was listening to all sorts of things behind the Iron Curtain I was just having a fabulous time so my dream was


always to live in the country and be a scientist and have my own scientific laboratory well on June 9th I got that award my brother John we were competitive and there's you know like brothers are you love him 80% of the time and 20% of time they kind of piss you off and so John really never respected you know my interest in my ecology what is this mushroom stuff but so when Triple A yes gave me this award it was highly vetted and I said Wow you know I can I get this is exciting I can tell my older brother now John I got


this award so I called him he didn't answer and then I emailed him and that was a day they discovered his body John I died from cardiac arrest standing up and I just want to tell all of you you have brothers and sisters that bug you you know think about the good times and how our life is so precious and so short so this talk is dedicated to my brother John who first got me into science so my main theme is biodiversity as biosecurity this is we live in I live in Washington State in the southern regions of the


Puget Sound and I want to point out the largest organism in the world is a mycelial mat 2,200 acres in size over two thousand years old and it's one cell wall thick surrounded by hundreds of millions of microbes per gram of soil we have several skin layers to protect us from infection the mycelium has one and yet achieves the largest mass of any organ in the world how is that possible well it's possible because it the is involved in I own his own microbiome it selects beneficial bacteria that it works in concert with and the mycelium


has based on a network like design the mycelium is digested nutrients externally we share more common ancestor with fungi than we do with any other Kingdom six hundred and fifty million years ago we split from fungi and there is an announcement a new super Kingdom it's been published called a pice the canta that joins Animalia and fungi together we exhale carbon dioxide we inhale oxygen and the fungi are able to stream nuclei to their tips and because of epigenesis the ability to adapt to change this is one of the few organisms


that actually benefits from disruption and so when these mats are disrupted the fork the us streaming of nuclei epigenesis comes into play reassortment nuclei at the end tips it codes for new genes for new enzymes acids to capture new food and then the information becomes back channeled into the mice illegal network so using the epidemic properties of mycelium i think is a way of the future of medicine so the mycelium is a lot more pervasive than most people realize virtually 90% of all plants have mycorrhizae fungi has now been


determined that these microbial networks and the mycelium communicates across landscapes in between plants and indeed all plants are part fungi so any research on botanical medicine the contribution of the end of it funds that are associated inside this plants needs to be taken into account because the conferring medicinal properties may be welcoming from the end of it expunged I as opposed to the plant by itself these my cellular networks stream across landscapes and I have these epiphanies and I believe habitats have immune


systems and the mice ileal networks are the foundation of the food web that's joining us all together now here is something that's I grow lots of mycelium twenty to thirty thousand kilos a week we have a small company sixty seven employees and this is differently frankly just not fair that I can tell you this in fifteen seconds which took me thirty years to discover the problem with mcgroin mycelium in laboratories is immunologically naive it's grown in pure culture when you throw it out into the ground all these organisms consume it


well we've soaked woodchips or a straw under water salt water or fresh water for two weeks in aerobic interwebz become predominant then we take this out and we drain off the water and then the oxygen becomes a sterilizer the anaerobes are largely killed simply our robes are in there and then the mycelium then becomes immunologically educated this is a profoundly powerful mycelium it's got an immune system and resident within this mycelium is the enormous amounts of bacteria we did next-gen sequencing here and this is a color heat


map a thousand full difference in the relative abundance of different genre of bacteria to different mushroom species selected out whole different constellations of bacteria this enables the mycelium to cell set up guilds and have commensal mutualistic organisms that i can combine with that allows it to conquer such large habitats well we all know that we have cancer 41% of us will get cancer 21% of us will die from it but did you know still that 73% of all against our cancer drugs have their origins and natural products we grow


about 500 different species turkey tails featured here or when the best described and studied medicinal mushrooms in the world we received a 2.2 million dollar breast cancer clinical grant from the nih for phase one breast cancer and the results of the studies have been published and on a dose-dependent basis well prior to radiation your immune system is as active and then when the turkey tail mushrooms eight capsules per day are consumed there is an upregulation of natural killer cells and then post radiation most of you know the immune


system is damaged it and then it has to recover and then on a dose-dependent basis two weeks and then four weeks the immune system kicks into gear natural killer cells are enhanced dramatically and also cytotoxic T cells look at the significance value here and so the immune system is activated by the consumption of these mushrooms and there's TLR tlr4 receptors I don't want to get into that right now but we've identified seven different distinct pathways of the immune systems activated than the consumption of these mushrooms


now this became deeply personal to me in june 2009 when my 83 year old mother called me up she's a charismatic Christian she's not seen a doctor since 1968 she called me a mrs. Paul I'm scared I didn't recognize her voice she was shaking I said what's wrong and she said my right breast is five times the size of my left I have six angry lymph nodes dark and swollen on my right side I said I couldn't believe it why didn't you tell me sooner and so I rushed her to Swedish breast cancer clinic in Seattle and then we got


the worst news after the second visit the oncologist said she should have been seen two years earlier the cancer is inoperable they could not do a mastectomy because of her age they couldn't give a radiation therapy because of the same reason because the likelihood of infection and so the oncologist tried to make best of it saying you live a long life and we kept on asking how long how long how long and she said you lucky if you had three months the tumor is erupting out of her breast across the Meridian invaded her


sternum and when it went into her liver so we had the circle meeting many of you have had this we planned for her funeral she chose a pink dress that bought the cheapest coffin that she could find because she was going to Jesus there's a lot of Tears and then on the third visit the oncologist said you know if your immune system could kick in patty you might be able to beat this and so she said you know there's these turkey tail mushroom study that's on a misty romantical school University of Minnesota Medical you might want to


take start taking turkey tail mushrooms well that's my mother said well that's what my son was talking about but she had to hear it from a doctor right so my mother started taking turkey tail she was on tax haul briefly had a horrific reaction refused to take it and she was ten then she's worth taking Herceptin a wonderful drug well that was in June of 2009 and I'm happy to say my mother she crossed the five year disease free period she's totally cancer-free this led to then a study saying well maybe turkey tail mushrooms can enhance


Herceptin now the good news that my mother survived but then she told me something her oncologist told her of the 50 women who joined at Herceptin program and listed in a new program in Ellensburg Washington where she enlisted of the 50 women 48 of them have died my mother was the only one taking turkey tail with Herceptin so how did this is interesting on multiple levels she's been written up as a best case outcome several medical journals she that no chem brain no nausea no loss of appetite so she's happy and she her


acumen has come back she's smarter now than were quick with her wit than I've ever ever seen so then a series of other articles came out this past year turkey tail enhances the microbiome specifically of lactobacillus and Bifidobacterium while suppressing and anti flattery bacteria so this is extremely interesting because this speaks to the fact that when we grow the much of mycelium in pure culture we do see a resident mutualistic population of bacteria which may we at first thought were contaminants but now we understand


better that they're part of the of the microbiome of the fungi when we split from fungi 650 million years ago we chose the relevance circulating our food in a gastric in a sack and a stomach basically digested nutrients within the mycelium went externally well just as we have a microbiome within us the mycelium is selected a micro biome also Muta listicle e to its advantage I'm very interested in the viral to cancer connection there are seven identified viruses or probably a lot more that caused cancer then Fred hutch


medical school called me up and said we have a very interesting case from the Merkel cell carcinoma one of the most deadly cancers of the world only ten people have ever been reported to it have ever recovered from it and I call it the NIMH hypothesis dr. paul NIMH MD PhD at Fred hutch and they had this patient he started taking a seven species mushroom blend and this is immune evasion and then after taking the mushrooms there's no chemotherapy no radiation therapy nothing can be done for these patients and then after taking


the seven mushroom species blend he had spontaneous recovery and he is alive today so we think that it D cloaks cancers for discovery by the immune system we don't know exactly how it does it but we've seen this over and over again your immune systems activated and your mutant cells can discover receptor sites in the stroma of tumors this could go abroad and be useful for addressing lots of solid tumors as an adjunct therapy so this case also was written up in the medical journals and then Hayling Lou and I submitted an application to


NIH to standardize the methodology for analyzing medicinal mushroom products it was amazed or so and unfortunately the sequester occurred and we did not get funded for this but it's a great paper that I'd be glad to provide to anyone who likes to see it the Bison team produces extracellular metabolites and in these metabolite droplets are all sorts of interesting compounds I was working with a bio shield program of the US Defense Department directly after 9/11 we submitted over 700 samples and this is the said the samples of


mushrooms in particular a gerakan reishi and chaga and this is the selectivity index and the viruses h5n1 h3n2 h1n1 and the rip of ayran is being the positive control the selectivity index is an indication of antiviral activity our extracts were diluted from the mycelium 100 to 1 and this is the selectivity index of the diluted extracts that were much far more powerful than the pharmaceutical control well then we were we did by our guided fractionation of the University Mississippi this is a school pharmacy and we've identified a


group of sterols this one has been unreported in the literature I've given it the name phony top straw so we have our first eight api's here that are active against in this case poxviruses now that's coming on the same mcgarrick on extract that was active against flu viruses but we sent these structures to to st. Jude University in order to st. Jude hospital in order for tests against HIV and when they were totally inactive which suggests that there's more than one antiviral API that's present within these these mushroom extracts NIH called


us three times in the past month we've submitted now ten of these structures they potential api's for testing against Ebola and a wide number of other viruses so resident when these mushrooms are very interesting complex molecules that were beginning to discover so after ten years I finally received a patent universality of opinion by the patent extent patent examiner's and it took a long time to get the patent but I was happy to see that vector in Russia published an article two years ago authenticating that a gerakan is highly


active against flu viruses this article was published yesterday so is there people are catching up but it's great that other researchers are authenticating that which we had discovered now working with a gerakan instructor scott franzblau who is the director of the tuberculosis research institute at the university of chicago we started doing experimentation and he started using our mycelium and we did by organic fraction we found a new active anti-tb set of molecules chlorinated chlorinated coumarins now this is interesting to me


this mushroom has a dual activity against viruses and bacteria very few medicines do that the majority people who die from viral pneumonia actually die from bacterial pneumonia and so to have something as a nutraceutical that can be broad-based against multiple viruses multiple bacteria I think is medically extremely interesting my wife and I spend a lot of time in the old-growth forest the force used to be resplendent around the world and now we are facing a radical change and in our ecosystems through deforestation so the composition in the


ecology of the forests have changed and 70% of the soils are composed of microbial mass of which 40% of the mouse's fungal but because of our practices of logging and harvesting and creating monoculture which in repetitive Harbor planting of trees leads to premature decline disease vectors spread the diameter of trees become smaller and you lose that that plurality of biodiversity of Ages of trees and their associated organisms we have really changed the face of this planet so I'm going to now take a radical left turn so


this is the case now imagine hundreds of millions of years our ancestors and other organisms in the ecosystem have been used these resplendent for us and now we've deforested much of the planet and the deforestation continues at an incredible clip we've now entered 6x the sixth greatest extinction event on the life of this planet and we're losing about 30,000 species per year of 8.3 million species on this planet that means that a hundred years we'll lose more than 30% of the biodiversity on this planet this is this


is an all-hands-on-deck moment so a friend of mine came to me he said Paul I do a lot of work with antibiotic genic fungi controlling insects and says can you help the bees and then whole foods provide this very interesting graphic here's your dairy choices with bees and there's your dairy choices without bees bees liberate pollinators 30% of the food in the grocery store is direct result of pollens it and pollination 70% is indirect and the President Obama came out with a presidential memorandum and there is a we call it click on a hex


efekta there is like six different converging stressors on the ecosystem deforestation is one the bees now don't have the ecosystem that it's evolved to to draw from as part of this menu it's banquet of food well then because of the pollution and one dozen speakers mentioned his blood was analyzed he has a thousand different xenobiotic toxins president is blood unprecedented a theater of evolution mites are carrying viruses and then you have of the the fact that the bees are being trucked hundreds of miles into almond orchards


in the middle of the desert in California in January and February this is totally unnatural so the bees fly out and we see bees around a flower that's the last seven or ten days of a life the bees flap their wings into the wings are shredded and the bees then with a colony collapse disorder the bees leave the Beehive and they just don't come back they just suddenly disappear now it's a very complicated set of stressors but just like there's colony collapse disorder I suggest to you that we are facing cultural collapse


disorder this is a proverbial canary in the coal mine so I had some very strange events in my life a lot more that I can tell you but but I was growing mushroom mycelium in my garden this is 1984 and I went out to my garden and this is the mushroom beds and I went Wow what's going on here I look very closely and bees had come to my mushroom bed move the wood chips away and started sucking on my mycelium I went what is going on from day to night for 40 days for 40 days a direct stream of bees from my beehives to my mycelium back and forth


all day long the mycelium shrunk from about 8 or 10 inches to about 3 inches well I noted this in one of my books and Harrow Smith magazine virtually everybody ignored me I got one beekeeper from Canada wrote me well maybe that's why do they go to sawdust piles so ok I put that in the back of my mind and then a friend of mine said you know what can you do to help the bees and I thought well you know I had this very weird experience in my garden 1984 so here's dusty in the old-growth forest and bear scratch trees well we used to have a lot


of bears but the tip industry put a bounty on them killed the Bears and only in the past 20 years we've come out with research knowing and finding out the Bears bring salmon carcasses up on the banks and returning sea phosphorus from the ocean into the roots of the trees which is a limiting nutrient for tree growth so the industry totally got it backwards Bears help trees grow well the Bears scratched the trees and dusty and I are hiking in the old-growth forest and the South Fork of the hoe and we go around the corner and dusty sees this


bear scratch BAM the bear scratched the trees the best bear scratch I've ever seen but that's why I photographed it now looked into this and wow the timid industry says the bear scratched the trees and it causes a mushroom to form which is related to a Garrett con so we went back two years later there's that bear scratch ok so think about this and the bear scratch the trees then resin comes out and bees go after the resins and they get propolis which is a very strong antimicrobial and they used for patching up you know spaces in the


Beehive so the red belted polypore sure enough was growing out of that tree with a bear scratch that we saw so in a sense the timber industry is correct this is a parasitic fungus that kills the trees and then grows Sacre physically well also interesting the mycelium breaks down pesticide herbicide and fungicides okay so that's another box another experience I had the garden now hiking the old-growth forest a bear scratch I looked into the you know why the timber industry was trying to kill the bears then this article comes comes


out these are all very very recent and it turns out that fungi produced the P chimeric acid related to the chloral chlorinated coumarins that Scott Franklin our fund their active against tuberculosis by the way and it turns out that the absence of P chimeric acid stops the up regulation of the cytochrome p450 pathway bees only have 47 cpy genes whereas most insects have 80 and the absence of P chimeric acid coming from fungi turns off their mono oxygenase pathway and so they can't detoxify this accumulation of all these


toxins that are become resident as they for a out into the farmer's fields sprayed with pesticides fungicides and herbicides okay interesting and so then it turns out that when the EPA license many of these fungicides and insecticides and herbicides they didn't look at the consortium of them all coming together and turn out the sub-lethal doses of these toxins defeat the microbiome in the gut of the bee so you have another problem happening here not only is this my oxidase pathway the cytochrome p450s turned off but the


microbiome now is being damaged by glycolysis by the way there's one of the big cult culprits some craig Venter hope you're listening so then fungicide contamination in the fields is harming the resident fungi and now we don't have rotting logs we've got agricultural crops may the species of which are not native okay so beekeepers feed bees sugar water and up to 50 these a 50 percent water 50 percent sugar this is because they need to have the sugar obviously for food and then the bees are truck hundreds of miles in this case the almond and walnut


orchards for pollination so the bees are now being fed pure Sugar's as opposed to the complex carbohydrates and polysaccharides was coming from the sweat of the Weisse ilium so I had an epiphany why don't we take our mycelium and my research team you know it gets credit for this and we came up with Michael honey this is totally made from mycelium it's like 90% sugars but they're complex sugars and guess what it has P chimeric acid in it as the antiviral agents in it as the antibacterial agents so we contacted


Washington State University working with dr. Steve Sheppard and and Brandon Hopkins and we started doing a series of experiments by feeding extracts of the mycelium to bees at different concentrations this is called a stress test they're in captivity they only live 30 days when the worker bees fly out and they're doing a pollination if they don't come back nurse bees are prematurely recruited to become worker bees and they fly out they abandon the brood so it's a doubling down every time the fewer and fewer worker bees come back nurse bees now


have to go out and get pollen and food for the hive so the the larvae and abandoned mites then proliferate mites are injecting viruses into the larva okay so the mushrooms that we're talking about clean the one of my hat Amadou reishi and chaga are polypore mushrooms and birch forests worldwide apis mellifera the honeybee is from europe it's not native to North America but it produced a four-digit amounts of honey so there's chaga there's Amadou and there's red reishi we provided the bees with twelve different species these are


the three ones I'm going to talk about I've never shown this before this this information just came came in the sugar control in one week's time the virus is increased by 63% when the bees started sipping on the mycelium the viral about pathogen payload plummeted across these three different species in week one versus a week two the sugar control the viruses you know increased dramatically and with with the bees that were taking sips of the mushroom mycelium extract the vibe is plummeted they went up here and they


plummeted down on a dose-dependent basis and so it also occurred with a red reishi so now we're trying to get the right concentrations and the often is obvious now that if we use a combination of these you know what benefit what a benefit will this be but we don't know the way of the B tomorrow the rosetta spaceship lands on a comet 300 million years 300 million miles out in space well we can find a comet we don't know the way of the B now I've spoken to entomologists about this they spoke through their friends no one's ever mentioned this I


spoke at a national mycological Congress I said any mycologist out there this 500 I call just has anyone ever heard of this B's go no one has bees go to rotted logs because of the immunological benefit increasing their hosts defensive resistance they're complex sugars of nutrition and the antiviral properties I'm the first one to have discovered this really how is that possible we grew up with Winnie the Pooh reading it to our kids they're going after the rotted logs and and and we don't know the way of the bee I think this says a lot so


doctor have Steve Shepherd it was so impressed he provided this wonderful quote as entomologist 39 years of experience I'm unaware of any reports extended the lives of worker bees this is this spread is incredibly important this is a period of high pollen acquisition and so if you increase the workers lifespans by 20% they had a tremendous effect on a tipping point a tipping point in favor of the colony survival so I suggest to you let's be friendly let's be mushroom the scientists cross disciplines need to work together biodiversity is our


biosecurity now think of the bigger picture here we were forced people bees evolved in the forest the mycelium is confirmed confirming an immunological benefit to to animals but it's unprecedented as far as I know that there is an antiviral agent that is duly active in helping bees and also helping humans and these are from polypore mushrooms are resident in the forest that our ancestors were dependent upon so I want to conclude that humans trees bears mushrooms are all terrestrial organisms that evolved to be interconnected within the mycelial


web of life Earth's natural Internet and I think the way of the future is using mycelial scaffolding with a mutualistic organisms in the bacteria using epigenesis and then being able to have the quorum sensing and there's response and being able to up regulate gene expressions that otherwise may not be present or up regulated with one organism but quorum sensing can give up regulation of multiple gene sequences otherwise hidden in nature this is the way of life so as much as many of you are ultra specialized I want you to


but steel bath with a temperature of 1,500 degrees sparks fly is a diesel engine is about to be born a veritable Colossus the huge engine will soon power a modern ship it will have to withstand enormous pressure and extremely high temperatures for many years manufacturing such a giant calls for high tech and precision one of the most powerful high speed diesel engines in the world it is part of the MTU concerned series 8000 made in Germany the engines are produced in philosophical on lake constance malte from here the high speed ferry

Exceptional Engineering | Mega Diesel Engine | Free Documentary


Jean de la Villette transports passengers as well as cars and other goods to and from Italy the catamaran ferry is the most modern of its kind in the Mediterranean and it is powered by four series 8000 engines from MTU three levels below deck the crew start up the ferries for mega diesel engines at idling speed the men check that everything is technically in order this takes ten minutes and is a must before every voyage the team then head back up because down here things are about to get very loud and above all extremely


hot Dejan de la Villette leaves Malta bound for the Italian island of Sicily 120 kilometres away [Music] weather permitting the 52 thousand horsepower ferry takes less than two hours for the crossing on the open sea the 1500 ton catamaran can reach a speed of up to 42 knots that's 75 kilometers an hour this extreme performance is achieved through its 4 MTU series 8000 diesel engines [Music] MTU series 8000 consists of 20 cylinder common-rail diesel engines each unit is 7 meters long 2 metres wide and three and a half meters high roughly the same


size as a steam locomotive it also weighs as much around 48 tons the 20 cylinders have a capacity of 350 liters and produced thirteen thousand six hundred horsepower fuel consumption is 2,000 liters an hour the core element of the engine and its biggest component is the huge crankcase [Music] like all the engines components the crankcase also comes from Germany this foundry and phone bag in Bavaria is the birthplace of the mega diesel engine and a location with a long tradition a small smithy existed here way back in the 15th


century so hot iron has been molded in phone bag for nearly 600 years today the foundries three smelting furnaces are operating at full speed to cast a series 8000 crankcase for 10 hours now the steel workers have been feeding the voracious furnaces different materials the materials we need for the series 8000 are steel scrap pig iron and deep drawn sheet metal we also need electrode graphite and silicon carbide the composition is calculated by our superiors in this case by the engineers in our production planning department


casting a crankcase calls for 16 tons of material hot hard work for the smelters who have to keep a constant eye on the temperature only when the molten metal has reached a temperature of 1500 degrees is it ready to be transported to the mold here we have a measuring sleeve made of press board inserted at the front here is a thermal element it's attached to a measuring probe and used to measure the temperature of the melt every steelworks has its own recipe time and again the smelters add this or that material the better the composition of


the smelt the harder and more durable the crankcase will be from time to time the smelt is checked in a modern foundry a sample of the molten metal is poured into a crucible containing a small measuring sleeve which in a matter of seconds transmits the temperature profile to a computer in the office the smelt has the perfect composition when the sample cools down in a certain way and then heats up again the enormous heat is no longer generated with fire but with electricity the principle is similar to the way an induction cooker


functions in a domestic kitchen the big difference is that the three electric induction furnaces in the foundry have the power of five thousand microwave ovens [Music] therefore megawatt have heated the molten metal to a temperature of over 1500 degrees 200 meters away the mold is already waiting speed is now essential the steel workers transfer the molten mass the final ingredient magnesium is added via chutes the bubbling sound you can hear stems partly from the magnesium at temperatures of over 1500 degrees


magnesium vaporizes much faster and that's what causes the bubbling sound the magnesium vapour flows through the molten metal like carbon dioxide through mineral water this changes the molecular lattice structure of the steel and will make the crankcase slightly elastic [Music] the 19 tons of molten mass now flow into two huge casting ladles there would be no point in having more than two because the crankcase mold has only two intake funnels [Music] this is what the various parts of a crankcase mold look like these two


discourse consist of resin soaked quartz and which after setting has become rock-hard ten of these cores and sequence comprised the mold of a series 8000 crankcase prior to casting each template is given a coating which has a horrific smell to it the coating prevents the sand from mixing with the steel during casting the molten steel flows into the mold via this channel it passes through the filter chamber with its ceramic filter and enters the mold cavity via this feed channel and then spreads throughout the entire mold but


before that the ten disk cores are placed in a casting box the cavities are filled with special sand without this sand parts of the mold would fly off during casting it's a job which calls for a lot of experience any mistake could render the mold useless all this takes place days before the actual casting which also demands total precision the clock is ticking the molten mass is cooling by the minute so the smelters have to check its temperature time and again on no account must have dropped below 1,400 degrees


then the mass would no longer be suitable for casting a huge overhead gantry transports the cauldrons to the mold the big moment is close the birth of a mega diesel engine the two casting ladles are positioned over the to filling funnels from now on everything will have to take place with absolute synchronicity the head smelter starts the countdown three two one go in a fascinating spectacle the molten metal flows into the two funnels for the moment the two plugs are still in place because the mass first has to settle


once again the command to remove the two plugs is given by the head smelter now pull the mass can now flow in it takes 70 seconds for the 16 tons of molten metal to flood the mold and fill every cavity completely the head smelter closely monitors the entire process giving instructions to his colleagues controlling the casting ladles a little bit faster faster although speed is of the essence all movements have to take place simultaneously otherwise swirls could form in the funnels now after a good minute the job is done


within a few hours the metal cast in the mold will cool by several hundred degrees but it will be two weeks before the men can peel the crankcase from its mold only then will it be hard and elastic enough and extremely durable LaShonda la Valette is surging through the Mediterranean at a rate of over 40 knots the high speed ferry is heading for the Sicilian port of Port Salut it's not a propeller that the four mega diesel engines in its hull are driving but water jet power units which function on the recoil principle


water is sucked in under the ship and then expelled again with tremendous force the engine on this vessel pumps out 9.1 megawatts of power at eleven hundred and fifty rpm the the aggregate power is 36.4 megawatts so in actual fact losing an engine will not affect the the vessel drastically the immense power from the diesel engines is a big help especially in berthing the 1500 ton ferry the ship is highly maneuverable and can be steered easily even in a confined space the captain slowly brings the 107 meter long jean de la Villette


against the key wall in sicily in just eight hours time the catamaran will be heading back to Malta that's not a long time because today a team of mechanics specialists in marine engines has come on board one of the four mega diesel engines needs two new cylinders and the old ones require maintenance the men have to dismantle the diesel engine and replace the cylinders since a stay in Port is very expensive the specialists need to get to work straight away this means heading down to levels awaiting the men at a temperature of 50 degrees in the


engine room are a booming auxiliary power unit and the 48 ton diesel engine they all know that this means a lot of work under time pressure brakes are out of the question back at the foundry and phone back two weeks have now passed that is how long the steel block has taken to cool down completely the heavy and in so far as steel permits elastic crankcase for the mega diesel engine is about to see daylight for the first time with the help of the overhead gantry the 16 ton Colossus is lifted out of the casting pit this causes the


courts and cocoon to crumble exposing the brand-new series 8000 crankcase what we now see here is the down sprue and the path taken by the steel into the mold the first task now falls to a welder who has to sever the two down sprues the gas jet slowly cuts its way through the tubes which are as thick as tank Armour other superfluous parts are also still attached to the seven meter long cast body they were needed for the casting process now they too are cut off [Music] there's another four to higher up and to


lower down then we'll tear everything down and it will remain standing in the middle the experienced welder burns his way along the crankcase bit by bit after an hour the block is fully exposed the casting channel can be removed but the material that has been cut off is by no means scrap it will be needed for the next melt the overhead gantry then comes into action again it lifts the block to the next station a huge mechanical vibrator the deafeningly loud vibrations cause any remaining quartz sand to fall off it too


will be recycled after being cleaned the sand will be used to produce the next casting three minutes later the mega diesel crankcase is roughly clean it's next stop is the shot blasting chamber where the crankcase will be bombarded with a storm of tiny steel pellets [Music] these are the steel pellets they're made of ordinary steel and have a maximum diameter of 2.5 millimeters so we can roughly estimate what the surface will look like afterwards and how clean it will be millions of the steel pellets are fired


to the chamber at a speed of 200 kilometers an hour this not only cleans the surface of the component the bombardment makes the steel harder and more corrosion resistant the process takes an hour then the doors are opened and with the help of a crane the men remove a perfectly cleaned engine block from the shot blasting chamber the razor-sharp casting noses also have to be cut off the crankcase can then leave its birthplace for transportation to Lake Constance Assembly of the mega engines takes place at the NTU plant in


fleet lease often the firm specializes in large high speed diesel engines ships submarines military vehicles and locomotives worldwide are powered by MTU systems [Music] a series 8000 naval engine has to pass through six assembly stations twenty-five of these mega diesel engines leave the plant every year prior to assembly the raw part is milled the Colossus is machined to the right dimensions in several stages losing up to a ton of material in the process this takes several days engines of this size are machined with extreme precision


the tolerances to be adhere to are so minut that they can no longer be perceived with the naked eye Lonnegan now we're talking about an accuracy of three or four hundredths of a millimeter such dimensions pose quite a challenge in other words a component the size of a small bus must not deviate from the ideal by one hundredth of the width of a human hair to check this the Colossus is placed on Europe's biggest measuring table Engineers measure every opening every drilled hole indeed every square millimeter to ensure that no


disaster can occur when the engine is in operation after two weeks the crankcase is ready now main assembly can get underway over the next five weeks these 35 fitters will turn the barre crankcase into an extra extra-large mega diesel engine the overhead gantry transports the 10-ton parts of the rotary station where it is positioned with millimeter accuracy during this process the entire area is sealed off little towards you now got it it's in [Music] first of all the crankcase is given a serial number in the past this had to be


hammered in with effort now a special tool performs the task mechanically in seconds the green light for assembly is about to be given by Foreman : ooh huh he is in charge of the workshop but first the fitters check the surfaces and the bearings once again large axis will illuminate the entire crankcase starting with the inlets and then continue with the main oil channels the two fitters job is to detect any contamination tiny burrs or metal splinters in one of the channels could cause serious problems later the main oil channel here has to