So far in these slides on regularization we've looked at the model. Now we're going to look at ways of playing with the data to give us more training to do on a smaller data set to remove some of the issues that we might not have considered in creating that model. And that's called data augmentation. How do we do that? Looks like we go and see a dog here. So here we have a picture of a dog and you can see that there's different distortions. So this is dog distortion that we have here. You can see on the right. Can you point out to one of them that looks like for example where is shift and scale? I think these are the combination of different transformations. Some of them are like a scale, some of them are a shift, some of them are rotate. So what about this one? Oh come on, there we are. So the nose is shifted to the left. Yeah, it's more in the center and the original picture and it's scaled as well. It doesn't take up as much of the frame. Okay, so here's one way to expand the training set. And the reason for that is that if we do not have enough training data, there's a chance that the network just overfeed on some specific features. So by artificial expanding and training data, we would let the network see different aspects or see more features or learn more features of the data. For example, if we rotate or shift number three, it still would be number three. Still number three. And if we rotate or shift or scale or do a lasting deformation on a dog picture, it still would be a dog picture. Yeah, it wouldn't be an elephant or a giraffe. That's good. So let's look at other ways. Yes. So another way to let the network learn different features of input image is actually input dropout to randomly mask out some parts of image. For example, there's a chance that the network just overfeed on the, I don't know, engine of the airplane to decide whether it's there or it's an airplane. So by randomly mask different parts, the enforced network to learn different paths or different ways or different features to classify this image as airplane or for other images. All right, let's see. So random erasing is what we call this. And let's see another set of augmentation creating artificial data. So actually there are like different innovative ways to artificially expand our training set. For example, here they use different transformation, large light deformation, the composition, actually deformation and image overlapping. For example, here if we change the background of this image, it still would be a tiger. So by doing that, we increase the size of our training set. So I have a question here. We've done, we've dealt with so far training images. But what if we're training signals like biomedical signals, we could do the same thing, right? Probably the data augmentation should be like specific to that domain, to that data. What can we change about that data that doesn't change a label, but it still would be the same data. So it would be represented, the presentation of the same class. Okay, so let's look at a summary slide for all these different effects of augmenting our data set. So this is another example. We have geometry based augmentation. We have color based augmentation. For example, we can artificially generate different methods for the same image. And again, all of this would be the same animal. So the label is not changed, but they have different representation of the same data. So we saw the geometry based in detail for the dog and the erasing occlusion we saw for the aircraft previously. We didn't really look at the color in detail, but that was in the tiger selection. So here we have a complete expansion of a single picture. And there's probably close to 20 different variations. So we actually can significantly increase the number of data and hopefully use that to train our network. Exactly.