Weight assignment for underfitting and overfitting

## Practice full name

Make a particular assignment to the weights in order to solve underfitting and overfitting

## About

Underfitting and overfitting are two cases that are not welcome when predictions are made as decidedly unrealistic estimates are provided. What we try to do is assign particular values to the weights in order to make the model neither too precise nor too imprecise during the training phase, so as to be able to correctly generalize its behavior on new datasets.

## What the crowd is saying

Is this practice best or bad?

In the opinion of the participants, this practice has a great tendency for which it can be considered as best.

Looking at more details, there is much more confidence in the answers that classify the practice as best.

## Involved topics

### What is a weight?

Weight is the parameter within a neural network that transforms input data within the network's hidden layers. As an input enters the node, it gets multiplied by a weight value and the resulting output is either observed, or passed to the next layer in the neural network. Often the weights of a neural network are contained within the hidden layers of the network [1].

### Weight initialization

Weight initialization is an important design choice when developing deep learning neural network models. Historically, weight initialization involved using small random numbers, although over the last decade, more specific heuristics have been developed that use information, such as the type of activation function that is being used and the number of inputs to the node. These more tailored heuristics can result in more effective training of neural network models using the stochastic gradient descent optimization algorithm [2].

### Underfitting and overfitting

Underfitting refers to a model that can neither model the training data nor generalize to new data. An underfit machine learning model is not a suitable model and will be obvious as it will have poor performance on the training data.

Overfitting refers to a model that models the training data too well. Overfitting happens when a model learns the detail and noise in the training data to the extent that it negatively impacts the performance of the model on new data. This means that the noise or random fluctuations in the training data is picked up and learned as concepts by the model. The problem is that these concepts do not apply to new data and negatively impact the models ability to generalize [3].

### What is model training?

Model training is a process in which a machine learning algorithm is fed with sufficient training data to learn from. To do this, it's necessary a training dataset, which consists of the sample output data and the corresponding sets of input data that have an influence on the output. The training model is used to run the input data through the algorithm to correlate the processed output against the sample output [4].

### What is a sensitive feature?

Sensitive features (or protected attributes) are attributes for which the machine learning algorithm could discriminate subjects. Some common examples of this attributes are age, disability, national origin, race/color, religion, sex. Some cases where discrimination has occurred are facial recognition and hiring. The theory holds that oversampling and undersampling techniques can cause discrimination [5].

## Sources

1. Weight (Artificial Neural Network)
2. Weight Initialization for Deep Learning Neural Networks
3. Overfitting and Underfitting With Machine Learning Algorithms
4. What Is Model Training?
5. Handling Discriminatory Biases in Data for Machine Learning

**Other useful links**

* Weight Initialization Techniques in Neural Networks

## How to contribute

* You can use the "comments" section of the wiki. You can find it at the bottom of the article page that you want to edit. Suggest here your edits, the administrator will evaluate whether to accept them and report them on the page.
* You can send an email to *abatefrancesco98@gmail.com* setting as subject the page you want to edit.

Any interesting information will be reported on this page.