Regularization pytorch 1

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Here is what the above code is Doing:
1. We first create a new network with the same architecture as the original network.
2. We then loop over all the parameters in the original network and copy them over to the new network.
3. We then set the new network’s parameters to be in “eval” mode, which turns off dropout and batch normalization.
4. We then loop over the validation dataset and calculate the validation loss.
5. We then calculate the average validation loss.
6. Finally, we return the validation loss.

Similar Posts