Cross Validation

Cross-Validation

• Used more commonly for small datasets - not so much in deep learning.
• Split data into folds, try each fold as validation & average the results
• Cycle through choosing which fold will be the validation set.
  • Train hyperparameters on folds 1-4, Validate on fold 5. Test
  • Train hyperparameters on folds 1-3, Validate on fold 4, train again on fold 5. Test
  • Train hyperparameters on folds 1-2, Validate on fold 3, train again on folds 4-5. - Test → so forth