Although neural networks are conventionally optimized towards zero training loss, it has been recently learned that targeting a non-zero training loss threshold, referred to as a flood level, often enables better test time generalization. Current approaches, however, apply the same constant flood level to all training samples, which inherently assumes all the samples have the same difficulty. We present AdaFlood, a novel flood regularization method that adapts the flood level of each training sample according to the difficulty of the sample. Intuitively, since training samples are not equal in difficulty, the target training loss should be conditioned on the instance. Experiments on datasets covering four diverse input modalities – text, images, asynchronous event sequences, and tabular – demonstrate the versatility of AdaFlood across data domains and noise levels.
Bibtex
@article{bae2024adaflood,
title={AdaFlood: Adaptive Flood Regularization},
author={Bae, Wonho and Ren, Yi and Ahmed, Mohamad Osama and Tung, Frederick and Sutherland, Danica J and Oliveira, Gabriel L},
journal={Transactions on Machine Learning Research (TMLR)},
year={2024}
}
Related Research
-
Interpretation for Variational Autoencoder Used to Generate Financial Synthetic Tabular Data
Interpretation for Variational Autoencoder Used to Generate Financial Synthetic Tabular Data
J. Wu, K. N. Plataniotis, *L. Z. Liu, *E. Amjadian, and Y. A. Lawryshyn. Special Issue Interpretability, Accountability and Robustness in Machine Learning (Algorithims)
Publications
-
ATOM: Attention Mixer for Efficient Dataset Distillation
ATOM: Attention Mixer for Efficient Dataset Distillation
*S. Khaki, *A. Sajedi, K. Wang, L. Z. Liu, Y. A. Lawryshyn, and K. N. Plataniotis. Oral presentation at The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)
Publications