Free Lunch for Few-shot Learning: Distribution Calibration

Learning from a limited number of samples is challenging since the learned model can easily become overfitted based on the biased distribution formed by only a few training examples. In this paper, we calibrate the distribution of these few-sample classes by transferring statistics from the classes with sufficient examples, then an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. We assume every dimension in the feature representation follows a Gaussian distribution so that the mean and the variance of the distribution can borrow from that of similar classes whose statistics are better estimated with an adequate number of samples. Our method can be built on top of off-the-shelf pretrained feature extractors and classification models without extra parameters. We show that a simple logistic regression classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy on two datasets (~5% improvement on miniImageNet compared to the next best). The visualization of these generated features demonstrates that our calibrated distribution is an accurate estimation.

https://arxiv.org/abs/2101.06395

少样本学习一直是一个具有挑战性的任务,因为在少量训练样本上训练出来的模型容易过拟合到偏移的分布上。在本文中,我们通过充足的数据集上迁移统计信息用于校正少样本数据集上的偏移分布,然后可以从校正后的分布中采样足够多的样本用于训练。我们假设特征表示的每一维都服从高斯分布,那么这就意味着我们可以借鉴由大量数据统计的相似类的均值和方差。我们的方法可以与现成的预训练特征提取器和分类器在顶层进行合作并不需要引入额外参数。实验结果展示了在一个简单的逻辑回归分类器上使用经过校正的特征进行训练后可以在miniImageNet数据集上获得5%的性能提升。

发表评论

邮箱地址不会被公开。 必填项已用*标注