Learning Transferable Architectures for Scalable Image Recognition

Developing neural network image classification models often requires significant architecture engineering. In this paper, we study a method to learn the model architectures directly on the dataset of interest. As this approach is expensive when the dataset is large, we propose to search for an architectural building block on a small dataset and then transfer the block to a larger dataset. The key contribution of this work is the design of a new search space (the “NASNet search space”) which enables transferability. In our experiments, we search for the best convolutional layer (or “cell”) on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell, each with their own parameters to design a convolutional architecture, named “NASNet architecture”. We also introduce a new regularization technique called ScheduledDropPath that significantly improves generalization in the NASNet models. On CIFAR-10 itself, NASNet achieves 2.4% error rate, which is state-of-the-art. On ImageNet, NASNet achieves, among the published works, state-of-the-art accuracy of 82.7% top-1 and 96.2% top-5 on ImageNet. Our model is 1.2% better in top-1 accuracy than the best human-invented architectures while having 9 billion fewer FLOPS – a reduction of 28% in computational demand from the previous state-of-the-art model. When evaluated at different levels of computational cost, accuracies of NASNets exceed those of the state-of-the-art human-designed models. For instance, a small version of NASNet also achieves 74% top-1 accuracy, which is 3.1% better than equivalently-sized, state-of-the-art models for mobile platforms. Finally, the learned features by NASNet used with the Faster-RCNN framework surpass state-of-the-art by 4.0% achieving 43.1% mAP on the COCO dataset.

https://arxiv.org/pdf/1707.07012.pdf

针对图像识别任务的神经网络设计工作往往费时费力。在本文中,我们提出一种可以直接从数据集学习特定模型结构的方法。因为在面对大型数据集的时候本方法比较笨重,我们提出了一种方法:在较小的数据集上搜索神经网络结构然后将这个结构传递至较大的数据集上。本文关键的贡献在于设计了一个新型的搜索空间(NASNet搜索空间)用于传递网络结构。在我们的试验中,我们在CIFAR-10数据集上搜索最佳的卷积层然后将这些层堆叠起来应用在ImageNet数据集上,这些层我们称为NASNet架构。我们还介绍了一种新正则化的方法叫ScheduledDropPath,它可以显著提高NASNet的泛化能力。

发表评论

邮箱地址不会被公开。 必填项已用*标注