Pixel-Adaptive Convolutional Neural Networks (CVPR2019)

PDF] Pixel-Adaptive Convolutional Neural Networks | Semantic Scholar

论文地址:https://openaccess.thecvf.com/content_CVPR_2019/papers/Su_Pixel-Adaptive_Convolutional_Neural_Networks_CVPR_2019_paper.pdf

代码地址:https://suhangpro.github.io/pac/

Convolutions are the fundamental building blocks of CNNs. The fact that their weights are spatially shared is one of the main reasons for their widespread use, but it is also a major limitation, as it makes convolutions content agnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yet effective modification of standard convolutions, in which the filter weights are multiplied with a spatially varying kernel that depends on learnable, local pixel features. PAC is a generalization of several popular filtering techniques and thus can be used for a wide range of use cases. Specifically, we demonstrate state-of the-art performance when PAC is used for deep joint image up sampling. PAC also offers an effective alternative to fully-connected CRF (Full-CRF), called PAC-CRF, which performs competitively compared to Full-CRF, while being considerably faster. In addition, we also demonstrate that PAC can be used as a drop-in replacement for convolution layers in pre-trained networks, resulting in consistent performance improvements.

卷积是CNN的基本构成模块。事实上,权重在空间上共享是卷积被广泛使用的原因,但这也是主要的限制,因为这使得卷积的内容不可知。我们提出一个像素适应的卷积(PAC)操作,一个对于标准卷及的简单并有效的调整,其中滤波权重被乘以一个依赖于学习的局部像素特征的空间变化核。PAC是几种流行滤波技术的概括,可以被广泛应用于多种情况。特别地,我们展示了当PAC应用与深度链接图像上采样时的优越性能。PAC也可以给FULL-CRF提供有效的其他选择,称为PAC-CRF,相比与FULL-CRF展现除了可竞争的效果,与此同时明显速度更快。别且,我们也展示了PAC能够用做预训练网络卷积层中的drop-in代替,可以带来持续的性能提升。

论文理解:

出发点:卷积网络的一个重要思想在于卷积核共享,这样一次卷积完,提取了每个像素处的同一类型特征。 但是共享卷积核对于像语义分割这样的密集预测有其不可避免的缺陷,就是每个像素处的类容有些雷同,不利于像素间的区分。

核心思想就是:在原卷积核的基础上,再乘上一个 内容自适应核K。层卷积核仍然共享,但具体到每个像素处执行卷积时,先对卷积核先 乘上一个根据图像内容得到的核K。

发表评论

邮箱地址不会被公开。 必填项已用*标注