Pixel-Adaptive Convolutional Neural Networks (CVPR2019)

PDF] Pixel-Adaptive Convolutional Neural Networks | Semantic Scholar



Convolutions are the fundamental building blocks of CNNs. The fact that their weights are spatially shared is one of the main reasons for their widespread use, but it is also a major limitation, as it makes convolutions content agnostic. We propose a pixel-adaptive convolution (PAC) operation, a simple yet effective modification of standard convolutions, in which the filter weights are multiplied with a spatially varying kernel that depends on learnable, local pixel features. PAC is a generalization of several popular filtering techniques and thus can be used for a wide range of use cases. Specifically, we demonstrate state-of the-art performance when PAC is used for deep joint image up sampling. PAC also offers an effective alternative to fully-connected CRF (Full-CRF), called PAC-CRF, which performs competitively compared to Full-CRF, while being considerably faster. In addition, we also demonstrate that PAC can be used as a drop-in replacement for convolution layers in pre-trained networks, resulting in consistent performance improvements.



出发点:卷积网络的一个重要思想在于卷积核共享,这样一次卷积完,提取了每个像素处的同一类型特征。 但是共享卷积核对于像语义分割这样的密集预测有其不可避免的缺陷,就是每个像素处的类容有些雷同,不利于像素间的区分。

核心思想就是:在原卷积核的基础上,再乘上一个 内容自适应核K。层卷积核仍然共享,但具体到每个像素处执行卷积时,先对卷积核先 乘上一个根据图像内容得到的核K。


邮箱地址不会被公开。 必填项已用*标注