Generic decoding of seen and imagined objects using hierarchical visual features

Generic decoding of seen and imagined objects using hierarchical visual  features | Nature Communications

Object recognition is a key function in both human and machine vision. While brain decoding of seen and imagined objects has been achieved, the prediction is limited to training examples. We present a decoding approach for arbitrary objects using the machine vision principle that an object category is represented by a set of features rendered invariant through hierarchical processing. We show that visual features, including those derived from a deep convolutional neural network, can be predicted from fMRI patterns, and that greater accuracy is achieved for low-/high-level features with lower-/higher-level visual areas, respectively. Predicted features are used to identify seen/imagined object categories (extending beyond decoder training) from a set of computed features for numerous object images. Furthermore, decoding of imagined objects reveals progressive recruitment of higher-to-lower visual representations. Our results demonstrate a homology between human and machine vision and its utility for brain-based information retrieval.

https://www.nature.com/articles/ncomms15037

目标识别是人类和机器视觉的主要功能。虽然对于看见或者想象目标时的大脑解码任务已经可以做到,但是预测性能还受制于训练样本。我们提出了一个可以作用于任意目标的解码方法,它利用机器视觉的机制:一个目标在不同层级的模型中会以无关的特征形式呈现。我们发现从深度卷积神经网络中获得的视觉特征也可以由fMRI图像中预测获得,对应的低-低,高-高层级特征可以获得更高的预测精度。预测出来的特征还能够用于识别看到或者想到的目标(甚至是训练时没有用到的类别)。试验结果更加证明了视觉特征是从高到低等级渐进表示的。我们的实验还证明了人类和机器视觉拥有相同的作用机制。

发表评论

邮箱地址不会被公开。 必填项已用*标注