Inducing brain-relevant bias in natural language processing models

Progress in natural language processing (NLP) models that estimate representations of word sequences has recently been leveraged to improve the understanding of language processing in the brain. However, these models have not been specifically designed to capture the way the brain represents language meaning. We hypothesize that fine-tuning these models to predict recordings of brain activity of people reading text will lead to representations that encode more brain-activity-relevant language information. We demonstrate that a version of BERT, a recently introduced and powerful language model, can improve the prediction of brain activity after fine-tuning. We show that the relationship between language and brain activity learned by BERT during this fine-tuning transfers across multiple participants. We also show that, for some participants, the fine-tuned representations learned from both magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) are better for predicting fMRI than the representations learned from fMRI alone, indicating that the learned representations capture brain-activity-relevant information that is not simply an artifact of the modality. While changes to language representations help the model predict brain activity, they also do not harm the model’s ability to perform downstream NLP tasks. Our findings are notable for research on language understanding in the brain.

http://papers.nips.cc/paper/9559-inducing-brain-relevant-bias-in-natural-language-processing-models

自然语言处理模型对于语言序列预测的处理流程已经被认为对于理解大脑语言处理的流程有重要帮助。但是,这些模型并没有被特别为了捕捉脑部表示语言含义流程而设计。我们提供一个假设:利用测试者在朗读时的脑部活动记录fine-tuning自然语言处理模型可以使得这些模型学习到更多脑部活动相关的语言信息。我们提出了一个在fine-tuning后可以提高预测脑部活动BERT的变种,BERT是一种最近提出的性能强大的语言模型。我们通过MEG和fMRI多域数据的实验揭示了语言和大脑活动的相关性,并且说明了语言模型学习到的特征是捕捉到脑部活动相关信息的,而不是无意义的。当利用语言表示去预测脑部活动的时候,模型依然能够很好地执行downstream的自然语言处理任务。

发表评论

邮箱地址不会被公开。 必填项已用*标注