CogniVal: A framework for cognitive word embedding evaluation

PDF] CogniVal: A Framework for Cognitive Word Embedding Evaluation |  Semantic Scholar

An interesting method of evaluating word representations is by how much they reflect the semantic representations in the human brain. However, most, if not all, previous works only focus on small datasets and a single modality. In this paper, we present the first multimodal framework for evaluating English word representations based on cognitive lexical semantics. Six types of word embeddings are evaluated by fitting them to 15 datasets of eye-tracking, EEG and fMRI signals recorded during language processing. To achieve a global score over all evaluation hypotheses, we apply statistical significance testing accounting for the multiple comparisons problem. This framework is easily extensible and available to include other intrinsic and extrinsic evaluation methods. We find strong correlations in the results between cognitive datasets, across recording modalities and to their performance on extrinsic NLP tasks.

https://arxiv.org/pdf/1909.09001.pdf

评价一种词表示的有趣的指标是评估它反映人脑语义表示的程度。但是以前的工作仅仅专注于小规模数据以及单一模态。在本文中,我们为评估基于英文词表示的认知词语义任务提出了第一种多模态的框架。我们评估了六种词嵌入,这些词嵌入是通过眼部追踪,EEG,fMRI信号的15个数据集获得。为了获得全局评估得分,我们在比较问题上使用统计显著性测试来解决。这个框架可以方便的扩展和包括内在或者外在的评估方法。我们发现认知数据集之间,多模态之间,外部NLP任务之间有强相关。

发表评论

邮箱地址不会被公开。 必填项已用*标注