Pre-trained models for natural language processing: A survey

PDF] Pre-trained Models for Natural Language Processing: A Survey |  Semantic Scholar

Recently, the emergence of pre-trained models (PTMs) has brought natural language processing (NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next, we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.

(本文是预训练语言模型的综述文章)近来,预训练模型的出现将自然语言处理带入新的领域。在本篇综述中,我们为自然语言处理任务的预训练模型提供一个全面的视角。然后我们系统地将现有的预训练模型分为四类。之后我们如何将现有的预训练模型适配下游任务。最后,我们指出了一些预训练模型潜在的发展方向。本篇综述的目的是作为一个理解、应用和开发自然语言处理预训练模型的上手指南。

发表评论

邮箱地址不会被公开。 必填项已用*标注