Baidu, China’s Google, beat Google and Microsoft in the AI competition General Language Understanding Evaluation (GLUE).
GLUE is recognized for how well an AI system understands human language. It consists of nine different tests and a language model that scores highly on GLUE, therefore, can handle diverse reading comprehension tasks. Out of a full score of 100, the average person scores around 87 points. Baidu is now the first team to surpass 90 with its model, ERNIE.
The public leaderboard for GLUE is constantly changing. What’s notable about Baidu’s achievement is that it illustrates how AI research benefits from a diversity of contributors. Baidu’s researchers had to develop a technique specifically for the Chinese language to build ERNIE (which stands for “Enhanced Representation through kNowledge IntEgration”).
GLUE (General Language Understanding Evaluation) is a widely-recognized multi-task benchmark and analysis platform for natural language understanding (NLU). It comprises multiple NLU tasks including question answering, sentiment analysis, textual entailment, and an associated online platform for model evaluation, comparison, and analysis.
According to Baidu:
“ERNIE is a continual pre-training framework that builds and learns incrementally by pre-training tasks through sequential multi-task learning. We introduced ERNIE 1.0 early this year and released the improved ERNIE 2.0 model in July. The latter outperformed Google’s BERT and Carnegie Mellon University’s XLNet – competing pre-training models – in 16 NLP tasks in both Chinese and English.”
ERNIE model topped the public GLUE leaderboard, followed by Microsoft’s MT-DNN-SMART and Google’s T5.
Why does ERNIE perform so well?
The major contribution of ERNIE 2.0 is continual pre-training. Baidu’s researchers create different kinds of unsupervised pre-training tasks with the available big data and prior knowledge, and incrementally update the framework via multi-task learning. You can learn more from Baidu and Ernie 2.0.
