An Empirical Study on Large Scale Text Classification with Skip-Gram Embeddings

Georgios Balikas, Massih-Reza Amini
Laboratoire d'Informatique de Grenoble
700, avenue Centrale
38058 Saint-Martin d'Hérès

We investigate the integration of word embeddings as classification features in the setting of large scale text classification. Such representations have been used in a plethora of tasks, however their application in classification scenarios with thousands of classes has not been extensively researched, partially due to hardware limitations. In this work, we examine efficient composition functions to obtain document-level from word-level embeddings and we subsequently investigate their combination with the traditional one-hot-encoding representations. By presenting empirical evidence on large, multi-class, multi-label classification problems, we demonstrate the eciency and the performance benefits of this combination.