A Novel Higher-Order Semantic Kernel for Text Classification

Abstract

In conventional text categorization algorithms, documents are symbolized as bag of words (BOW) with the fact that documents are supposed to be independent from each other. While this approach simplifies the models, it ignores the semantic information between terms of each document. In this study, we develop a novel method to measure semantic similarity based on higher-order dependencies between documents. We propose a kernel for Support Vector Machines (SVM) algorithm using these dependencies which is called Higher-Order Semantic Kernel. With the aim of presenting comparative performance of Higher-Order Semantic Kernel we performed many experiments not only with our algorithm but also with existing traditional first-order kernels such as Polynomial Kernel, Radial Basis Function Kernel, and Linear Kernel. The experiments using Higher-Order Semantic Kernel on several well-known datasets show that classification performance improves significantly over the first-order methods.