An Adaptive Support Method for Training SVM Problems

Abstract

In recent years, extensive research has been conducted on support vector machines (SVMs) and their applications across various scientific fields. Among the various algorithms used for classification and regression in different domains, SVMs are distinguished by their remarkable power and stability. In this context, this study introduces the design and implementation of the Adaptive Support Method-Support Vector Machine (ASM-SVM), a fast training algorithm for SVM classification, a fundamental task in machine learning. The proposed approach is iterative and determines in a finite number of iterations an optimal or suboptimal solution for the QP (Quadratic Programming) problem. At each iteration of our algorithm, a descent feasible direction and a step size are calculated in order to decrease the value of the objective function and the suboptimality estimate. The major advantage of our algorithm resides in its ability to modify several non-basic variables simultaneously, unlike active-set methods, which only permit changing one variable per iteration. This feature permit reducing the number of iterations and the computational cost that is particularly important for many applications. Furthermore, our solution eliminates the need to compute the null-space matrix, making it possible to propose a simple and efficient method suitable for machine implementation. Numerical results obtained on several datasets prove its effectiveness in training both linear and nonlinear SVM classification problems. The ASM-SVM method is designed for easy integration into machine learning software systems.