SABSVM: Speed-Accuracy Boosted SVM with Compact Basis Set from Analytic Mappers via Krein-Hilbert Subspace Compression

Abstract

MKL with SVM learns a combination of kernels that are individually associated with a RKHS and the resultant kernel induces a corresponding synthesized rich RKHS that makes the machine ready to handle data from different and heterogeneous subspaces of the input space sources. However, while providing many and highly flexible base kernels with SVM oriented MKL algorithm may overfit data and boost the test cost by generating numerous Support Vectors (SVs), usually, best performing MKL involve complex optimizations like SDP, SOCP, SILP turning these methods unviable in training phase by the surge of combined kernel number with memory exhaustion. In this circumstance, moving towards extreme challenge to achieve the top in both classification accuracy and speed simultaneously, in our novel SABSVM algorithm, we avoid overfitting, reduce training and test load by taking efficient combination of a universal Positive Definite (PD) kernel and a learning-theory supported optimal analytic mapper that resultantly induces a combined RKHS-RKKS generated rich space as the base of our SVM based detector. Then finding its subspace compression gives further compact subspace approximation of the target function using a significantly much smaller subset of the main kernel-vector (SV) set. Experiments on benchmark and synthetic data exposes that our method poses classification accuracy similar to that of state-of-the-art classifiers while demanding substantially lower classification cost. It gives test classification accuracy almost as good as these higher complexity based top-accuracy giving SKL & MKL classifiers while competes easily with the reduced versions of such, which is clear from its lowest Machine's Error-Cost Loss (MECL) on benchmark data. Moreover, sparsity of our model increases with the increment of the training data-size indicating its higher efficiency on larger data.