An Empirical Comparative Review of Classifiers for Real-Time IoT Cyber Attack Detection

Abstract

The rapid growth of the Internet of Things (IoT) has significantly expanded the attack surface of modern networks, generating an urgent need for robust and efficient intrusion detection mechanisms. Machine learning (ML) is a promising approach to detect cyber attacks in IoT environments by automatically learning discriminative patterns from heterogeneous, real-world traffic. In this work we provide a comparative review and a thorough empirical evaluation of ten classifiers: logistic regression, decision trees, k-nearest neighbors, support vector machines, multilayer perceptrons, Random Forests, XGBoost, LightGBM, CatBoost, and a soft-voting ensemble on the RT-IoT2022 dataset, a realistic benchmark for real-time IoT attack detection. In addition to raw performance, we measure the impact of aggressive dimensionality reduction by comparing the full 83-feature space to a compact subset of 15 features, as determined by the consensus ranking of LightGBM and CatBoost feature importances. Experiments indicate that tree-based ensembles, CatBoost, XGBoost, and LightGBM, achieve the best performance, producing accuracy and AUC values above 99.7% and 99.9%, respectively, for both feature configurations, while maintaining balanced precision and recall across different attack types. Among the non-ensemble models, k-nearest neighbours is the strongest and loses only 0.05 accuracy points under the reduction, and the multilayer perceptron is equally insensitive, whereas logistic regression and the linear SVM lose 16--17 macro-F1 points. All comparisons are supported by stratified five-fold cross-validation, 95% confidence intervals and paired McNemar and Wilcoxon tests under Holm--Bonferroni correction, by a deployment cost profile (training and per-flow inference time), by a per-class analysis of minority-attack recall after SMOTE, and by external validation on NSL-KDD. This paper summarizes the advantages, disadvantages, and deployment considerations of state-of-the-art ML techniques and provides practical guidance for the development of adaptive, lightweight, and interpretable IoT intrusion detection systems.