Adaptive Multi-level Backward Tracking for Sequential Feature Selection
DOI:
https://doi.org/10.5614/itbj.ict.res.appl.2021.15.1.1Keywords:
classification accuracy, data mining, dimensionality reduction, sequential feature selection, supervised learning, wrapper approachAbstract
In the past few decades, the large amount of available data has become a major challenge in data mining and machine learning. Feature selection is a significant preprocessing step for selecting the most informative features by removing irrelevant and redundant features, especially for large datasets. These selected features play an important role in information searching and enhancing the performance of machine learning models. In this research, we propose a new technique called One-level Forward Multi-level Backward Selection (OFMB). The proposed algorithm consists of two phases. The first phase aims to create preliminarily selected subsets. The second phase provides an improvement on the previous result by an adaptive multi-level backward searching technique. Hence, the idea is to apply an improvement step during the feature addition and an adaptive search method on the backtracking step. We have tested our algorithm on twelve standard UCI datasets based on k-nearest neighbor and naive Bayes classifiers. Their accuracy was then compared with some popular methods. OFMB showed better results than the other sequential forward searching techniques for most of the tested datasets.
Downloads
References
Pavya, K. & Srinivasan, B., Feature Selection Techniques in Data Mining: A Study, International Journal of Scientific Development and Research (IJSDR), 2(6), pp. 594-598, 2017.
Cai, J., Luo, J., Wang, S. & Yang, S., Feature Selection in Machine Learning: A New Perspective, Neurocomputing, pp. 70-79, 2018.
Sutha, K. & Tamilselvi, D.J.J., A Review of Feature Selection Algorithms for Data Mining Techniques, International Journal on Computer Science and Engineering (IJCSE), pp. 63-67, 2015.
Somol, P., Novovicova, J. & Pudil, P., Flexible-Hybrid Sequential Floating Search in Statistical Feature Selection, Structural, Syntactic, and Statistical Pattern Recognition, Springer, pp. 632-639, 2006.
Jovic, A., Brkic, K. & Bogunovic, N., A Review of Feature Selection Methods with Applications, 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1200-1205, 2015.
Homsapaya, K. & Sornil, O., Improving Floating Search Feature Selection using Genetic Algorithm, Journal of ICT Research and Applications, 11(3), pp. 299-317, 2017.
Kadhum, M., Manaseer, S. & Dalhoum, A.L.A., Evaluation Feature Selection Technique on Classification by Using Evolutionary ELM Wrapper Method with Features Priorities, Journal of Advances in Information Technology, 12(1), pp. 21-28, 2021.
Al-tashi, Q., Abdulkadir, S.J., Rais, H. M., Mirjalili, S. & Alhussian, H., Approaches to Multi-Objective Feature Selection: A Systematic Literature Review, IEEE Access, 8, pp. 125076-125096, 2020.
Wan, Y., Ma, A., Zhong, Y., Hu, X. & Zhang L., Multiobjective Hyperspectral Feature Selection Based on Discrete Sine Cosine Algorithm, IEEE Transactions on Geoscience and Remote Sensing, 58(5), pp. 3601-3618, 2020.
Al-tashi, Q., Abdulkadir, S.J., Rais, H.M., Mirjalili, S., Alhussian, H., Ragab, M.G. & Alqushaibi, A., Binary Multi-Objective Grey Wolf Optimizer for Feature Selection in Classification. IEEE Access, 8, pp. 106247-106263, 2020.
Chotchantarakun, K. & Sornil, O., An Adaptive Multi-levels Sequential Feature Selection, International Journal of Computer Information Systems and Industrial Management Applications, 13, pp. 010-019, 2021.
Bolon-Canedo, V. & Alonso-Betanzos, A., Ensembles for Feature Selection: A Review and Future Trends, Information Fusion, 52, pp. 1-12, 2019.
Cisotto, G., Capuzzo, M., Guglielmi, A.V. & Zanella, A., Feature Selection for Gesture Recognition in Internet-of-Things for Healthcare, International Conference on Communication(ICC), Dublin, Ireland, 7-11 June, 2020.
Raj, R.J.S., Shobana, S.J., Pustokhina, I.V., Pustokhin, D.A., Gupta, D. & Shankar, K., Optimal Feature Selection-Based Medical Image Classification Using Deep Learning Model in Internet of Medical Things, IEEE Access, 8, pp. 58006-58017, 2020.
Liu, W. & Wang, J., A Brief Survey on Nature-Inspired Metaheuristics for Feature Selection in Classification in this Decade, Proceedings of the 2019 IEEE 16th International Conference on Networking, Sensing and Control, pp. 424-429, 2019.
Huda, R.K. & Banka, H., New Efficient Initialization and Updating Mechanisms in PSO for Feature Selection and Classification, Neural Computing and Applications, 32, pp. 3283-3294, 2019.
Whitney, A.W., A Direct Method of Nonparametric Measurement Selection, IEEE Transactions on Computers, C-20(9), pp. 1100-1103, 1971.
Pudil, P., Novovicova, J. & Kittler, J., Floating Search Methods in Feature Selection, Pattern Recognition Letters, 15(11), pp. 1119-1125, 1994.
Somol, P., Pudil, P., Novovicova, J. & Paclik, P., Adaptive Floating Search Methods in Feature Selection, Pattern Recognition Letters, 20(11-13), pp. 1157-1163, 1999.
Nakariyakul, S. & Casasent, D.P., An Improvement on Floating Search Algorithms for Feature Subset Selection, Pattern Recognition, 42(9), pp. 1932-1940, 2009.
