CSLM: Levenberg Marquardt based Back Propagation Algorithm Optimized with Cuckoo Search
DOI:
https://doi.org/10.5614/itbj.ict.res.appl.2013.7.2.1Abstract
Training an artificial neural network is an optimization task, since it is desired to find optimal weight sets for a neural network during training process. Traditional training algorithms such as back propagation have some drawbacks such as getting stuck in local minima and slow speed of convergence. This study combines the best features of two algorithms; i.e. Levenberg Marquardt back propagation (LMBP) and Cuckoo Search (CS) for improving the convergence speed of artificial neural networks (ANN) training. The proposed CSLM algorithm is trained on XOR and OR datasets. The experimental results show that the proposed CSLM algorithm has better performance than other similar hybrid variants used in this study.Downloads
References
Radhika, Y. & Shashi, M., Atmospheric Temperature Prediction using Support Vector Machines, International Journal of Computer Theory and Engineering, 1(1), pp. 1793-8201. 2009.
Akcayol, M.A. & Cinar, C., Artificial Neural Network Based Modeling of Heated Catalytic Converter Performance, Journal of Applied Thermal Engineering, 25, pp. 2341-2350, 2005.
Shereef, K.I. & Baboo, S.S., A New Weather Forecasting Technique using Back Propagation Neural Network with Modified Levenberg-Marquardt Algorithm for Learning, IJCSI International Journal of Computer Science, 8(6), pp. 1694-0814, 2011.
Kosko, B., Neural Network and Fuzzy Systems, 1st ed., Prentice Hall of India, 1994.
Krasnopolsky, V.M. & Chevallier, F., Some Neural Network application in environmental sciences. Part II: Advancing Computational Efficiency of Environmental Numerical Models, Neural Networks, 16(3-4), pp. 335-348, 2003.
Coppin, B., Artificial Intelligence Illuminated, Jones and Bartlet Illuminated Series, USA, Chapter 11, pp. 291- 324, 2004.
Basheer, I.A. & Hajmeer, M., Artificial Neural Networks: Fundamentals, Computing, Design, and Application, Journal of Microbiological Methods, 43(1), pp. 03-31, 2000.
Zheng, H., Meng, W. & Gong, B., Neural Network and its Application on Machinery Fault Diagnosis, IEEE International Conference on Systems Engineering (ICSYSE), pp. 576-579, 17-19 September, Kobe, Japan, 1992.
Rehman, M.Z. & Nawi, N.M., Improving the Accuracy of Gradient Descent Back Propagation Algorithm (GDAM) on Classification Problems, International Journal of New Computer Architectures and their Applications (IJNCAA), 1(4), pp.838-847, 2012.
Rumelhart D.E., Hinton G.E. & Williams R.J., Learning Representations by Back-Propagating Errors, Nature, 323, pp. 533-536, 1986.
Lahmiri, S., A Comparative Study of Backpropagation Algorithms in Financial Prediction, International Journal of Computer Science, Engineering and Applications (IJCSEA), 1(4), 2011.
Nawi, N.M., Ransing, R.S. & AbdulHamid, N., BPGD-AG: A New Improvement of Back-Propagation Neural Network Learning Algorithms with Adaptive Gain, Journal of Science and Technology, 2(2), 2011.
Ahmed, W., Saad, E. & Aziz, E., Modified Back Propagation Algorithm for Learning Artificial Neural Networks, The 18th National Radio Science Conference (NRSC), pp. 345-352, 27-29 March, Mansoura, Egypt, 2001.
Wen, J. Zhao, J.L., Luo. S.W. & Han, Z., The Improvements of BP Neural Network Learning Algorithm, 5th Int. Conf. on Signal Processing WCCC-ICSP, pp.1647-1649, 21-25 August, Beijing, China, 2000.
Salchenberger, L.M., Cinar, E.M. & Lash N.A., Neural Networks: A New Tool for Predicting Thrift Failures, Decision Sciences, 23(2), pp. 899-916, 1992.
Sexton, R.S., Dorsey, R.E. & Johnson, J.D., Toward Global Optimization of Neural Networks: A Comparison of the Genetic Algorithm and Backpropagation, Decision Support Systems, 22, pp. 171-186, 1998.
SNNS (Stuttgart Neural Network Simulator), http://wwwra.informatik. unituebingen.de/SNNS/, 25th January, 2013.
Fahlman, S.E., Faster-Learning Variations on Back Propagation: An Empirical Study, 1988 Connectionist Models Summer School by Scott E. Fahlman, pp. 38-51, San Mateo, CA, 1988.
Riedmiller, M. & Braun. H., A Direct Adaptive Method for Faster Back Propagation Learning: The RPROP Algorithm, IEEE International Conference on Neural Networks (ICNN93), 28th March-1st April, San Francisco, CA, 1993.
Fahlman, S.E. & Lebiere, C., The Cascade-Correlation Learning Architecture, Advances in Neural Information Processing Systems, 2, pp. 524-532, San Mateo, Calif, 1990.
Hagan, M.T. & Menhaj, M.B., Training Feed Forward Networks with the Marquardt Algorithm, IEEE Trans. on Neural Networks, 23, pp. 899-916, 1994.
Wilamowski, B.M., Cotton, N., Hewlett, J. & Kaynak, O., Neural Network Trainer with Second Order Learning Algorithms, 11th International Conference on Intelligent Engineering Systems, Budapest, Hungary, IEEE, 2007.
Hagan, M.T. & Menhaj, M.B., Training Feed Forward Networks with the Marquardt Algorithm, IEEE Trans. Neural Network., 5(6), pp. 989-993, 1994.
Xiao-ping, C., Chang-hua, H., Zhi-qiang, Z. & Ying-jie, L., Fault Prediction for Inertial Device Based on LMBP Neural Network, Electronics Optics & Control, 12 (6), pp.38-41, 2005.
Haykin, S., Neural Networks, Beijing, China Machine Press, pp. 501-522, 2004.
Nawi, N.M., Khan, A. & Rehman, M.Z., A New Levenberg Marquardt Based Back Propagation Algorithm Trained with Cuckoo Search, ICEEI, UKM, 2013.
Yan, J., Cao, H., Wang, J., Liu, Y. & Zhao, H., Levenberg-Marquardt Algorithm Applied to Forecast the Ice Conditions in Ningmeng Reach of The Yellow River, 5th International Conference on Natural Computation, pp. 184-188, 14-16 August, Tianjin, China, 2009.
Yang, X. & Deb, S., Cuckoo Search via Levy Flights, Proceedings of World Congress on Nature & Biologically Inspired Computing, India, pp. 210-214, 2009.
Yang, X. & Deb, S., Cuckoo Search via Levy Flights, Nature and Biologically Inspired Computing (NaBIC 2009), pp. 210-214, 2009.
Nawi, N.M., Khan, A. & Rehman, M.Z., A New Levenberg-Marquardt based Back-propagation Algorithm trained with Cuckoo Search, 4th International Conference on Electrical Engineering and Informatics (ICEEI), 24-25 June, Bangi, Malaysia, 2013.
