Emotion Recognition from Facial Expressions using Images with Pose, Illumination and Age Variation for Human-Computer/Robot Interaction
DOI:
https://doi.org/10.5614/itbj.ict.res.appl.2018.12.1.2Keywords:
classification, emotion, facial expression, feature extraction, illumination, pose, Raspberry Pi II.Abstract
A technique for emotion recognition from facial expressions in images with simultaneous pose, illumination and age variation in real time is proposed in this paper. The basic emotions considered are anger, disgust, happy, surprise, and neutral. Feature vectors that were formed from images from the CMU-MultiPIE database for pose and illumination were used for training the classifier. For real-time implementation, Raspberry Pi II was used, which can be placed on a robot to recognize emotions in interactive real-time applications. The proposed method includes face detection using Viola Jones Haar cascade, Active Shape Model (ASM) for feature extraction, and AdaBoost for classification in real- time. Performance of the proposed method was validated in real time by testing with subjects from different age groups expressing basic emotions with varying pose and illumination. 96% recognition accuracy at an average time of 120ms was obtained. The results are encouraging, as the proposed method gives better accuracy with higher speed compared to existing methods from the literature. The major contribution and strength of the proposed method lie in marking suitable feature points on the face, its speed and invariance to pose, illumination and age in real time.
Downloads
References
Leo, M., Coco, M.D., Carcagni, P., Distante, C., Bernava, M., Pioggia, G. & Palestra, G., Automatic Emotion Recognition in Robot-Children Interaction for ASD Treatment, International Conference on Computer Vision Workshops, IEEE, pp. 145-153, 2015.
Happy, S.L., Dasgupta, A., Priyadarshi, P. & Routray, A., Automated Alertness and Emotion Detection for Empathic Feedback during E-Learning, 5th International Conference on Technology for Education (T4E), IEEE, pp. 47-50, 2013.
Ian J. Goodfellow, Erhan, D., Carrier, P.L., Courville A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.H., Zhou, Y., Ramaiah, C., Feng, F., Li, R., Wang, X., Athanasakis, D., Taylor, J.S., Milakov, M., Park, J., Ionescu, R., Popescu, M., Grozea, C., Bergstra, J., Xie, J., Romaszko, L., Xu, B., Chuang, Z. & Bengio Y., Challenges in Representation Learning: a Report on Three Machine Learning Contests, Workshop in Challenges in Representation Learning (ICML), Atlanta, United States, pp. 1-8, 2013.
Leo, M., Medioni, G., Trivedi, M., Kanade, T. & Farinella, G.M., Computer Vision for Assistive Technologies, Computer Vision and Image Understanding, 154, pp. 1-15, 2017.
Suchitra, Suja, P. & Shikha, T., Real Time Emotion Recognition from Facial Images using Raspberry Pi II, 3rd International Conference on Signal Processing and Integrated Networks (SPIN), Noida, India, pp. 666-670, 2016.
Suja, P. & Shikha, T., Analysis of Emotion Recognition from Facial Expressions using Spatial and Transform Domain Methods, International Journal of Advanced Intelligence Paradigms, 7(1), pp. 57-73, 2015.
Suja, P., Thomas, S.M., Shikha, T. & Madan, V.K., Emotion Recognition from Images under Varying Illumination Conditions, 6th International Workshop on Soft Computing Applications (SOFA), Springer, pp. 913-921, 2016.
Happy, S.L. & Routray, A., Automatic Facial Expression Recognition using Features of Salient Facial Patches, IEEE Transactions on Affective Computing, 6(1), pp. 1-12, 2015.
Rudovic, O., Maja P. & Ioannis (Yiannis), P., Coupled Gaussian Processes for Pose-invariant Facial Expression Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6), pp. 1357-1369, 2013.
Benta, K.I. & Vaida, M.F., Towards Real-life Facial Expression Recognition Systems, Advances in Electrical and Computer Engineering, 15(2), pp. 93-102, 2015.
Cootes, T.F, Edwards, G.J. & Taylor, C.J., Active Appearance Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), pp. 681-685, 2001.
Rudovic, O., P. Ioannis (Yiannis) & Maja, P., Facial Expression Invariant Head Pose Normalization using Gaussian Process Regression, International Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 28-33, 2010.
Paul, V. & Micheal., J., Robust Real-time Face Detection, International Journal of Computer Vision, 57(2), pp.137-154, 2004.
Cootes, T.F., Taylor, C.J. & Graham, J., ASM - Their Training and Application, Computer Vision and Image Understanding, 61(1), pp.38-59, 1995.
Indraneel, M. & Schapire, R.E., A Theory of Multiclass Boosting, Journal of Machine Learning Research, 14, pp. 437-497, 2013.
Gross, R., Matthews, I., Cohn, J., Kanade, T. & Baker S., Guide to the CMU Multi-PIE database, The Robotics Institute, Carnegie Mellon University, Technical report, 2007.
Abdat, F., Maaoui, C. & Pruski, C., Human-computer Interaction using Emotion Recognition from Facial Expression, 5thUKSim European Symposium on Computer Modeling and Simulation. IEEE, pp. 196-201, 2011.
Myunghoon, S. & Prabhakaran, B., Real-time Mobile Facial Expression Recognition System - a Case Study, Conference on Computer Vision and Patten Recognition Workshops (CVPR),IEEE, pp. 132-137, 2014.
Anderson, K. & McOwan, P.W.,A Real-time Automated System for Recognition of Human Facial Expressions, IEEE Transactions on Systems, Man, and Cybernetics, Part B, 36(1), pp. 96-105, 2006.
Barnard, E. & Botha, E.C., Back-propagation Uses Prior Information Efficiently, IEEE Transactions on Neural Networks, 4(5), pp. 794-802, 1993.
