Heuristic Evaluation of Play4Fit Health and Fitness App: A Comparison Between Experts and Novices Evaluators
DOI:
https://doi.org/10.5614/j.vad.2023.15.1.2Keywords:
experts, gamification, heuristic evaluation, health and fitness, novices, smartphone appAbstract
Heuristic evaluation (HE) can be used to effectively identify usability issues in various interfaces. However, it has not been widely used in evaluating smartphone apps, especially in the health and fitness domain. One reason is the lack of HCI experts, which makes incorporating HE into the design process difficult. This paper presents the results of a study that compared HE performed by three HCI experts and three novices in evaluating a gamification app for health and fitness on a smartphone. The study used Smartphone Mobile Application heuRisTics (SMART), which focuses on smartphone apps, and a severity rating scale to determine the severity of the usability issues. These issues were mapped to the SMART heuristic. The findings indicate that novices may identify usability issues that the experts overlooked. While the experts identified eighteen usability issues, the novices found only four; however, the novice?s findings may be used as a substitute for HE when experts are unavailable. Both experts and novices identified two similar usability issues, but their severity ratings differed. One possible solution to address the lack of usability issues identified by novices in HE is to use more novices instead of experts in the evaluation process.
References
Mellin, A.E., Neumark-Sztainer, D., Story, M., Ireland, M. & Resnick, M.D., Unhealthy Behaviors and Psychosocial Difficulties among Overweight Adolescents: The Potential Impact of Familial Factors. Journal of Adolescent Health, 31(2), pp. 145-153, 2002.
Thomas, L., Children Spend More Time at Computers or TV than Exercising Every Week, Daily Mail Online, http://www.dailymail.co.uk/sciencetech/ article-1352361/Children-spend-time-computers-TV-exercising-week.html, 2011.
Wurmser, Y., Time Spent with Media 2019, https://www.emarketer.com/content/us-time-spent-with-mobile-2019, 2019.
Shephard, R.J., A History of Health & Fitness: Implications for Policy Today, Springer International Publishing, 2018.
Hamilton, M.T., Healy, G.N., Dunstan, D.W., Zderic, T.W. & Owen, N., Too Little Exercise and Too Much Sitting: Inactivity Physiology and the Need for New Recommendations on Sedentary Behavior, Current Cardiovascular Risk Reports, 2(4), pp. 292-298, 2008.
Jin, J., Yun, J. & Agiovlasitis, S., Impact of Enjoyment on Physical Activity and Health among Children with Disabilities in Schools, Disability and Health Journal, 11(1), pp. 14-19, 2018.
Arps, E., The Use of Internet and Mobile Phone Based Health Promotion Interventions in Youth Populations, In Health Promotion Forum (HPF), Newmarket, Auckland, 2014.
Khadjesari, Z., Murray, E., Hewitt, C., Hartley, S. & Godfrey, C., Can Stand?alone Computer?based Interventions Reduce Alcohol Consumption? A Systematic Review, Addiction, 106(2), pp. 267-282, 2011.
O?Reilly, G.A. & Spruijt-Metz, D., Current mHealth Technologies for Physical Activity Assessment and Promotion, American Journal of Preventive Medicine, 45(4), pp. 501-507, 2013.
Yoganathan, D. & Kajanan, S., Persuasive Technology for Smartphone Fitness Apps, PACIS, p. 185, 2013.
Ilhan, A. & Henkel, M., 10,000 Steps a Day for Health? User-based Evaluation of Wearable Activity Trackers, 51st Hawaii International Conference on System Sciences, pp. 3376-3385, 2018.
Kapp, K.M., The Gamification of Learning and Instruction: Game-based Methods and Strategies for Training and Education, John Wiley & Sons, 2012.
Deterding, S., Dixon, D., Khaled, R. & Nacke, L., From Game Design Elements to Gamefulness: Defining Gamification, 15th International Academic Mindtrek Conference: Envisioning Future Media Environments, pp. 9-15, 2011.
Cugelman, B., Gamification: What it is and Why it Matters to Digital Health Behavior Change Developers, JMIR Serious Games, 1(1), pp. e3. 2013.
Pereira, P., Duarte, E., Rebelo, F. & Noriega, P., A Review of Gamification for Health-related Contexts, International Conference of Design, User Experience, and Usability, pp. 742-753, 2014.
Charles, D., Charles, T., McNeill, M., Bustard, D. & Black, M., Game?based Feedback for Educational Multi?user Virtual Environments, British Journal of Educational Technology, 42(4), pp. 638-654, 2011.
Domguez, A., Saenz-de-Navarrete, J., De-Marcos, L., Ferndez-Sanz, L., Pag, C. & Martez-Herrz, J.J., Gamifying Learning Experiences: Practical Implications and Outcomes, Computers & Education, 63, pp. 380-392, 2013.
Shneiderman, B., Plaisant, C., Cohen, M.S., Jacobs, S., Elmqvist, N., & Diakopoulos, N., Designing the User Interface: Strategies for Effective Human-computer Interaction, Pearson, 2016.
Nielsen, J., How to Conduct a Heuristic Evaluation?, Nielsen Norman Group, 1, pp. 1-8, 1995.
Muniz, An Introduction to Heuristic Evaluation - Usability Geek, https://usabilitygeek.com/heuristic-evaluation-introduction/, 2016.
Zhang, J., Johnson, T.R., Patel, V.L., Paige, D.L. & Kubose, T., Using Usability Heuristics to Evaluate Patient Safety of Medical Devices, Journal of Biomedical Informatics, 36(1-2), pp. 23-30, 2003.
Wong, Heuristic Evaluation: How to Conduct a Heuristic Evaluation, Interaction Design Foundation, https://www.interaction-design.org/literature/article/heuristic-evaluation-how-to-conduct-a-heuristic-evaluation, 2017.
Looyestyn, J., Kernot, J., Boshoff, K., Ryan, J., Edney, S. & Maher, C., Does Gamification Increase Engagement with Online Programs? A Systematic Review, PloS one, 12(3), e0173403, 2017.
Gartner, Gartner Predicts over 70 Percent of Global 2000 Organisations Will Have at least One Gamified Application by 2014, https://www.gartner.com/newsroom/id/1844115, 2012.
Sydes, L., Why Gamified Health Apps aren?t as Successful as They Could Be ? Badgeville, https://badgeville.com/why-gamified-health-apps-arent-as-successful-as-they-could-be/, 2014.
Lee, J.J. & Hammer, J., Gamification in Education: What, How, Why Bother?, Academic Exchange Quarterly, 15(2), pp. 146, 2011.
Beauchemin, M., Gradilla, M., Baik, D., Cho, H. & Schnall, R., A Multi-step Usability Evaluation of a Self-management App to Support Medication Adherence in Persons Living with HIV, International Journal of Medical Informatics, 122, pp. 37-44, 2019.
Kekkonen, M. & Oinas-Kukkonen, H., Social Comparison in Behavior Change Support Systems: Heuristic Evaluation of a System's Usability, BCSS@ PERSUASIVE, pp.1-6, 2019.
Katurura, M. & Cilliers, L., Privacy in Wearable Health Devices: How Does POPIA Measure up?, 4th International Conference on the Internet, Cyber Security and Information Systems 2019, 12, pp. 112-122, 2019.
Edwards, E.A., Lumsden, J., Rivas, C., Steed, L., Edwards, L.A., Thiyagarajan, et al., Gamification for Health Promotion: Systematic Review of Behaviour Change Techniques in Smartphone Apps, BMJ Open, 6(10), 2016.
Tu, R., Hsieh, P. & Feng, W., Walking for Fun or for ?Likes?? The Impacts of Different Gamification Orientations of Fitness Apps on Consumers? Physical Activities, Sport Management Review, 22(5), pp. 682-693, 2019.
Kappen, D.L., Mirza-Babaei, P. & Nacke, L.E., Motivational Affordances for Older Adults? Physical Activity Technology: An Expert Evaluation, International Conference on Human-Computer Interaction, pp. 388-406, 2019.
Garc-Holgado, A., Reiris, I.T., Kearney, N., Martinus, C. & Garc-Pelvo, F.J., An App to Support Yoga Teachers to Implement a Yoga-based Approach to Promote Wellbeing among Young People: Usability Study, In International Conference on Human-Computer Interaction, pp. 38-49, 2019.
Zaror, C., Espinoza?Espinoza, G., Atala?Acevedo, C., Muz?Mill, P., Li, Y., Clarke, K., et al., Validation and Usability of a Mobile Phone Application for Epidemiological Surveillance of Traumatic Dental Injuries, Dental Traumatology, 35(1), pp. 33-40, 2019.
Kumar, A. & Vithani, T., A Comprehensive Mobile Application Development and Testing Lifecycle, 2014 IT Professional Conference, pp. 1-27, 2014.
Joyce, G. & Lilley, M., Towards the Development of Usability Heuristics for Native Smartphone Mobile Applications, International Conference of Design, User Experience, and Usability, pp. 465-474, 2014.
Othman, M.K., Sulaiman, M.N.S. & Aman, S., Heuristic Evaluation: Comparing Generic and Specific Usability Heuristics for Identification of Usability Problems in a Living Museum Mobile Guide App, Advances in Human-Computer Interaction, 2018.
Inostroza, R., Rusu, C., Roncagliolo, S., Rusu, V. & Collazos, C.A., Developing SMASH: A Set of SMArtphone?s uSability Heuristics, Computer Standards & Interfaces, 43, pp. 40-52, 2016.
Bertini, E., Gabrielli, S., Kimani, S., Catarci, T. & Santucci, G., Appropriating and Assessing Heuristics for Mobile Computing, Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 119-126, 2006.
Inostroza, R., Rusu, C., Roncagliolo, S. & Rusu, V., Usability Heuristics for Touchscreen-based Mobile Devices: Update, Proceedings of the 2013 Chilean Conference on Human-Computer Interaction, pp. 24-29, 2013.
Salazar, L.H.A., Lacerda, T., von Wangenheim, C.G. & Barbalho, R.A., Customisation of Usability Heuristics for Mobile Phones, In Companion Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems, Brazilian Computer Society, pp. 37?38, 2012.
Nielsen. J., Usability Engineering, London, UK: Academic, 1993
Perfetti, C., Usability Testing Best Practices: An Interview with Rolf Molich, http://www.webpronews.com/topnews/2003/07/30/usabilitytesting, 2003.
Borys, M. & Laskowski, M., Expert vs Novice Evaluators-comparison of Heuristic Evaluation Assessment, International Conference on Enterprise Information Systems, 2, pp. 144-149, 2014.
Rossi, L.C. Website Navigation Design: How to Provide Clear Instructions and Directions to Your Readers, https://www.masternewmedia.org/website-navigation-design-how-to-provide-clear-instructions-and-directions-to-your-readers/, 2010.
Petrie, H. & Buykx, L., Collaborative Heuristic Evaluation: Improving the Effectiveness of Heuristic Evaluation, Proceedings of UPA 2010 International Conference, 2010.
Othman, M.K., Mahudin, F., Ahaguk, C.H. & Rahman, M.F.A., Mobile Guide Technologies (Smartphone Apps): Collaborative Heuristic Evaluation (CHE) with Expert and Novice Users, 2014 3rd International Conference on User Science and Engineering (i-USEr), pp. 232-236, 2014.
Petrie, H. & Power, C., What do Users Really Care about? A Comparison of Usability Problems Found by Users and Experts on Highly Interactive Websites, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2107-2116, 2012.
Adelson, B., When Novices Surpass Experts: The Difficulty of a Task May Increase with Expertise, Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(3), pp. 483, 1984.
