Email this article 
Views
The review of usability evaluation methods on telehealth or telemedicine systems
1. PhD. Student in Software Engineering, Department of Computer Engineering, Urmia Branch, Islamic Azad University of Urmia, Urmia, Iran, 2. Assistant Professor, Department of Computer Engineering, Urmia Branch, Islamic Azad University of Urmia, Urmia, Iran
Abstract
Telemedicine in the pandemic of coronavirus disease 2019 (COVID-19) has responded to societal distancing in medical treatments by protecting health workers while also managing available resources. To attain best practices in telemedicine, a platform must be functional, and both patients and clinicians must be satisfied with the technology. To ensure the benefits of telehealth systems, usability refers to how easy the user interfaces of telehealth systems are to use. In this study, the usability of telemedicine systems has been investigated.
The authors of this study review the study from 2015 to 2021 using a combination of the keywords "health", "telemedicine", "telemedicine", "mobile health", "usability" "Software", "System" and "Program", which led to the extraction of 119 articles in this field.
Articles in the field of remote health software and evaluation of the usability of remote health applications in the form of applications based on mobile health technologies, web-based applications or a combination of both types with sample devices Primary are wearable electronics, sensors or robots.
In this study, most of the remote health software are mobile based and their usability has been evaluated by a questionnaire. Satisfaction is the most important usability attributes to consider when designing Health mobile apps.
Received: 2022 January 21; Accepted: 2022 March 15
Keywords: Key Words Healthcare, Telemedicine, Telehealth, Mobile Health, System, Software, Application, Usability.
MATERIAL AND METHODS
In this systematic review, the most essential phrase was "usability of the telehealth or telemedicine system". The scope was limited to English literature. Six digital databases were searched to find the target articles: IEEE Xplore, the Web of Science (WoS), Science Direct, PubMed, Google Scholar, and Elsevier's Scopus. In this study, a review was conducted to search for every article related to healthcare, apps, and usability to 2021. A mix of keywords containing "Healthcare", "Tele Health", "Tele Medicine", " mobile health " combined with the "OR" and "AND" operators followed by "Usability" and "AND "Software" and "OR" "System" and "OR" "Application" were used for search.
The initial query resulted in 350 papers. After reviewing the titles of the articles by two of the authors, 235 articles remained and after deleting the duplicate articles and reviewing the abstracts, 170 articles remained. In the final full-text review, 119 articles were included in the study of usability factors in health and health care, and to better explain the concepts and methods of papers in the final set in the study of usability factors in health and health care. The telemedicine system and how to check the usability of those system was assessed. Each article's authors, title, journal, year of publication, usability check model and abstract were entered into an excel file (Fig 1).
RESULTS
The telemedicine or telehealth systems in which their usability were assessed and included in this review are such as:
smoking cessation mobile apps, tele rehabilitation mobile apps, pain management mobile apps, cardiopulmonary resuscitation mobile apps, diabetes care mobile apps, a self-management of anxiety and depression mobile apps, rheumatoid arthritis mobile apps, oncology cancer care mobile apps, hypertension Management mobile apps, urology care mobile apps, pediatric otolaryngology mobile apps, self-care for pregnant women.
The usability attributes which assessed in each of the articles are shown in Table 1.
| Usability Factors | References |
|---|---|
| Learnability | [15, 19-35] |
| Satisfaction | [19, 20, 25-27, 29-34, 36-96] |
| Efficiency | [19, 23, 25, 26, 28-35, 38, 40, 41, 43-45, 47, 53-56, 58, 59, 63, 65, 66, 69, 74, 75, 78-80, 82, 83, 86-89, 91, 93-95, 97-102] |
| Effectiveness | [13, 19, 20, 22, 23, 25-29, 32-35, 38, 40-44, 47, 53-56, 59, 60, 63, 65-67, 70, 74, 75, 78, 79, 81-83, 85-89, 91-96, 98, 99, 102-111] |
| Memorability | [35] |
| Errors | [21-23, 25, 27, 29, 76, 77, 79, 88, 98, 106, 107] |
| Safety | [20, 27, 56, 92, 106, 111-113] |
| Performance | [19, 20, 23, 25-28, 40, 45, 54, 65, 70-72, 77, 80, 82, 101, 106, 107, 112, 114, 115] |
| Design | [19-23, 25-33, 35, 40, 42-44, 53, 54, 65, 69, 71, 72, 76-79, 81, 82, 85, 88, 91, 92, 95, 97, 98, 101, 103-105, 107, 109, 110, 112, 114-117] |
| Navigation | [13, 19-23, 25, 27-33, 35, 38, 40, 42, 43, 49, 53, 54, 66, 67, 72, 75, 79, 80, 82, 83, 88, 91, 106, 107, 109, 114, 117] |
| Compatibility | [19, 45, 71, 76, 106] |
| Visibility | [29, 45, 97, 106, 107, 114] |
The usability evaluation methods which assessed in each of the articles are shown in Table 2.
DISCUSSION
This study aims to survey usability evaluation methods on tele health or telemedicine systems that run on the Web or Android platform and determine the variety of usability attributes in them. This review focused on the usability evaluation of applications or software designed for telehealth, telemedicine, and health care systems such as telecare monitoring system, mobile applications for Pain Assessment and Management, self-management of hypertension, HIV Interventions, Diabetes, Multiple Sclerosis, Cancer, cardiovascular disease, supporting health care during pregnancy, Psychological Experiences of Web-Based Psychosocial Interventions, interventions for substance use disorder [57, 99].
The results of this study show that mobile phones were the most used because mobile phones can both connect to sensors, run mobile applications, and receive and store data and information and send to o the server located in the health care center for monitoring by health care providers. Mobile apps have a great potential to support patients in healthcare, and to encourage healthy behavioral changes, a set of factors that has impact on the app effectiveness is related to the quality of those features that lead to positive user experiences when using the app. there are so many health mobile apps available, but app usage by patients is low [100], Evaluation of Mobile application usability may be influenced by patient factors such as age, sex, and psychological needs. Mobile Apps can help patients connect with health care providers by supporting email communication and sharing home-monitored data [100]. The need for mobile health approaches to address particular aging features of older adults grows as the population of older adults as a possible consumer group of mobile health grows [10].
The World Health Organization's Global Observatory for eHealth defines mobile health “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices” [118].
| Usability Evaluation Method | Articles | |
|---|---|---|
| Interview | Interview | [13, 14, 16, 20, 25, 26, 36-44, 47, 49, 53, 54, 57-59, 97-99, 103-105, 114, 116] |
| Semi Structured Interviews | [13, 37, 39, 50, 51, 58, 62, 64, 66, 99, 114] | |
| Qualitative Interview | [45, 55, 57, 59, 103, 105] | |
| Post study phone interview | [28, 36] | |
| questionnaire | Questionnaire | [19, 25-32, 45, 46, 48, 54-57, 60, 64-66, 69-71, 77, 78, 80-82, 85, 90-92, 101, 107, 113, 115, 117, 119, 120] |
| Post-Study System Usability Questionnaire (PSSUQ) | [27, 45, 76, 79, 98] | |
| Computer Systems Usability Questionnaire (CSUQ) | [36, 61, 80, 84] | |
| Telehealth Usability Questionnaire | [34, 94, 96, 108] | |
| System Usability Scale (SUS) | [19, 22, 46, 48, 50, 51, 63, 64, 67, 77, 79, 85, 89, 99, 100, 106, 109, 112] | |
| Technology Acceptance Model-2 (TAM-2) | [77] | |
| Client Satisfaction Questionnaire-8, (CSQ-8) | [52] | |
| User Experience Questionnaire (UEQ) | [65] | |
| NASA Task Load Index (NASA TLX) | [84, 106] | |
| Observations | [37, 47, 50, 51, 55, 60, 66, 68, 69, 118] | |
| Descriptively Reported | [55, 56] | |
| Expert Review | [46] | |
| Think a Loud | [45, 47, 67] | |
| Eye Tracking | [21, 67] | |
| Motion tracking device | [21, 67, 69] | |
Smartphones have made it easier for healthcare practitioners and the general population to collaborate, gathers data, and makes clinical decisions [22, 88]. The need for mobile health approaches to address particular aging features of older adult's increases as the population of older adults as a potential user group grows. Usability research, however, indicate that mobile health is still not planned properly for older adults and their preferences [87, 117], Older patients are a significant target population for Web-based health information since numerous ailments, such as cancer, diabetes, and hypertension, are diseases that affect the elderly. In elderly individuals who access Web-based health information, improvements in self-efficacy, blood pressure, hemoglobin levels, and cholesterol levels have been noted [69, 83].
To evaluate the usability of mHealth apps, the most common data collection technique utilized in the studies was questionnaire, followed by field study, interview, observation, think-aloud, and app-use generated data.
The SUS was the most common standard questionnaire used by the studies in the review. The SUS, the Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire, and the Post Study System Usability Questionnaire were the most commonly used scales that studies used to create their own questionnaires.
In this review, we found that questionnaire was the most common data collection technique of the included studies; however, researchers either used standard questionnaires, such as SUS or USE, which were not specifically designed for the mental health domain, or adapted a standard questionnaire or developed a new one. Owing to the great variety of the questionnaires, there is a need to establish a common standardized usability questionnaire targeted specifically at mHealth apps.
Tele Health and Telemedicine are a timely response to the limitations that COVID-19 societal distancing imposes on traditional healthcare delivery methods and the COVID-19 pandemic is favoring digital transitions. Health-care organizations quickly adopted digital solutions and advanced technology tools in response to the pandemic's first phase. Telehealth has emerged as a critical component of patient healthcare delivery during the COVID-19 pandemic.
Telehealth can help with enhancing compliance and clinical efficiency, as well as reducing in-person contacts for clinical routes, which is critical during the COVID-19 epidemic [68, 82]. In the most of the reviewed articles, the factors of the ISO model, 9241-11, or Dr. Nielson's model were used to evaluate the usability of the telehealth systems. The effectiveness, efficiency, satisfaction, learnability, memorability, and errors factor were measured in the majority of the reviewed articles, but given the issue of telehealth and the fact that users vary depending on the type of disease and care or intervention required, age and physical condition must improve the usability of systems for upgrading. To make the system designed for users more usable, it is suggested that the usability of telehealth systems be evaluated using twelve factors [16]. Questionnaires are the most often used approach for evaluating health-care systems. The results showed that Satisfaction, Efficiency, Design and Effectiveness are the most important usability attributes to consider when designing Health mobile apps. the authors had previously designed a telemedicine system for heart control for diabetics [121, 122], but its usability had not been studied, so for future work, they will decide to design a telehealth system and study its usability based on the usability evaluation model with twelve factors included Learnability, Satisfaction, Efficiency, Effectiveness, Memorability, Errors, Safety, Performance, Design, Navigation, Compatibility, and Visibility [121, 122].
CONCLUSION
This study provides usability evaluations methods and usability attributes in telemedicine or telehealth systems especially mhealth apps. the mhealth apps are the most developing due for the availability of mobile devices to users, their popularity, and device affordances. the standard questionnaires are the most used for systems usability evaluation. the satisfaction, efficiency, design and effectiveness are the most important systems usability attributes.
| 1. | Carreiro, S. Newcomb, M. Leach, R. Ostrowski, S. Boudreaux, ED. Amante, D. Current reporting of usability and impact of mHealth interventions for substance use disorder: A systematic review. Drug Alcohol Depend. 2020 215:108201. |
| 2. | Zhou, L. Bao, J. Parmanto, B. Systematic review protocol to assess the effectiveness of usability questionnaires in mhealth app studies. JMIR Res Protoc. 2017 6(8):e151. |
| 3. | Luna-Perejon, F. Malwade, S. Styliadis, C. Civit, J. Cascado-Caballero, D. Konstantinidis, E. Evaluation of user satisfaction and usability of a mobile app for smoking cessation. Comput Methods Programs Biomed. 2019 182:105042. |
| 4. | Tubb, MR. Meulen, M. Pallerla, H. Regan, S. Doarn, CR. Clinical evaluation of e-Quit worRx: A mobile app to enhance smoking cessation shared decision making in primary care. Mhealth. 2019 5 |
| 5. | Gowarty, MA. Longacre, MR. Vilardaga, R. Kung, NJ. Gaughan-Maher, AE. Brunette, MF. Usability and acceptability of two smartphone apps for smoking cessation among young adults with serious mental illness: Mixed methods study. JMIR Ment Health. 2021 8(7):e26873. |
| 6. | Lai, B. Wilroy, J. Young, HJ. Howell, J. Rimmer, JH. Mehta, T. A mobile app to promote adapted exercise and social networking for people with physical disabilities: Usability study. JMIR Form Res. 2019 3(1):e11689. |
| 7. | Wilroy, J. Lai, B. Currie, M. Young, HJ. Thirumalai, M. Mehta, T. Teleassessments for enrollment of adults with physical or mobility disability in a home-based exercise trial in response to COVID-19: Usability study. JMIR Form Res. 2021 5(11):e29799. |
| 8. | Sun, T. Dunsmuir, D. Miao, I. Devoy, GM. West, NC. Görges, M. In‐hospital usability and feasibility evaluation of Panda, an app for the management of pain in children at home. Paediatr Anaesth. 2018 28(10):897–905. |
| 9. | Birnie, KA. Nguyen, C. Amaral, TD. Baker, L. Campbell, F. Lloyd, S. A parent–science partnership to improve postsurgical pain management in young children: Co-development and usability testing of the Achy Penguin smartphone-based app. Can J Pain. 2018 2(1):280–91. |
| 10. | Ehrler, F. Weinhold, T. Joe, J. Lovis, C. Blondon, K. A mobile app (BEDSide mobility) to support nurses' tasks at the patient's bedside: Usability study. JMIR Mhealth Uhealth. 2018 6(3):e57. |
| 11. | Tonga, E. Williamson, E. Srikesavan, C. Özen, T. Sarıtaş, F. Lamb, SE. A hand exercise mobile app for people with rheumatoid arthritis in Turkey: Design, development and usability study. Rheumatol Int. 2021 41(6):1151–60. |
| 12. | Constantinescu, G. Kuffel, K. King, B. Hodgetts, W. Rieger, J. Usability testing of an mHealth device for swallowing therapy in head and neck cancer survivors. Health Informatics J. 2019 25(4):1373–82. |
| 13. | International, OfS. Ergonomics of human-system interaction - part 11: Usability: Definitions and concepts [Internet]. |
| 14. | Salari, R. Kalhori, S. Ghazisaeedi, M. Jeddi, M. Nazari, M. Fatehi, F. Mobile-based and cloud-based system for self-management of people with type 2 diabetes: Development and usability evaluation. J Med Internet Res. 2021 23(6) |
| 15. | Bevan, N. Measuring usability as quality of use. Software Quality Journal. 1995 4(2):115–30. |
| 16. | Tark, R. Metelitsa, M. Akkermann, K. Saks, K. Mikkel, S. Haljas, K. Usability, acceptability, feasibility, and effectiveness of a gamified mobile health intervention (Triumf) for pediatric patients: Qualitative study. JMIR Serious Games. 2019 7(3):e13776. |
| 17. | van, BM. Velu, A. van, dBL. Kok, M. Mol, BW. Frings-Dresen, M. Usability and usefulness of a mobile health app for pregnancy-related work advice: Mixed-methods approach. JMIR Mhealth Uhealth. 2019 7(5):e11442. |
| 18. | Welbie, M. Wittink, H. Westerman, MJ. Topper, I. Snoei, J. Devillé, W. A mobile patient-reported outcome measure app with talking touchscreen: Usability assessment. JMIR Form Res. 2019 3(3):e11617. |
| 19. | The, HR. Services, A. Telehealth programs [Internet]. :2020 [cited: 01 Dec 2021]. Available from: https://www.hrsa.gov/rural. |
| 20. | Shah, UM. Chiew, TK. A systematic literature review of the design approach and usability evaluation of the pain management mobile applications. Symmetry. 2019 11(3) |
| 21. | Agha, Z. Weir, CR. Chen, Y. Usability of telehealth technologies. International Journal of Telemedicine and Applications. 2013 |
| 22. | Almeida, AF. Rocha, NP. Silva, AG. Methodological quality of manuscripts reporting on the usability of mobile applications for pain assessment and management: A systematic review. Int J Environ Res Public Health. 2020 17(3):785. |
| 23. | Wildenbos, GA. Peute, L. Jaspers, M. Aging barriers influencing mobile health usability for older adults: A literature based framework (MOLD-US). Int J Med Inform. 2018 :114: 66–75. |
| 24. | International, OfS. ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs) [Internet]. |
| 25. | Gowarty, MA. Aschbrenner, KA. Brunette, MF. Acceptability and usability of mobile apps for smoking cessation among young adults with psychotic disorders and other serious mental illness. Front Psychiatry. 2021 :12:656538. |
| 26. | Frías, A. Solves, L. Navarro, S. Palma, C. Farriols, N. Aliaga, F. Technology-based psychosocial interventions for people with borderline personality disorder: A scoping review of the literature. Psychopathology. 2020 53(5-6):254–63. |
| 27. | Thirumalai, M. Rimmer, JH. Johnson, G. Wilroy, J. Young, HJ. Mehta, T. TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mHealth app: Participant-centered development and usability study. JMIR Mhealth Uhealth. 2018 6(5):e10181. |
| 28. | Dunsmuir, D. Wu, H. Sun, T. West, NC. Lauder, GR. Görges, M. A postoperative pain management mobile app (panda) for children at home after discharge: Usability and feasibility. JMIR Perioper Med. 2019 2(2) |
| 30. | McCall, T. Ali, MO. Yu, F. Fontelo, P. Khairat, S. Development of a mobile app to support self-management of anxiety and depression in African American women: A usability study. JMIR Form Res. 2021 5(8):e24393. |
| 31. | Bruggers, CS. Baranowski, S. Beseris, M. Leonard, R. Long, D. Schulte, E. A prototype exercise–empowerment mobile video game for children with cancer, and its usability assessment: Developing digital empowerment interventions for pediatric diseases. Front Pediatr. 2018 |
| 32. | Ingadottir, B. Blondal, K. Thue, D. Zoega, S. Thylen, I. Jaarsma, T. Development, usability, and efficacy of a serious game to help patients learn about pain management after surgery: An evaluation study. JMIR Serious Games. 2017 5(2):e10. |
| 33. | Radhakrishnan, K. Toprac, P. O'Hair, M. Bias, R. Kim, MT. Bradley, P. Interactive digital e-Health game for heart failure self-management: A feasibility study. Games Health J. 2016 5(6):366–74. |
| 34. | Vanosdoll, M. Ng, N. Ho, A. Wallingford, A. Xu, S. Matin, SB. A novel mobile health tool for home-based identification of neonatal illness in Uganda: Formative usability study. JMIR Mhealth Uhealth. 2019 7(8):e14540. |
| 35. | Stuti, D. Karanam, C. Gómez-Orozco, C. Gómez-Marín, O. Mobile phone intervention for heart failure in a minority urban county hospital population: Usability and patient perspectives. Telemed J E Health. 2017 23(7):544–54. |
| 36. | Pimmer, C. Tulenko, K. The convergence of mobile and social media: Affordances and constraints of mobile networked communication for health workers in low-and middle-income countries. Mobile Media & Communication. 2016 4(2):252–69. |
| 37. | Nguyen, M. Waller, M. Pandya, A. Portnoy, J. A review of patient and provider satisfaction with telemedicine. Curr Allergy Asthma Rep. 2020 20(11):72 . |
| 38. | Wang, Y. Zeng, L. Yao, S. Zhu, F. Liu, C. Laura, AD. Recommendations of protective measures for orthopedic surgeons during COVID-19 pandemic. Knee Surg Sports Traumatol Arthrosc. 2020 28(7):2027–35. |
| 39. | Sangar, AB. Rastari, S. A model for increasing usability of mobile banking apps on smart phones. Indian Journal of Science and Technology. 2015 8(30):1–9. |
| 40. | Frías, A. Palma, C. Salvador, A. Aluco, E. Navarro, S. Farriols, N. B·RIGHT: Usability and satisfaction with a mobile app for self-managing emotional crises in patients with borderline personality disorder. Australas Psychiatry. 2021 29(3):294–8. |
| 41. | Metelmann, B. Metelmann, C. Schuffert, L. Hahnenkamp, K. Brinkrolf, P. Medical correctness and user friendliness of available apps for cardiopulmonary resuscitation: Systematic search combined with guideline adherence and usability evaluation. JMIR Mhealth Uhealth. 2018 6(11):e190. |
| 42. | Fu, HN. Adam, TJ. Konstan, JA. Wolfson, JA. Clancy, TR. Wyman, JF. Influence of patient characteristics and psychological needs on diabetes mobile app usability in adults with type 1 or type 2 diabetes: Crossover randomized trial. JMIR Diabetes. 2019 4(2):e11462. |
| 43. | Harvey, C. Koubek, R. Bégat, V. Jacob, S. Usability evaluation of a blood glucose monitoring system with a spill-resistant vial, easier strip handling, and connectivity to a mobile app: Improvement of patient convenience and satisfaction. J Diabetes Sci Technol. 2016 10(5):1136–41. |
| 44. | Diamantidis, CJ. Ginsberg, JS. Yoffe, M. Lucas, L. Prakash, D. Aggarwal, S. Remote usability testing and satisfaction with a mobile health medication inquiry system in CKD. Clin J Am Soc Nephrol. 2015 10(8):1364–70. |
| 45. | Gladman, T. Tylee, G. Gallagher, S. Mair, J. Rennie, SC. Grainger, R. A tool for rating the value of health education mobile apps to enhance student learning (MARuL): Development and usability study. JMIR Mhealth Uhealth. 2020 8(7):e18015. |
| 46. | Liew, MS. Zhang, J. See, J. Ong, YL. Usability challenges for health and wellness mobile apps: mixed-methods study among mHealth experts and consumers. JMIR Mhealth Uhealth. 2019 7(1):e12160. |
| 47. | Nitsch, M. Dimopoulos, CN. Flaschberger, E. Saffran, K. Kruger, JF. Garlock, L. A guided online and mobile self-help program for individuals with eating disorders: An iterative engagement and usability study. J Med Internet Res. 2016 18(1):e7. |
| 48. | Rodríguez, S. Sanz, AM. Llano, G. Navarro, A. Parra-Lara, LG. Krystosik, AR. Acceptability and usability of a mobile application for management and surveillance of vector-borne diseases in Colombia: An implementation study. PLoS One. 2020 15(5):e0233269. |
| 49. | Teo, CH. Ng, CJ. Lo, SK. Lim, CD. White, A. A mobile web app to improve health screening uptake in men (ScreenMen): Utility and usability evaluation study. JMIR Mhealth Uhealth. 2019 7(4):e10216. |
| 50. | Thies, K. Anderson, D. Cramer, B. Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: Interview analysis of staff and patients in a failed randomized trial. JMIR Hum Factors. 2017 4(4):e24. |
| 51. | Chantler, T. Paton, C. Velardo, C. Triantafyllidis, A. Shah, SA. Stoppani, E. Creating connections- the development of a mobile-health monitoring system for heart failure: Qualitative findings from a usability cohort study. Digit Health. 2016 :2: 2055207616671461. |
| 52. | Ding, H. Fatehi, F. Russell, AW. Karunanithi, M. Menon, A. Bird, D. User experience of an innovative mobile health program to assist in insulin dose adjustment: Outcomes of a proof-of-concept trial. Telemed J E Health. 2018 24(7):536–43. |
| 53. | Tay, I. Garland, S. Gorelik, A. Wark, JD. Development and testing of a mobile phone app for self-monitoring of calcium intake in young women. JMIR Mhealth Uhealth. 2017 5(3):e27. |
| 54. | Chapman, SSN. Brown, PC. Waits, KH. Wong, JS. Bhatti, MS. Toqeer, Q. Development and evaluation of a user-centered mobile telestroke platform. Telemed J E Health. 2019 25(7):638–48. |
| 55. | Pinem, AA. Yeskafauzan, A. Handayani, PW. Azzahro, F. Hidayanto, AN. Ayuningtyas, D. Designing a health referral mobile application for high-mobility end users in Indonesia. Heliyon. 2020 6(1):e03174. |
| 56. | Stütz, T. Emsenhuber, G. Huber, D. Domhardt, M. Tiefengrabner, M. Janneke, OG. Mobile phone-supported physiotherapy for frozen shoulder: Feasibility assessment based on a usability study. JMIR Rehabil Assist Technol. 2017 4(2) |
| 57. | Liu, YC. Chen, CH. Lin, YS. Chen, HY. Irianti, D. Jen, TN. Design and usability evaluation of mobile voice-added food reporting for elderly people: Randomized controlled trial. JMIR Mhealth Uhealth. 2020 8(9):e20317. |
| 58. | Brinkel, J. May, J. Krumkamp, R. Lamshöft, M. Kreuels, B. Owusu-Dabo, E. Mobile phone-based interactive voice response as a tool for improving access to healthcare in remote areas in Ghana: An evaluation of user experiences. Trop Med Int Health. 2017 22(5):622–30. |
| 59. | Moeini, S. Watzlaf, V. Zhou, L. Abernathy, RP. Development of a weighted well-being assessment mobile app for trauma affected communities: A usability study. Perspect Health Inf Manag. 2020 18(Winter):1o. |
| 60. | Alanzi, T. Istepanian, R. Philip, N. Design and usability evaluation of social mobile diabetes management system in the gulf region. JMIR Res Protoc. 2016 5(3):e93. |
| 61. | Kristjansdottir, OB. Borosund, E. Westeng, M. Ruland, C. Stenberg, U. Zangi, HA. Mobile app to help people with chronic illness reflect on their strengths: Formative evaluation and usability testing. JMIR Form Res. 2020 4(3):e16831. |
| 62. | Heiney, SP. Donevant, SB. Adams, SA. Parker, PD. Chen, H. Levkoff, S. A smartphone app for self-management of heart failure in older African Americans: Feasibility and usability study. JMIR Aging. 2020 3(1):e17142. |
| 63. | Marques, A. Moreira, T. Jorge, TV. Rabelo, S. Lima, dR. Felipe, GF. Usability of a mobile application on diabetic foot self-care. Rev Bras Enferm. 2020 73(4):e20180862 . |
| 64. | Rasche, P. Mertens, A. Miron-Shatz, T. Berzon, C. Schlick, CM. Jahn, M. Seamless recording of glucometer measurements among older experienced diabetic patients–A study of perception and usability. PLoS One. 2018 13(5):e0197455. |
| 65. | Castellano-Tejedor, C. Moreno, J. Knittle, K. Nurmi, J. Ginchev, T. Parramón, G. Assessing the user experience and usability of the PRECIOUS system: a randomized controlled trial in obese patients. Inform Health Soc Care. 2020 45(4):410–27. |
| 66. | Marzuki, M. Yaacob, NA. Yaacob, NM. Hassan, M. Ahmad, SB. Usable mobile app for community education on colorectal cancer: development process and usability study. JMIR Hum Factors. 2019 6(2):e12103. |
| 67. | Pereira-Azevedo, N. Osório, L. Fraga, A. Roobol, MJ. Rotterdam prostate cancer risk calculator: Development and usability testing of the mobile phone app. JMIR Cancer. 2017 3(1) |
| 68. | Heynsbergh, N. Heckel, L. Botti, M. Livingston, PM. A smartphone app to support carers of people living with cancer: A feasibility and usability study. JMIR Cancer. 2019 5(1):e11779. |
| 69. | Fu, MR. Axelrod, D. Guth, AA. Wang, Y. Scagliola, J. Hiotis, K. Usability and feasibility of health IT interventions to enhance self-care for lymphedema symptom management in breast cancer survivors. Internet Interv. 2016 :5: 56–64. |
| 70. | Hill, JR. Harrington, AB. Adeoye, P. Campbell, NL. Holden, RJ. Going remote-Demonstration and evaluation of remote technology delivery and usability assessment with older adults: Survey study. JMIR Mhealth Uhealth. 2021 9(3):e26702. |
| 71. | Bente, BE. Roderick, v'tKJ. Schreijer, MA. Berkemeier, L. van, GJE. Hendrik, SPJ. The Dutch COVID-19 contact tracing app (the CoronaMelder): Usability study. JMIR Form Res. 2021 5(3):e27882. |
| 72. | English, LL. Dunsmuir, D. Kumbakumba, E. Ansermino, JM. Larson, CP. Lester, R. The pediatric risk assessment (PARA) mobile app to reduce postdischarge child mortality: Design, usability, and feasibility for health care workers in Uganda. JMIR Mhealth Uhealth. 2016 4(1):e16. |
| 73. | Bolle, S. Romijn, G. Smets, EM. Loos, EF. Kunneman, M. van, WJC. Older cancer patients' user experiences with web-based health information tools: A think-aloud study. J Med Internet Res. 2016 18(7):e208. |
| 74. | Barbabella, F. Poli, A. Hanson, E. Andréasson, F. Salzmann, B. Döhner, H. Usage and usability of a web-based program for family caregivers of older people in three European countries: A mixed-methods evaluation. Comput Inform Nurs. 2018 36(5):232–41. |
| 75. | Agnisarman, SO. Chalil, MK. Smith, K. Ashok, A. Welch, B. McElligott, JT. Lessons learned from the usability assessment of home-based telemedicine systems. Appl Ergon. 2017 :58: 424–34. |
| 76. | Anzinger, H. Elliott, SA. Hartling, L. Comparative usability analysis and parental preferences of three web-based knowledge translation tools: Multimethod study. J Med Internet Res. 2020 22(3):e14562. |
| 77. | Lauritsen, L. Andersen, L. Olsson, E. Søndergaard, SR. Nørregaard, LB. Løventoft, PK. Usability, acceptability, and adherence to an electronic self-monitoring system in patients with major depression discharged from inpatient wards. J Med Internet Res. 2017 19(4):e123. |
| 78. | Kapoor, A. Nambisan, P. Usability and acceptance evaluation of ACESO: A web-based breast cancer survivorship tool. J Cancer Surviv. 2018 12(3):316–25. |
| 79. | Wang, T. Dolezel, D. Usability of web-based personal health records: An analysis of consumers' perspectives. Perspect Health Inf Manag. 2016 13(Spring) |
| 80. | Takano, A. Miyamoto, Y. Kawakami, N. Matsumoto, T. Web-based cognitive behavioral relapse prevention program with tailored feedback for people with methamphetamine and other drug use problems: Development and usability study. JMIR Ment Health. 2016 3(1):e1. |
| 81. | Neville, C. Da, CD. Rochon, M. Peschken, CA. Pineau, CA. Bernatsky, S. Development of the lupus interactive navigator as an empowering web-based eHealth tool to facilitate lupus management: Users perspectives on usability and acceptability. JMIR Res Protoc. 2016 5(2):e44. |
| 82. | Bennion, MR. Hardy, GE. Moore, RK. Kellett, S. Millings, A. Usability, acceptability, and effectiveness of web-based conversational agents to facilitate problem solving in older adults: Controlled study. J Med Internet Res. 2020 22(5):e16794. |
| 83. | Tao, D. Shao, F. Wang, H. Yan, M. Qu, X. Integrating usability and social cognitive theories with the technology acceptance model to understand young users' acceptance of a health information portal. Health Informatics J. 2020 26(2):1347–62. |
| 84. | García-Casal, JA. Martínez-Abad, F. Cid-Bartolomé, T. Smith, SJ. Llano-Ordóñez, K. Perea-Bartolomé, MV. Usability study and pilot validation of a computer-based emotion recognition test for older adults with Alzheimer's disease and amnestic mild cognitive impairment. Aging Ment Health. 2019 23(3):365–75. |
| 85. | Gerłowska, J. Skrobas, U. Grabowska-Aleksandrowicz, K. Korchut, A. Szklener, S. Szczęśniak-Stańczyk, D. Assessment of perceived attractiveness, usability, and societal impact of a multimodal robotic assistant for aging patients with memory impairments. Front Neurol. 2018 :9: 392. |
| 86. | Marien, S. Legrand, D. Ramdoyal, R. Nsenga, J. Ospina, G. Ramon, V. A web application to involve patients in the medication reconciliation process: A user-centered usability and usefulness study. J Am Med Inform Assoc. 2018 25(11):1488–500. |
| 87. | Klaassen, B. van, BB. Held, JP. Reenalda, J. van, MFB. Veltink, PH. Usability evaluations of a wearable inertial sensing system and quality of movement metrics for stroke survivors by care professionals. Front Bioeng Biotechnol. 2017 :5: 20. |
| 88. | Jia, Y. Wang, W. Wen, D. Liang, L. Gao, L. Lei, J. Perceived user preferences and usability evaluation of mainstream wearable devices for health monitoring. PeerJ. 2018 :6: e5350. |
| 89. | Lee, H. Kim, J. Kim, S. Kong, HJ. Joo, H. Lee, D. Usability evaluation of user requirement-based teleconsultation robots: A preliminary report from south Korea. Methods Inf Med. 2020 59(2-03):86–95. |
| 90. | Tonino, R. Larimer, K. Eissen, O. Schipperus, MR. Remote patient monitoring in adults receiving transfusion or infusion for hematological disorders using the VitalPatch and accelerateIQ monitoring system: Quantitative feasibility study. JMIR Hum Factors. 2019 6(4):e15103. |
| 91. | Poncette, AS. Mosch, L. Spies, C. Schmieding, M. Schiefenhövel, F. Krampe, H. Improvements in patient monitoring in the intensive care unit: Survey study. J Med Internet Res. 2020 22(6):e19091. |
| 92. | Bonsignore, L. Bloom, N. Steinhauser, K. Nichols, R. Allen, T. Twaddle, M. Evaluating the feasibility and acceptability of a telehealth program in a rural palliative care population: TapCloud for palliative care. J Pain Symptom Manage. 2018 56(1):7–14. |
| 93. | Lefler, LL. Rhoads, SJ. Harris, M. Funderburg, AE. Lubin, SA. Martel, ID. Evaluating the use of mobile health technology in older adults with heart failure: Mixed-methods study. JMIR Aging. 2018 1(2):e12178. |
| 94. | Ancona, GD. Murero, M. Feickert, S. Kaplan, H. Öner, A. Ortak, J. Implantation of an innovative intracardiac microcomputer system for web-based real-time monitoring of heart failure: Usability and patients' attitudes. JMIR Cardio. 2021 5(1):e21055. |
| 95. | Wallin, E. Norlund, F. Olsson, E. Burell, G. Held, C. Carlsson, T. Treatment activity, user satisfaction, and experienced usability of internet-based cognitive behavioral therapy for adults with depression and anxiety after a myocardial infarction: Mixed-methods study. J Med Internet Res. 2018 20(3):e87. |
| 96. | Kim, Y. Seo, J. An, SY. Sinn, DH. Hwang, JH. Efficacy and safety of an mhealth app and wearable device in physical performance for patients with hepatocellular carcinoma: Development and usability study. JMIR Mhealth Uhealth. 2020 8(3):e14435. |
| 97. | Silveira, DV. Marcolino, MS. Machado, EL. Ferreira, CG. Moreira, AMB. Resende, ES. Development and evaluation of a mobile decision support system for hypertension management in the primary care setting in Brazil: Mixed-methods field study on usability, feasibility, and utility. JMIR Mhealth Uhealth. 2019 7(3):e9869. |
| 98. | Cho, S. Lee, JH. Kim, IK. Kim, MG. Sik, KY. Lee, E. The educational and supportive mobile application for caregivers of dementia people. Stud Health Technol Inform. 2016 :225: 1045–6. |
| 99. | Moradian, S. Krzyzanowska, MK. Maguire, R. Morita, PP. Kukreti, V. Avery, J. Usability evaluation of a mobile phone-based system for remote monitoring and management of chemotherapy-related side effects in cancer patients: Mixed-methods study. JMIR Cancer. 2018 4(2):e10932. |
| 100. | Janatkhah, R. Tabari-Khomeiran, R. Asadi-Louyeh, A. Kazemnejad, E. Usability of a disease management mobile application as perceived by patients with diabetes. Comput Inform Nurs. 2019 37(8):413–9. |
| 101. | Fucà, E. Costanzo, F. Bonutto, D. Moretti, A. Fini, A. Ferraiuolo, A. Mobile-health technologies for a child neuropsychiatry service: Development and usability of the assioma digital platform. Int J Environ Res Public Health. 2021 18(5) |
| 102. | Sahu, D. Rathod, V. Phadnis, A. Bansal, SS. Telehealth for consultation and shoulder rehabilitation: A preliminary study on the perspectives of 30 patients during the COVID-19 lockdown. Clin Shoulder Elb. 2021 24(3):156–65. |
| 103. | Alessa, T. Abdi, T. Hawley, MS. de, WL. Mobile apps to support the self-management of hypertension: Systematic review of effectiveness, usability, and user satisfaction. JMIR Mhealth Uhealth. 2018 6(7):e10723. |
| 104. | Young, HJ. Mehta, T. Lai, B. TEAMS (Tele-Exercise and Multiple Sclerosis), a tailored telerehabilitation mHealth app: Participant-centered development and usability study. JMIR Mhealth Uhealth. 2018 6(5):e10181. |
| 105. | Seo, NJ. Arun, KJ. Hur, P. Crocher, V. Motawar, B. Lakshminarayanan, K. Usability evaluation of low-cost virtual reality hand and arm rehabilitation games. J Rehabil Res Dev. 2016 53(3):321–34. |
| 106. | Himes, KP. Donovan, H. Wang, S. Weaver, C. Grove, JR. Facco, FL. Healthy beyond pregnancy, a web-based intervention to improve adherence to postpartum care: Randomized controlled feasibility trial. JMIR Hum Factors. 2017 4(4):e26. |
| 107. | Martinez, W. Threatt, AL. Rosenbloom, ST. Wallston, KL. Hickson, GB. Elasy, TA. A patient-facing diabetes dashboard embedded in a patient web portal: Design sprint and usability testing. JMIR Hum Factors. 2018 5(3) |
| 108. | Layfield, E. Triantafillou, V. Prasad, A. Deng, J. Shanti, RM. Newman, JG. Telemedicine for head and neck ambulatory visits during COVID-19: Evaluating usability and patient satisfaction. Head Neck. 2020 42(7):1681–9. |
| 109. | Dubin, JM. Wyant, WA. Balaji, NC. Ong, WL. Kettache, RH. Haffaf, M. Telemedicine usage among urologists during the COVID-19 pandemic: Cross-sectional study. J Med Internet Res. 2020 22(11):e21875. |
| 110. | Waqar-Cowles, LN. Chuo, J. Weiss, PF. Gmuca, S. LaNoue, M. Burnham, JM. Evaluation of pediatric rheumatology telehealth satisfaction during the COVID-19 pandemic. Pediatr Rheumatol Online J. 2021 19(1):170. |
| 111. | Moulaei, K. Sheikhtaheri, A. Ghafaripour, Z. Bahaadinbeigy, K. The development and usability assessment of an mHealth application to encourage self-care in pregnant women against COVID-19. J Healthc Eng. 2021 |
| 112. | Khodambashi, S. Gilstad, H. Nytro, O. Usability evaluation of clinical guidelines on the web using eye-tracker. Stud Health Technol Inform. 2016 :228: 95–9. |
| 113. | Sengupta, A. Beckie, T. Dutta, K. Dey, A. Chellappan, S. A mobile health intervention system for women with coronary heart disease: Usability study. JMIR Form Res. 2020 4(6) |
| 114. | Kessel, KA. Me, VM. Kessel, C. Bier, H. Biedermann, T. Friess, H. Mobile health in oncology: A patient survey about app-assisted cancer care. JMIR Mhealth Uhealth. 2017 5(6) |
| 115. | McClellan, MA. Karumur, RP. Vogel, RI. Petzel, SV. Cragg, J. Chan, D. Designing an educational website to improve quality of supportive oncology care for women with ovarian cancer: An expert usability review and analysis. Int J Hum Comput Interact. 2016 32(4):297–307. |
| 116. | Tu, MH. Chang, P. Lee, YL. Avoiding obsolescence in mobile health: Experiences in designing a mobile support system for complicated documentation at long-term care facilities. Comput Inform Nurs. 2018 36(10):501–6. |
| 117. | Talboom-Kamp, E. Tossaint-Schoenmakers, R. Goedhart, A. Versluis, A. Kasteleyn, M. Patients' attitudes toward an online patient portal for communicating laboratory test results: Real-world study using the eHealth impact questionnaire. JMIR Form Res. 2020 4(3):e17060. |
| 118. | World, HO. mHealth: New horizons for health through mobile technologies [Internet]. |
| 119. | Schnall, R. Cho, H. Liu, J. Health information technology usability evaluation scale (Health-ITUES) for usability assessment of mobile health technology: Validation study. JMIR Mhealth Uhealth. 2018 6(1) |
| 120. | Jeffrey, B. Bagala, M. Creighton, A. Leavey, T. Nicholls, S. Wood, C. Mobile phone applications and their use in the self-management of type 2 diabetes mellitus: A qualitative study among app users and non-app users. Diabetol Metab Syndr. 2019 :11: 84. PMID: 31636719 DOI: 10.1186/s13098. |
| 121. | Soltanzadeh, L. Taheri, A. Rabiee, M. Implementing a prototype diabetic telemedicine system. International Journal of Computer Science Issues. 2014 11(4):86–91. |
| 122. | Soltanzadeh, L. Taheri, A. Rabiee, M. A portable ECG device for diabetic patients's telemedicine. Frontiers in Health Informatics. 2014 3(1):9–11. |
Refbacks
- There are currently no refbacks.
Frontiers in Health Informatics (Official Journal of Iranian Association of Medical Informatics) ISSN-Online: 2676-7104
No. 4, Rashid Yasami Av., Vali-e-Asr St., Tehran-Iran; SMS: (+98) 9383621232 email: info@ijmi.ir
This work is licensed under a Creative Commons Attribution 4.0 International License which allows users to read, copy, distribute and make derivative works from the material, as long as the author of the original work is cited properly.
The theme is developed by: Gostaresh Afzar Hamara