User Testing of an Admission, Discharge, Transfer System: Usability Evaluation

Majid Jangi, Reza Khajouei, Mahmoud Tara, Mohammad Reza Mazaheri Habibi, Azade Kamel Ghalibaf, Sara Zangouei, Mostafa Mostafavi



Introduction: To improve the first step of the hospitalization procedure, appropriate interaction must be established between users and the admission, discharge, and transfer system. The aim of this study was to evaluate the usability of the ADT system in some of selected Iranian non-teaching hospitals.

Material and Methods: This study was cross-sectional research that has evaluated the usability of a selected ADT system using the think-aloud method by 11 medical record administrators. Users were asked to follow the provided scenario, then share and elaborate on what they saw, thought about, did, felt, and decided during their interaction with the system. Users' feedbacks were collected and organized into four main categories for further processing.

Results: To evaluate the usability of an ADT system, four routine scenario tasks were followed by users and only 45.45% of them could implement all tasks. Overall, 36 independent problems were identified. All problems were related to the data entry categories that accounted for the largest share. The most important problems were related to the issues regarding "date of birth" field in this category which deals with the outpatient admission process.

Conclusion: The study of the usability testing method indicated that the ADT subsystem of non-teaching hospital has many problems in interact with real users with the system. It showed that more than half of the users could not completely and successfully perform the entire real-world scenario tasks. Furthermore, the most usability problems were found in data entry categories.


Sultan F, Aziz MT, Khokhar I, Qadri H, Abbas M, Mukhtar A, et al. Development of an in-house hospital information system in a hospital in Pakistan. Int J Med Inform. 2014; 83(3): 180-8. PMID: 24373713 DOI: 10.1016/j.ijmedinf.2013.12.004

Nielsen J. Usability 101: Introduction to usability [Internet]. 2003 [cited: 1 Dec 2020]. Available from:

Walji MF, Kalenderian E, Tran D, Kookal KK, Nguyen V, Tokede O, et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform. 2013; 82(2): 128-38. PMID: 22749840 DOI: 10.1016/j.ijmedinf.2012.05.018

Karat CM. Cost-benefit analysis of usability engineering techniques. Proceedings of the Human Factors Society Annual Meeting. 1990; 34(12): 839-43.

Khajouei R, deJongh D, Jaspers MW. Usability evaluation of a computerized physician order entry for medication ordering. Stud Health Technol Inform. 2009; 150: 532-6. PMID: 19745368

Anderson J, Wagner J, Bessesen M, Williams LC. Usability testing in the hospital. Human Factors and Ergonomics in Manufacturing & Service Industries. 2012; 22(1): 52-63.

International Standards Organisation. ISO 13407: Human centered design processes for interactive systems. International Standards Organisation, Genève; 1999.

Bastien JC. Usability testing: A review of some methodological and technical aspects of the method. Int J Med Inform. 2010; 79(4): e18-23. PMID: 19345139 DOI: 10.1016/j.ijmedinf.2008.12.004

Nielsen J. Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies. 1994; 41(3): 385-97.

Hartson HR, Andre TS, Williges RC. Criteria for evaluating usability evaluation methods. International Journal of Human-Computer Interaction. 2003; 15(1): 145-81.

Agharezaei Z, Khajouei R, Ahmadian L, Agharezaei L. Usability evaluation of a laboratory information system. Health Information Management. 2012; 10(2): 1-12.

Khajouei R, Azizi A, Atashi A. Usability evaluation of an emergency information system: A heuristic evaluation. Journal of Health Administration. 2013; 16(52): 61-72.

Dianat I, Ghanbari Z, AsghariJafarabadi M. Psychometric properties of the Persian language version of the system usability scale. Health Promot Perspect. 2014; 4(1): 82–9. PMID: 25097841 DOI: 10.5681/hpp.2014.011

Moattari M, Moosavinasab E, Dabbaghmanesh MH, ZarifSanaiey N. Validating a web-based diabetes education program in continuing nursing education: Knowledge and competency change and user perceptions on usability and quality. J Diabetes Metab Disord. 2014; 13: 70. PMID: 26086025 DOI: 10.1186/2251-6581-13-70

Nabovati E, Vakili-Arki H, Eslami S, Khajouei R. Usability evaluation of laboratory and radiology information systems integrated into a hospital information system. J Med Syst. 2014; 38(4): 35. PMID: 24682671 DOI: 10.1007/s10916-014-0035-z

Shah MH, Peikari HR. Electronic prescribing usability: Reduction of mental workload and prescribing errors among community physicians. Telemed J E Health. 2016; 22(1): 36-44. PMID: 26244488 DOI: 10.1089/tmj.2014.0246

Jaspers MW. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inform. 2009; 78(5): 340-53. PMID: 19046928 DOI: 10.1016/j.ijmedinf.2008.10.002

Van Someren MW, Barnard YF, Sandberg JA. The think aloud method: A practical guide to modelling cognitive processes. Academic Press London; 1994.

Van den Haak MJ, de Jong MD, Schellens PJ. Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: A methodological comparison. Interacting with Computers. 2004; 16(6): 1153-70.

Van Engen-Verheul M, Peute L, Kilsdonk E, Peek N, Jaspers M. Usability evaluation of a guideline implementation system for cardiac rehabilitation: Think aloud study. Stud Health Technol Inform. 2012; 180: 403-7. PMID: 22874221

Ahmadian L, Nejad SS, Khajouei R. Evaluation methods used on health information systems (HISs) in Iran and the effects of HISs on Iranian healthcare: A systematic review. Int J Med Inform. 2015; 84(6): 444-53. PMID: 25746766 DOI: 10.1016/j.ijmedinf.2015.02.002

Marty P, Twidale M. Usability at 90mph: Presenting and evaluating a new, high–speed method for demonstrating user testing in front of an audience. First Monday. 2005; 10(7): 1.

Te'eni D, Carey JM, Zhang P. Human-computer interaction: Developing effective organizational information systems. John Wiley & Sons; 2005.

Khajouei R, Hasman A, Jaspers MW. Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system. Int J Med Inform. 2011; 80(5): 341-50. PMID: 21435943 DOI: 10.1016/j.ijmedinf.2011.02.005

Peute L, Jaspers M. Usability evaluation of a laboratory order entry system: Cognitive walkthrough and think aloud combined. Stud Health Technol Inform. 2005; 116: 599-604. PMID: 16160323

Kushniruk AW, Patel VL, Cimino JJ. Usability testing in medical informatics: Cognitive approaches to evaluation of information systems and user interfaces. Proc AMIA Annu Fall Symp. 1997; 218-22. PMID: 9357620

Khajouei R, Peute LW, Hasman A, Jaspers MW. Classification and prioritization of usability problems using an augmented classification scheme. J Biomed Inform. 2011; 44(6): 948-57. PMID: 21782036 DOI: 10.1016/j.jbi.2011.07.002

Kallio T, Kaikkonen A. Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability Studies. 2005; 1(4-16): 23-28.



  • There are currently no refbacks.