Measuring User Satisfaction in Website Usability by Considering Stress Level

Surya Raka - Informatics Department, Universitas Atma Jaya,Yogyakarta, 55281,Yogyakarta, Indonesia
Djoko Setyohadi - Informatics Department, Universitas Atma Jaya,Yogyakarta, 55281,Yogyakarta, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.5.3.512

Abstract


Usability testing, part of the human-computer interaction problem, is a method commonly used to evaluate the performance of a website interface. Today's study of human-computer interaction is important. It is now developed into UI UX (User Interface / User Experience). The website interface can affect user satisfaction and is a principal factor in a website being liked by users and meeting user needs. When a website is first developed, it can be difficult for a developer to understand the need for a website interface that matches the user persona of that website. Therefore, this research is important as an evaluation material for websites that are developed in the context of a website interface to provide a satisfactory appearance according to user personas. This study evaluates a website by conducting a usability test using the ISO-9241 aspect and getting an average result of 50,967, which is categorized as a bad website is taken from all users in the experiment. This study can also prove the psychological factors of users that influence usability testing, namely differences in the perceptions of website ratings between users with normal and (mild, moderate, severe, too severe) psychological stress levels. Users with normal stress levels will have better usability test results than users with stress levels. This study shows better results than user ratings with a normal stress level of 73,750, so it is categorized as a reliable website.


Keywords


Usability testing; factor of usability; stress level of user; UI UX.

Full Text:

PDF

References


D. Reuschke and C. Mason, “The engagement of home-based businesses in the digital economy,†Futures, no. October 2019, p. 102542, 2020, doi: 10.1016/j.futures.2020.102542.

S. K. Boell and D. Cecez-Kecmanovic, “What is an information system?,†Proc. Annu. Hawaii Int. Conf. Syst. Sci., vol. 2015-March, no. January, pp. 4959–4968, 2015, doi: 10.1109/HICSS.2015.587.

A. K. Jha and I. Bose, “Innovation research in information systems: A commentary on contemporary trends and issues,†Inf. Manag., vol. 53, no. 3, pp. 297–306, 2016, doi: 10.1016/j.im.2015.10.007.

K. E. Huang, J. H. Wu, S. Y. Lu, and Y. C. Lin, “Innovation and technology creation effects on organizational performance,†J. Bus. Res., vol. 69, no. 6, pp. 2187–2192, 2016, doi: 10.1016/j.jbusres.2015.12.028.

M. K. Qteishat, H. H. Alshibly, and M. A. Al-ma’aitah, “The impact of e-ticketing technique on customer satisfaction: an empirical analysis,†J. Inf. Syst. Technol. Manag., vol. 11, no. 3, 2014, doi: 10.4301/s1807-17752014000300001.

E. Crespo-Almendros and S. Del Barrio-García, “Online airline ticket purchasing: Influence of online sales promotion type and Internet experience,†J. Air Transp. Manag., vol. 53, pp. 23–34, 2016, doi: 10.1016/j.jairtraman.2016.01.004.

I. Dianat, P. Adeli, M. Asgari Jafarabadi, and M. A. Karimi, “User-centred web design, usability and user satisfaction: The case of online banking websites in Iran,†Appl. Ergon., vol. 81, no. November 2018, p. 102892, 2019, doi: 10.1016/j.apergo.2019.102892.

J. R. Chou, “A psychometric user experience model based on fuzzy measure approaches,†Adv. Eng. Informatics, vol. 38, pp. 794–810, 2018, doi: 10.1016/j.aei.2018.10.010.

L. Kneale, S. Mikles, Y. K. Choi, H. Thompson, and G. Demiris, “Using scenarios and personas to enhance the effectiveness of heuristic usability evaluations for older adults and their care team,†J. Biomed. Inform., vol. 73, pp. 43–50, 2017, doi: 10.1016/j.jbi.2017.07.008.

J. M. C. Bastien, “Usability testing: a review of some methodological and technical aspects of the method,†Int. J. Med. Inform., vol. 79, no. 4, pp. e18–e23, 2010, doi: 10.1016/j.ijmedinf.2008.12.004.

R. Upitis, P. Abrami, J. Brook, D. Pickup, and L. Johnson, “Usability Testing for DREAM,†Procedia - Soc. Behav. Sci., vol. 171, pp. 543–552, 2015, doi: 10.1016/j.sbspro.2015.01.159.

S. J. Sinclair, C. J. Siefert, J. M. Slavin-Mulford, M. B. Stein, M. Renna, and M. A. Blais, “Psychometric Evaluation and Normative Data for the Depression, Anxiety, and Stress Scales-21 (DASS-21) in a Nonclinical Sample of U.S. Adults,†Eval. Heal. Prof., vol. 35, no. 3, pp. 259–279, 2012, doi: 10.1177/0163278711424282.

E. FrØkjaer, M. Hertzum, and K. Hornbæk, “Measuring usability: Are effectiveness, efficiency, and satisfaction really correlated?,†Conf. Hum. Factors Comput. Syst. - Proc., vol. 2, no. 1, pp. 345–352, 2000, doi: 10.1145/332040.332455.

H. W. Alomari, V. Ramasamy, J. D. Kiper, and G. Potvin, “A User Interface (UI) and User eXperience (UX) evaluation framework for cyberlearning environments in computer science and software engineering education,†Heliyon, vol. 6, no. 5, p. e03917, 2020, doi: 10.1016/j.heliyon.2020.e03917.

M. Weenk, A. P. B. Alken, L. J. L. P. G. Engelen, S. J. H. Bredie, T. H. van de Belt, and H. van Goor, “Stress measurement in surgeons and residents using a smart patch,†Am. J. Surg., vol. 216, no. 2, pp. 361–368, 2018, doi: 10.1016/j.amjsurg.2017.05.015.

K. M. P. Escanillan-Galera and C. M. Vilela-Malabanan, “Evaluating on user experience and user interface (UX/UI) of Enertrapp a mobile web energy monitoring system,†Procedia Comput. Sci., vol. 161, pp. 1225–1232, 2019, doi: 10.1016/j.procs.2019.11.236.

C. K. Coursaris and D. J. Kim, “A qualitative review of empirical mobile usability studies,†Assoc. Inf. Syst. - 12th Am. Conf. Inf. Syst. AMCIS 2006, vol. 5, no. May, pp. 2873–2879, 2006.

C. Andrzejczak and D. Liu, “The effect of testing location on usability testing performance, participant stress levels, and subjective testing experience,†J. Syst. Softw., vol. 83, no. 7, pp. 1258–1266, 2010, doi: 10.1016/j.jss.2010.01.052.

E. H. Lee et al., “The 21-Item and 12-Item Versions of the Depression Anxiety Stress Scales: Psychometric Evaluation in a Korean Population,†Asian Nurs. Res. (Korean. Soc. Nurs. Sci)., vol. 13, no. 1, pp. 30–37, 2019, doi: 10.1016/j.anr.2018.11.006.

P. J. Norton, “Depression Anxiety and Stress Scales (DASS-21): Psychometric analysis across four racial groups,†Anxiety, Stress Coping, vol. 20, no. 3, pp. 253–265, 2007, doi: 10.1080/10615800701309279.

S. Noohi, M. Ghalamfarsa, and E. Davoudi Monfared, “Comparing Anxiety, Depression, and Stress in Consanguineous Versus Non-Consanguineous Parents of Children With Deafness in Baqiyatallah Hospital’s Cochlear Implant Center From 2007 to 2009,†Hosp. Pract. Res., vol. 3, no. 3, pp. 85–89, 2018, doi: 10.15171/hpr.2018.19.

R. I. R. Willy Arief Pramono, Hanifah Muslimah Az-Zahra, “Evaluasi Usability pada Aplikasi MyTelkomsel dengan Menggunakan Metode Usability Testing,†vol. 3, no. 3, pp. 2235–2242, 2019.

A. I. Martins, A. F. Rosa, A. Queirós, A. Silva, and N. P. Rocha, “European Portuguese Validation of the System Usability Scale (SUS),†Procedia Comput. Sci., vol. 67, no. June 2017, pp. 293–300, 2015, doi: 10.1016/j.procs.2015.09.273.

G. Shirali, M. Shekari, and K. A. Angali, “Assessing Reliability and Validity of an Instrument for Measuring Resilience Safety Culture in Sociotechnical Systems,†Saf. Health Work, vol. 9, no. 3, pp. 296–307, 2018, doi: 10.1016/j.shaw.2017.07.010.

M. M. Mohamad, N. L. Sulaiman, L. C. Sern, and K. M. Salleh, “Measuring the Validity and Reliability of Research Instruments,†Procedia - Soc. Behav. Sci., vol. 204, no. November 2014, pp. 164–171, 2015, doi: 10.1016/j.sbspro.2015.08.129.

R. Bajpai and S. Bajpai, “Goodness of Measurement: Reliability and Validity,†Int. J. Med. Sci. Public Heal., vol. 3, no. 2, p. 112, 2014, doi: 10.5455/ijmsph.2013.191120133.

J. Song, J. Kim, D. R. Jones, J. Baker, and W. W. Chin, “Application discoverability and user satisfaction in mobile application stores: An environmental psychology perspective,†Decis. Support Syst., vol. 59, no. 1, pp. 37–51, 2014, doi: 10.1016/j.dss.2013.10.004.

D. V. Canter and K. H. Craik, “Environmental psychology,†J. Environ. Psychol., vol. 1, no. 1, pp. 1–11, 1981, doi: 10.1016/S0272-4944(81)80013-8.

W. Wetzlinger, A. Auinger, and M. Dörflinger, “Comparing Effectiveness, Efficiency, Ease of Use, Usability and User Experience When Using Tablets and Laptops,†vol. 8517, no. March 2015, 2014, doi: 10.1007/978-3-319-07668-3.

J. R. Lewis, “The System Usability Scale: Past, Present, and Future,†Int. J. Hum. Comput. Interact., vol. 34, no. 7, pp. 577–590, 2018, doi: 10.1080/10447318.2018.1455307.