Player's Affective States as Meta AI Design on Augmented Reality Games

Andry Chowanda - Bina Nusantara University, Jakarta, Indonesia
Vincentius Dennis - Bina Nusantara University, Jakarta, Indonesia
Virya Dharmawan - Bina Nusantara University, Jakarta, Indonesia
Joseph Ramli - Bina Nusantara University, Jakarta, Indonesia


Citation Format:



DOI: http://dx.doi.org/10.30630/joiv.7.2.1022

Abstract


Games are considered one of the most popular entertainment forms worldwide. The interaction in the game environment makes the players addicted to playing the game. One technique to build an addicting game is utilizing the player's emotions using Meta Artificial Intelligence (AI). The player's emotions can be utilized by adjusting the game difficulty. Most of the game offers static and steady difficulty development throughout the game. This research proposes a Meta AI game design using the player's affective states. We argue that a dynamic difficulty development throughout the game will increase the player's game experiences. The player's facial expressions are utilized to extract the player's affective state information. To recognize the player's facial expressions, a Facial Expressions Recognition (FER) model was trained using VGG-16 architecture and The Indonesian Mixed Emotion Dataset (IMED) dataset in addition to a self-collected dataset. The emotions recognition model (from player's facial expressions) achieved the best validation accuracy of 99.98%. The model was implemented in the proposed Meta AI game design. The Meta AI game design proposed in this game was implemented in several game scenarios to be compared and evaluated. The proposed Meta AI game design was evaluated by 31 respondents using Game Experiences Questionnaire (GEQ). Overall, the results show that the game with Meta AI and Augmented Reality implemented significantly improved the Game Experiences Questionnaire (GEQ) score and the player's overall satisfaction compared to the other game scenarios.

Keywords


Affective States, Meta AI, Game Design, FER, Augmented Reality

Full Text:

PDF

References


Andry Chowanda, Peter Blanchfield, Martin Flintham, and Michel Valstar. Erisa: Building emotionally realistic social game-agents companions. In International conference on intelligent virtual agents, pages 134–143. Springer, 2014.

M Taufik Akbar, M Nasrul Ilmi, Imanuel V Rumayar, Jurike Moniaga, Tin-Kai Chen, and Andry Chowanda. Enhancing game experience with facial expression recognition as dynamic balancing. Procedia Computer Science, 157:388–395, 2019.

Doyo Setiono, David Saputra, Kaleb Putra, Jurike V Moniaga, and Andry Chowanda. Enhancing player experience in game with affective computing. Procedia Computer Science, 179:781–788, 2021.

Daiki Satoi and Yuta Mizuno. Meta artificial intelligence and artificial intelligence director. pages 1–8, 2019.

Vincentius Dennis, Virya Dharmawan, Joseph Ramli, Jurike Moniaga, and Andry Chowanda. Meta ai integrated with facial expression recognition in games. ICIC Express Letter, 15(9):925–932, 2021.

Robin Hunicke. The case for dynamic difficulty adjustment in games. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in computer entertainment technology, pages 429–433, 2005.

Rhio Sutoyo, Davies Winata, Katherine Oliviani, and Dedy Martadinata Supriyadi. Dynamic difficulty adjustment in tower defence. Procedia Computer Science, 59:435–444, 2015.

Jurike V Moniaga, Andry Chowanda, Agus Prima, M Dimas Tri Rizqi, et al. Facial expression recognition as dynamic game balancing system. Procedia Computer Science, 135:361–368, 2018.

Adeleh Ebrahimi and Mohammad-R. Akbarzadeh-T. Dynamic difficulty adjustment in games by using an interactive self-organizing architecture. In 2014 Iranian Conference on Intelligent Systems (ICIS), pages 1–6, 2014.

Chris Pedersen, Julian Togelius, and Georgios N Yannakakis. Modeling player experience in super mario bros. In 2009 IEEE Symposium on Computational Intelligence and Games, pages 132–139. IEEE, 2009.

Johan Hagelback and Stefan J Johansson. Measuring player experience on runtime dynamic difficulty scaling in an rts game. In 2009 IEEE Symposium on Computational Intelligence and Games, pages 46–52. IEEE, 2009.

Eva Hudlicka. Affective game engines: motivation and requirements. In Proceedings of the 4th international conference on foundations of digital games, pages 299–306. ACM, 2009.

Jan K Argasinski and Pawel Wegrzyn. Affective patterns in serious games. Future Generation Computer Systems, 92:526–538, 2019.

Eva Hudlicka. Virtual affective agents and therapeutic games. pages 81–115, 2016.

Nicole Lazzaro. Why we play games: Four keys to more emotion in player experiences. In Proceedings of GDC, volume 306, pages 1–8, 2004.

Andrew Andrew, Adithya Nugraha Tjokrosetio, and Andry Chowanda. Dynamic difficulty adjustment with facial expression recognition for improving player satisfaction in a survival horror game. ICIC Express Letters, 14(11):1097–1104, Nov 2020.

Vasileios Terzis, Christos N Moridis, and Anastasios A Economides. Measuring instant emotions during a self-assessment test: the use of facereader. In Proceedings of the 7th international conference on methods and techniques in behavioral research, pages 1–4, 2010.

Ji Ho Park, Nayeon Lee, Dario Bertero, Anik Dey, and Pascale Fung. Emojive! collecting emotion data from speech and facial expression using mobile game app. In INTERSPEECH, pages 827–828, 2017.

Wijnand A Ijsselsteijn, Yvonne AW de Kort, and Karolien Poels. The game experience questionnaire. Eindhoven: Technische Universiteit Eindhoven, 46(1), 2013.

Dewi Yanti Liliana, T Basaruddin, and Imelda Ika Dian Oriza. The indonesian mixed emotion dataset (imed) a facial expression dataset for mixed emotion recognition. In Proceedings of the 2018 International Conference on Artificial Intelligence and Virtual Reality, pages 56–60, 2018.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.