Grouches, Extraverts, and Jellyfish: Assessment Validity and Game Mechanics in a Gamified Assessment

Abstract

Research into the use of both commercial and custom video games to assess individual differences, like personality, of players has revealed promising results. Virtual environments can allow researchers to analyze a variety of player behaviors and actions that correlate strongly with inherent personality traits. What is less understood is how an assessment game’s mechanics might affect a player’s inputs that determine the assessment’s validity. In this study, we developed a custom game and logging framework for an online study assessing the reliability and validity of transferring a traditional personality questionnaire into a game environment. The game was played by 212 college- aged participants in one of three conditions. The conditions represented different levels of game mechanics; including enemies and point earning. Using results from a traditional personality assessment as our ground truth, we compared player responses and play behavior in the game. We found that responses between the traditional assessment and game-based assessment in all conditions were consistent, indicating that the game mechanics did not interfere or alter significantly a player’s ability or decision to make personality-based responses. Additionally, we found several gameplay behaviors that can be used as predictors of individual differences.