LOCEN Research Topic: Developing a virtual reality system for psychological experiments with apes and humans

Research topic

The goal of this research is to build a simple virtual reality system, actually a 3D system similar to a videogame (employing a computer, a screen, and a joystick), to support psychological experiements to be carried out with primates, in particular with Chimpanzees. This tool will be used by CNR-ISTC in collaboration with the primatologist Francine Dolins (), of , to implement experiments in the realm of intrinsic motivations. The next paragraph gives some background information on intrinsic motivation research to explain the context of the research, but the main challenge of this particular research described in this page is to build the virtual reality system that can support any type of behavioural experiment as the ones related to intrinsic motivations. So the challenge we proposed here is one of computer science. The type of experiments that will be carried out on the basis of the virtual reality system involve a subject that sits in front of the computer screen and through the joystick navigates an indoor virtual environment in search of resources and of interesting stimuli (i..e, intrinsically motivatint stimuli). This should give an idea of the type of computer application we are aiming for. We now give more information on the issues of intrinsic motivations. 

Psychology defines IMs by contrasting them to extrinsic motivations. Extrinsic motivations (EMs) motivate behaviour (i.e., energise, focus, and guide its learning) to accomplish biologically relevant states/resources in the environment (e.g., achievement of food/water, pain avoidance) relevant for the organism's homoeostatic regulation and procreation (Baldassarre, 2011). In the case of robots, EMs can be related to the accomplishment of the user tasks assigned to the robot. In contrast, IMs drive behaviour and actions for their own sake, e.g. to gain knowledge about the world (curiosity, exploration), to be exposed to novel and surprising stimuli (Berlyne, 1960), to reach higher levels of competence in changing the environment with an agent's own actions (White, 1959; Ryan and Deci, 2000). In general, IMs can drive the acquisition of knowledge and skills in the absence of tasks directly established by biological fitness or by the robot users (e.g., to learn to master the manipulation of different objects without being instructed to do so). Later, however, such knowledge and skills can become very useful to increase fitness or solve user tasks, and this justifies the evolutionary emergence of IMs in organisms and their utility for robots (Singh et al., 2010; Baldassarre, 2011; Baldassarre and Mirolli, 2013).  Computational investigations are clarifying the specific mechanisms that might underlie IMs (both in robots and organisms; Schmidhuber et al., 1991, 2010; Barto et al., 2004, 2010, 2013; Oudeyer et al., 2007; Baldassarre et al., 2013). Here we adopt a threefold classification of such IM mechanisms (Oudeyer and Kaplan, 2007; Barto et al., 2013; Baldassarre and Mirolli, 2013): (a) novelty-based IMs: related to novel, previously non-experienced, stimuli; (b) prediction-based IMs: related to the violation of the agent's predictions; (c) competence-based IMs: related to action, i.e. to the agent's competence to change the world and accomplish its own goals. Importantly, note how IM mechanisms have in common the fact that they are related to an increase of the agent's knowledge (broadly defined). These mechanisms can serve different proximal functions within the robot controller/organism brain, e.g. to drive attention, produce learning signals, select actions that maximise learning progress, or generate goals autonomously (Baldassarre and Mirolli, 2013).

Research specific problems

• Elaboration of the sofware project.
• Selection of the programming language to use to implement the virtual reality system.
• Selection of suitable existing libraries to rely upon.
• Implementing the virtual reality system to support the psychological experiments with primates (chimpanzees, human adults, children). 

Research method

• Computer science.
• (Simple) virtual reality system.
• Integration of hardware, in particular a computer, a screen a joystick, to implement simple virtual reality systems.
• Integration, use, and modification of out-of-the-shelf open-source components to implement the system.

Examples of research of this type carried out by the group

(see full references below; the pdf files of the paper are retrievable from here)

• Caligiore et al. (in press).
• Polizzi di Sorrentino et al. (2014).
• Taffoni et al. (2014).

Requested motivations of the candidate

• Strong interest in the topic and motivation to carry out research on it (very important)
• Desire to acquire the knowledge and methods of the group
• Professionality, reliability.

Requested knowledge of the candidate

Requested skills of the candidate

• Capacity to read and understand scientific papers in English
• Capacity to contribute to the design of sofware systems
• Capacity to implement the simple virtual reality system of the project
• Capacity to contribute to write reports in English

References

• Baldassarre, G. (2011). What are intrinsic motivations? A biological perspective. In IEEE ICDL 2011, e1-8.
• Baldassarre, G., Mannella, F., Fiore, V.G., Redgrave, P., Gurney, K., Mirolli, M. (2013). Intrinsically motivated action-outcome learning and goal-based action recall: A bio-constrained computational model. Neural Networks, 41, 168-187.
• Baldassarre, G., Mirolli, M. (eds.)(2013). Intrinsically motivated learning in natural and artificial systems. Berlin: Springer.Barto A., Mirolli M., Baldassarre G. (2013). Novelty or surprise? Frontiers in Cognitive Sci., 4 (907).
• Barto A.G., Singh S., Chentanez N. (2004), Intrinsically Motivated Learning of Hierarchical Collections of Skills International Conference on Developmental Learning (ICDL), LaJolla, CA, USA.Berlyne, D.E. (1960). Conflict, Arousal. and Curiosity. New York: McGraw-Hill. 
• Caligiore, D.; Mustile, M.; Cipriani, D.; Redgrave, P.; Triesch, J.; De Marsico, M. & Baldassarre, G. (in press). Intrinsic motivations driving learning of eye movements: an experiment with human adults. PLOS One. 
• Oudeyer P.-Y., Kaplan F. (2007). What is Intrinsic Motivation? A Typology of Computational Approaches. Frontiers in Neurorobotics, 1, 6. 
• Oudeyer P.-Y., Kaplan F., Hafner V.V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Transactions on Evolutionary Computation, 11 (2), 265-286.
• Polizzi di Sorrentino, E.; Sabbatini, G.; Truppa, V.; Bordonali, A.; Taffoni, F.; Formica, D.; Baldassarre, G.; Mirolli, M. & Guglielmelli, Eugenio, V. E. (2014). Exploration and learning in capuchin monkeys (Sapajus spp.): the role of actionoutcome contingencies. Animal Cognition, 17 (5), 1081-1088.
• Ryan R.M., Deci E.L. (2000). Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemp Educ Psychol, 25 (1), 54-67.
• Schmidhuber, J. (1991). Curious model-building control systems. International Joint Conference on Artificial Neural Networks, 2, 1458-1463.Schmidhuber, J. (2010). Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990-2010). IEEE TAMD, 2 (3), 230 -247.
• Singh S., Lewis R.L., Barto A.G., Sorg J. (2010). Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development, 2, 70-82.
• Taffoni, F.; Tamilia, E.; Focaroli, V.; Formica, D.; Ricci, L.; Di Pino, G.; Baldassarre, G.; Mirolli, M.; Guglielmelli, E. & Keller, F. (2014). Development of goal-directed action selection guided by intrinsic motivations: an experiment with children. Experimental Brain Research, 232 (7), 2167-2177.
• White R.W. (1959). Motivation reconsidered: the concept of competence. Psych Review, 66, 297-333.