Viewing the Viewers: A Novel Challenge for Automated Crowd Analysis

We focus on the automated analysis of spectator crowd, that is, people watching sport contests alive (in stadiums, amphitheaters etc.), or, more generally, people "watching the activities of an event [...] interested in watching something specific that they came to see" [ 2]. This scenario differs substantially from the typical crowd analysis setting (e. g. pedestrians): here the dynamics of humans is more constrained, due to the architectural environments in which they are situated; people are expected to stay in a fixed location most of the time, limiting their activities to applaud, support/heckle the players or discuss with the neighbors. In this paper, we start facing this challenge by following a social signal processing approach, which grounds computer vision techniques in social theories. More specifically, leveraging on social theories describing expressive bodily conduct, we will show how, by using computer vision techniques, it is possible to distinguish fan groups belonging to different teams by automatically detecting their liveliness in different moments of the match, even when they are merged in the stands. Moreover, we will show how, only by automatically detecting crowd's motions on the stands, it is possible to single out the most salient events of the match, like goals, fouls or shots on goal.