Affective Computing

You are here

Affective Computing is a field of research that "relates to, arises from, or deliberately influences emotion or other affective phenomena" (Picard, MIT Press 1997).

Many recent theories of emotion have shown the autonomic nervous system (ANS) activity as one of the major component of the emotion response. Indeed, changes in emotional states often reflect modifications in facial expressions, voice, gestures and other autonomic correlates (e.g., electrodermal activity (EDA), heart rate variability (HRV)), in order to communicate, sometimes unconsciously, personal feelings to other people.

The automatic emotion recognition is one of the most important applications in neuroscience and, often, is identified within the so-called affective computing field. Mainly, such a technical field refers to the engineering approaches able to link physiological patterns to different emotions.

Our research focuses mainly on the development of new mathematical methods, models and devices in order to advance prior understanding of emotional processes. Particularly, we design new technology to acquire ANS signals such as HRV, EDA, respiration, eye-gaze, and we developed novel mathematical methods to estimate possible biomarkers able to characterize the emotional state of a subject and its possible dysregulation.

Concerning the analysis of cardiovascular signals, we have been studying novel methods in the field of nonlinear dynamic analysis, especially applied to the Heart Rate Variability (HRV) series. This signal is extracted from the RR series, a series given by the time duration of the intervals between each R peak and the successive in ECG.

Many studies in the recent literature demonstrated that the use of nonlinear methods in HRV analysis provide a relevant additional prognostic information on the autonomic regulation during physiological and pathological conditions.

We applied several nonlinear techniques to HRV series, obtaining great results in the recognition of psychophysical diseases and in the frame of affective computing, in order to recognize emotional responses to sensorial elicitation (visual, acoustic and tactile).

       

 

Figure (left) The Poincaré Plot of a RR series. (Right) Example of the trend of the area of Poincaré Plot as a function of normalzed HF power

Among several studies, a special mention goes to the inhomogeneous point-process framework for heartbeat dynamics to perform instantaneous emotion recognition. More details can be found in the dedicated web page.

Also for EDA analysis, we have developed a novel model for the decomposition of the signal into its main components (tonic and phasic), called cvxEDA.

EDA can be considered one of the most common observation channels of sympathetic nervous system activity, and manifests itself as a change in electrical properties of the skin, such as skin conductance (SC). The proposed cvxEDA model describes SC as the sum of three terms: the phasic component, the tonic component, and an additive white Gaussian noise term incorporating model prediction errors as well as measurement errors and artifacts. This model is physiologically inspired and fully explains EDA through a rigorous methodology based on Bayesian statistics, mathematical convex optimization and sparsity.

 

Application of EDA decomposition procedure to the EDA signal recorded during affective experimental paradigm.

In addition to the study of the ANS, our group performed several studies on the central nervous system (CNS) through the analysis of the electroencephalographic signals.

Example of EEG signals

 

Example of power spectral density and functional connectivity analysis results.

Our research is the results of a combination of standard quantitative EEG and functional connectivity analyses together with advanced machine learning tools in order to assess the central response to different kinds of affective stimulations.

 

Info

Computational Physiology & Biomedical Instruments Group, University of Pisa

SPIN-OFF

Feel-ING is a new start-up specialized in developing and implementing innovative Information and Communication Technology solutions for the optimization of the psychophysical performance and the promotion of wellbeing and mental health