Published January 1, 2016 | Version v1
Journal article Open

Multimodal emotion recognition based on peak frame selection from video

  • 1. INRS EMT, Montreal, PQ, Canada
  • 2. Univ Udine, I-33100 Udine, Italy
  • 3. Bahcesehir Univ, Istanbul, Turkey

Description

We present a fully automatic multimodal emotion recognition system based on three novel peak frame selection approaches using the video channel. Selection of peak frames (i.e., apex frames) is an important preprocessing step for facial expression recognition as they contain the most relevant information for classification. Two of the three proposed peak frame selection methods (i.e., MAXDIST and DEND-CLUSTER) do not employ any training or prior learning. The third method proposed for peak frame selection (i.e., EIFS) is based on measuring the "distance" of the expressive face from the subspace of neutral facial expression, which requires a prior learning step to model the subspace of neutral face shapes. The audio and video modalities are fused at the decision level. The subject-independent audio-visual emotion recognition system has shown promising results on two databases in two different languages (eNTERFACE and BAUM-1a).

Files

bib-cd5cd645-fbab-4473-af59-a35d697a6e30.txt

Files (170 Bytes)

Name Size Download all
md5:80f2fa03a61ab51c25b7fe6f69ff812b
170 Bytes Preview Download