improving the accuracy of automatic facial expression recognition in speaking subjects with deep learning

Clicks: 327
ID: 129895
2020
Free PDF
Article Quality & Performance Metrics
Overall Quality
Not rated
Combines reader engagement with the AI quality analysis. This article has not been analysed, so there is no overall score — reader engagement is measured and shown alongside.
AI Quality Assessment
Not analyzed
Readership in this journal
Steady

Ranked #5 of 27 articles by views in cancer immunology, immunotherapy : cii

Most read Least read

Bar heights use a square-root scale.

Mint this article as an NFT
Not yet minted

Create a permanent, verifiable on-chain record of this article on the Scimatic Network. The NFT is held in your Journament account, and you can withdraw it to your own wallet at any time.

5 SUSD one-off · no wallet required
Sign in to mint
Abstract
When automatic facial expression recognition is applied to video sequences of speaking subjects, the recognition accuracy has been noted to be lower than with video sequences of still subjects. This effect known as the speaking effect arises during spontaneous conversations, and along with the affective expressions the speech articulation process influences facial configurations. In this work we question whether, aside from facial features, other cues relating to the articulation process would increase emotion recognition accuracy when added in input to a deep neural network model. We develop two neural networks that classify facial expressions in speaking subjects from the RAVDESS dataset, a spatio-temporal CNN and a GRU cell RNN. They are first trained on facial features only, and afterwards both on facial features and articulation related cues extracted from a model trained for lip reading, while varying the number of consecutive frames provided in input as well. We show that using DNNs the addition of features related to articulation increases classification accuracy up to 12%, the increase being greater with more consecutive frames provided in input to the model.
Reference Key
bursic2020appliedimproving Use this key to autocite in the manuscript while using SciMatic Manuscript Manager or Thesis Manager
Authors ;Sathya Bursic;Giuseppe Boccignone;Alfio Ferrara;Alessandro D’Amelio;Raffaella Lanzarotti
Journal cancer immunology, immunotherapy : cii
Year 2020
DOI
10.3390/app10114002
URL
https://www.mdpi.com/2076-3417/10/11/4002
Keywords
emotion recognition affective computing facial expression recognition

Citations

No citations found. To add a citation, contact the admin at info@scimatic.org

Comments

No comments yet. Be the first to comment on this article.