Sequential Belief-Based Fusion of Manual and Non-manual Information for Recognizing Isolated Signs

This work aims to recognize signs which have both manual and non-manual components by providing a sequential belief-based fusion mechanism. We propose a methodology based on belief functions for fusing extracted manual and non-manual features in a sequential two-step approach. The belief functions based on the likelihoods of the hidden Markov models are used to decide whether there is an uncertainty in the decision of the first step and also to identify the uncertainty clusters. Then we proceed to the second step which utilizes only the non-manual features within the identified cluster, only if there is an uncertainty.