My current research addresses the problem of sensing and understanding users’ interactive actions and intentions for achieving multimodal human-computer interaction in natural settings. A critical issue here is that human face and body exhibit complex and rich dynamic behavior that is all non-linear, time varying, and context dependent (person, task, mood/affect dependent). Thus, the main focus of my research is on multimodal human-computer interaction models from multi-sensory observations. The research done so far addressed the area of analyzing the user’s behavior (e.g., facial expressions, body and head pose, eye tracking, etc.) in his personal environment (e.g., home or office) as well as bimodal emotion recognition from facial expressions and audio information. Another important aspect is the analysis of multimedia information retrieval techniques toward extracting affective information from the multimedia data