Abstract: Systematic principles for integrating symbolic and subsymbolic processing will be developed in the project. Key aims are to ensure that the resulting total hybrid system retains desirable properties of both processing levels. On the one side the signal processing abilities, robustness and learning capability of neural networks should be preserved. On the other side advantage should be taken of the ability of rule-based systems to exploit high level knowledge and existing algorithms and to explain (to a user) why conclusions were reached in particular cases. The methodologies developed in the project are tested in a challenging multimedia application related to human computer interaction, which is recognition of emotion based on both voice and visual cues. Low level features are extracted from signals using neural networks and subsequent formulation of rules provides a conceptual framework, substantial for emotion analysis