-
Enseignant chercheur
- Tel : (+33) 01 77 48 70 78
Manolo HINA
Axe(s) de recherche : Systèmes intelligents communiquants
Domaine(s) de compétence :
intelligence artificielle, apprentissage automatique, informatique autonomique
Diplôme(s) :
Ph.D. en informatique (France), Ph.D. en génie (Canada), Master en informatique (Canada), License en génie informatique (Philippines)
Biographie
Enseignant-chercheur, ECE Paris Ecole d'Ingénieurs (2015 - à ce jour)
Postdoctorat en informatique - Université de Versailles St-Quentin-en-Yvelines, 2014-2016
Ph.D. en informatique - Université de Versailles St-Quentin-en-Yvelines, France, 2011
Ph.D. en génie (Recherche appliquée) - Université du Québec, Ecole de technologie supérieure, Canada, 2010
Master en informatique - Université Concordia, Montréal, Québec, Canada, 2004
License en génie informatique, Université Mapua (auparavant "Mapua Institute of Technology"), Manille, Philippines, 1986
Doyen, College of Computer Studies, AMA International University - Bahrain, Bahreïn, 2012-2013
Professeur en Informatique, Bahrain Polytechnic, Bahreïn, 2013-2014
Professeur en Informatique, Collège Vanier, Montréal, Québec, Canada, 2011
Enseignant en informatique, Université Concordia, Montréal, Québec, Canada, 1997-2003
Senior Programmeur, Interpacific Data Management Ltd., Hongkong, 2012-2013
Enseignant en informatique, Université de la Ville de Manille, Philippines, 1987-1990
Membre, Advisory Board -- The Society of Digital Information and Wireless Communications (SDIWC), 2015 à ce jour
Retour à la liste des membresPublications de Manolo HINA
« Cognition of Driving Context for Driving Assistance »
par Manolo HINA et Assia SOUKANE
ICAIA 2018: 20th International Conference on Artificial Intelligence and Applications , Prix: Best Paper Award , Kuala Lumpur, Malaisie , 2018
Liste des auteurs : Manolo Dulva Hina, Clément Thierry, Assia Soukane, Amar Ramdane-Cherif
Abstract
In this paper, we presented our innovative way of determining the driving context for a driving assistance system. We invoke the fusion of all parameters that describe the context of the environment, the vehicle and the driver to obtain the driving context. We created a training set that stores driving situation patterns and from which the system consults to determine the driving situation. A machine-learning algorithm predicts the driving situation. The driving situation is an input to the fission process that yields the action that must be implemented when the driver needs to be informed or assisted from the given the driving situation. The action may be directed towards the driver, the vehicle or both. This is an ongoing work whose goal is to offer an alternative driving assistance system for safe driving, green driving and comfortable driving. Here, ontologies are used for knowledge representation.
« Ontological and Machine Learning Approaches for Managing Driving Context in Intelligent Transportation »
par Manolo HINA et Assia SOUKANE
KEOD 2017, 9th International Conference on Knowledge Engineering and Ontology Development , Madeira, Portugal , 2017
Liste des auteurs : Manolo Dulva Hina, Clément Thierry, Assia Soukane & Amar Ramdane-Cherif
Abstract
In this paper, a novel approach of managing driving context information in smart transportation is presented. The driving context refers to the ensemble of parameters that make up the contexts of the environment, the vehicle and the driver. To manage this rich information, knowledge representation using ontology is used and through it, such information becomes a source of knowledge. When this context information (i.e. basically a template or model) is instantiated with actual instances of objects, we can describe any kind of driving situation. Furthermore, through ontological knowledge management, we can find the answers related to various queries of the given driving situation. A smart vehicle is equipped with machine learning functionalities that are capable of classifying any driving situation, and accord assistance to the driver or the vehicle or both to avoid accident, when necessary. This work is a contribution to the ongoing research in safe driving, and a specific application of using data from the internet of things.
« Cognition of Driving Context in a Connected and Semi-Autonomous Vehicle: A Perspective »
par Manolo HINA et Assia SOUKANE
DeCPS 2017, Workshop on Dependable Cyber Physical Systems - Ada Europe 2017 , Vienne, Autriche , 2017
Liste des auteurs : Manolo Dulva Hina, Assia Soukane, Amar Ramdane-Cherif
Abstract
In this paper, we present our approach for the cognition of driving context in a connected and semi-autonomous vehicle. This cyber-physical vehicle has three main components: the embedded system, the networking and real-time system and the intelligent system that communicate with each other via Cloud computing infrastructure. This paper is about the functionalities of the intelligent system in the cognition of driving context and the actions that it invokes for each of these driving situations. The driving context is determined as a result of the fusion of various parameters representing the context of the environment, the vehicle and the driver. The fission process yields the action that must be implemented with regards to the driving situation in consideration. Ontology is used as a tool for knowledge representation. This work is a contribution to safe driving in a connected and semi-autonomous vehicle.
« Computational Intelligence on the Cognition of Driving Context for a Safe Driving Assistance System »
par Manolo HINA et Assia SOUKANE
International Journal of Software Science and Computational Intelligence , 2017
Liste des auteurs : Giacomo Brioschi, Manolo Dulva Hina, Assia Soukane, Amar Ramdane-Cherif and Marco Colombetti
Abstract
Given the context of the driver, of the vehicle and of the environment, this safe driving assistance system deduces the traffic context and provide the driver with the necessary assistance by providing message notification or alert about the situation or of the infraction that was committed, or trigger an action directly affecting the vehicle. To do so, the system performs signal processing cognition related to these context parameters. This paper illustrates the system’s knowledge representation using ontology, the rules related to the fusion of context parameters and the deduction of driving context using the Semantic Web Rule Language. A fission component deals with the steps needed to implement the desired safe driving action that is directed towards the driver, to the vehicle, or both. The paper shows how ontology is used to represent driving model, the road environment and the set of actions related to invoke safe driving assistance. This work is a contribution to the ongoing research for the prevention of vehicular traffic accident.
« Novel Approaches in Human-Vehicle Interaction Interface of a Vehicle Driving Assistance System »
par Manolo HINA
IHCI 2017, 11th International Conference on Interfaces and Human-Computer Interaction , Lisbonne, Portugal , 2017
Liste des auteurs : Manolo Dulva Hina, Hongyu Guan, Amar Ramdane-Cherif
Abstract
Project CASA is a collaborative work of five industrial partners and one educational institution. Its goal is to contribute to the advancement of safe and green driving by proposing ADAS (advanced driving assistance system) based on smartphone. It is implemented by capturing and combining all the parameters related to the context of the driver, the vehicle, the environment and the infrastructure altogether to deduce the current driving situation. Based upon such driving situation, we may invoke driving assistance scheme in a form of an audio and visual message to notify or alert the driver. In the area of human-computer interaction, we have developed an interface that alerts or notifies a driver. We also work on the detection of level of stress of the driver. It is necessary on our part that a stressed driver should not be further stressed by our driving assistance messages. This work has been presented to the public and had been received warmly.
« Enhanced Mobility for the Elderly through a Context-Aware Vehicular System »
par Manolo HINA et Assia SOUKANE
Institute of Computer Science Embedded Systems and Assistive Technology for Last Mile Mobility , Chapter 6, "Enhanced Mobility for the Elderly through a Context-Aware Vehicular System" , pages 63-78 , 2017
Liste des auteurs : Manolo Dulva Hina, Giacomo Brioschi, Assia Soukane, Hongyu Guan, Amar Ramdane-Cherif and Marco Colombetti
Abstract
This work is an undertaking intended to contribute to the promotion of autonomy and good health for the elderly using smart, context-aware vehicular system. The project’s principal users are the old people, normally in their retirement age. Serious Gaming is integrated into this system, providing physical and mental activity challenges to the actors concerned, including an option to participate in events with their peers. The secondary users are medical practitioners who provide the challenges that are suitable to the health status of the participants. Results of the activities are also evaluated by medical personnel. The vehicular system is smart and context-aware, allowing it to invoke actions directed towards the driver, the vehicle or both during driving situations where safety driving intervention is needed. The results of preliminary evaluation based on one-month test with participants indicated that their mobility, autonomy and health conditions improve with participation in the project.
« Secured Data Processing, Notification and Transmission for a Human-Vehicle Interaction System »
par Manolo HINA
ITSC 2016, 19th IEEE International Conference on Intelligent Transportation Systems , Rio de Janeiro, Brésil , 2016
Liste des auteurs : Manolo Dulva Hina, Hongyu Guan, Nan Deng, and Amar Ramdane-Cherif
Abstract
Project CASA (Car Safety Apps) is a joint project of a consortium composed of 5 industrial and one academic partners in France with an aim of building an Android app that promotes safe and green driving, and secured data transmission. In this paper, we focus on the security aspects and features of our project. We present that the multimodal data processing of our work is safe and secured. The driver notification and alert messages are intended to keep the driver safe from road accidents. The data transmission using LiFi, instead of WiFi, makes the communication secured due to the absence of radio wave’ interference with other road users and the absence of electromagnetic interference makes it a green technology. The contributions of our work are as follows: (i) generation of assistive messages that are suitable for safe and green driving, and (ii) secured and green data transmission.
« Multimodal Fusion and Fission Techniques for a Safe Driving Environment »
par Manolo HINA et Assia SOUKANE
ITS European Congress 2016 , Glasgow, Scotland, UK , 2016
Liste des auteurs : Manolo Dulva Hina, Assia Soukane, Amar Ramdane-Cherif
Abstract
In this paper, we present a synopsis of Project ASSIA aimed at delivering safe driving environment. We also present our multimodal signal processing techniques in fusioning various signals denoting context of the driver, the vehicle and the environment in order to obtain the actual driving situation. For each driving situation, an action might be needed. The fission component determines what action needs to be done for such situation. The recipient of the action may be the driver, the vehicle or both. The paper contains significant contributions with regards to techniques in multimodal fusion, fission, knowledge representation using ontology, machine learning and formal verification. A scenario is presented to simulate the system’s functionalities.
« Techniques for Cognition of Driving Context for Safe Driving Application »
par Manolo HINA et Assia SOUKANE
ICI*CC 2016, 15th IEEE International Conference on Cognitive Informatics and Cognitive Computing , Stanford, CA, USA , 2016
Liste des auteurs : Giacomo Brioschi, Manolo Dulva Hina, Assia Soukane, Amar Ramdane-Cherif and Marco Colombetti
Abstract
In this work, given the context of the driver, of the vehicle and of the environment, our objective is to correctly recognize the traffic situation and provide the driver with the corresponding assistance by providing notification or alert about the situation or the infraction that was committed, or acting directly on the vehicle. To do so, we need to consider the signal processing related to these context parameters. We built knowledge representation using ontology, built rules related to the fusion of context parameters and the deduction corresponding to the traffic situation using Semantic Web Rule Language. We built fission component that deals with traffic situation and the corresponding action directed towards the driver or the vehicle. Ontology is used to represent driving model and road environment. This work is our contribution in the ongoing research for the prevention of vehicular traffic accident.
« Serious Gaming: Autonomy and Better Health for the Elderly »
par Manolo HINA
CompSysTech 2016, 17th ACM International Conference on Computer Systems and Technologies , Palermo, Italy , 2016
Liste des auteurs : Manolo Dulva Hina, Amar Ramdane-Cherif and Sébastien Dourlens
Abstract
“Serious Gaming for the Elderly” is an undertaking that is intended to contribute to the promotion of autonomy and good health for the elderly. Its principal users are the old people, normally in their retirement age. They are given physical activity challenges as well as mental challenge and an option to participate in events with their peers. The secondary users are medical practitioners who provide exercises suitable to the health status of the participants. Results of the activities are also evaluated by medical personnel. New challenges are designed based on results of previous ones. The beta version of the system is initially tested with five specimen participants. The results of preliminary evaluation based on one-month test with participants indicated that their autonomy and health conditions improve with participation in serious gaming.
« Project CASA: Towards Safe and Green Driving for Real-time Driving Events »
par Manolo HINA
Springer Lecture Notes in Networks and Systems, Proceedings of IntelliSys 2016, SAI Intelligent Systems Conference , 2016
Liste des auteurs : Manolo Dulva Hina, Hongyu Guan, Nan Deng, Amar Ramdane-Cherif
Abstract
Project CASA (Car Safety Apps) is a project of a consortium of industrial and academic partners in France, with an aim of building an Android app that promotes safe and green driving. In this paper, an innovative way of capturing real-time signals, combining them to determine the current driving situation, associate the required action and generate the corresponding message towards the driver and the vehicle will be presented. Every message is suitable to the real-time driving context and event. The contributions of this work are as follows: (i) generation of assistive messages that are suitable for safe and green driving, (ii) dynamic and real-time consideration of driver status, vehicle status, and of the environment (iii) providing priority level to traffic messages based on the danger involved in the scenario.
« Multimodal Fusion, Fission and Virtual Reality Simulation for an Ambient Robotic Intelligence »
par Manolo HINA
Elsevier Procedia Computer Science , vol. 52 , pages 218 - 225 , London, UK , 2015
Liste des auteurs : Omar Adjali, Manolo Dulva Hina, Sébastien Dourlens, Amar Ramdane-Cherif
Abstract
In this paper, we present an architecture that demonstrates multimodal fusion and fission using semantic agents and web services that interacts with worldwide-web computers, services and users. This solution extracts the meaning of a situation using the semantic memory of agents to manage the interaction process involved. The fusion of values from different sensors produces an event that needs implementation. The fission process suggests a detailed set of actions that are for implementation. Before such actions are implemented by actuators, these actions are first evaluated in a virtual environment which mimics the real-world environment. If no danger arises from such virtual evaluation, then implementation is feasible. Otherwise, there might be a need to add one or more smaller actions to render the action safe and free from danger. Our work presents the following contributions: (i) a design of agent memory and a model of the world environment using a knowledge representation language that is compatible with the current standards, (ii) creation of a pervasive architecture with several scenarios of composition and adaptation, (iii) presentation of how agents and services interact to provide support in a real-world, and (iv) simulation of an event in a virtual environment to assess the feasibility of the event’s implementation.
« Techniques for Multimodal Fusion and Fission for an Intelligent Robotic Application »
par Manolo HINA
Journal of Advances in Robotics and Automation, Special Issue: Robotics in Surgery , vol 1 , pages 1-17 , 2015
Liste des auteurs : Omar Adjali, Manolo Dulva Hina, Sébastien Dourlens, Amar Ramdane-Cherif
Abstract
In this paper, we present our techniques of multimodal fusion and fission for an intelligent robotic application. The fusion of data from different sensors produces an event that needs implementation. The fission process produces a detailed set of actions that are to be implemented. Before such set of actions is implemented, this set of actions is evaluated first in a virtual environment to determine if the actions can be carried out in a real-world environment. If an obstacle is noted during the virtual evaluation, this implies that the action cannot be carried out in real environment. To mitigate such situation, one or more smaller actions may be added in order to get over the obstacle. We consider a sample robotic application scenario to demonstrate this situation. The contributions of our work are as follows: (i) a design of agent memory and a model of environment using a knowledge representation language that is compatible with current standards, (ii) creation of a pervasive architecture with several scenarios of composition and adaptation, (iii) presentation of how agents and services interact to provide support in real-world, and (iv) simulation of event on a virtual environment to assess the feasibility of an action’s implementation.
« Architecture and Ontological Modelling for Assisted Driving and Interaction »
par Manolo HINA
International Journal of Advanced Computer Research , Vol 5, No 20 , 2015
Liste des auteurs : Meryem Boudra, Manolo Dulva Hina, Amar Ramdane-Cherif, and Chakib Tadj
Abstract
Various proprietary vehicular technologies have been integrated on commercial vehicles. Most of these assistance systems, however, are systems of perception, of alert or of very simple commands which are attached to one or two sensors only. There is not even a support with regards to information involving environmental context. There is not even use of the user profile and actual state of the driver. There is not even sharing of modalities (such as voice synthesis for example) because the products and software of each OEM (original equipment manufacturer) or designer are often closed to proprietary constraints. These are the weaknesses that our work would like to address. We take a traffic situation, consider the traffic rules, and consider the context of the driver, the car and the environment and provide guidance to the driver. Our aim is to provide assistance to prevent traffic accident. We begin with “Turn Left in an intersection” situation and progress forward. An architectural design has already been developed to cover all kinds of traffic conditions. Ontology is used to represent driving modelling and road environment. Our aim is to contribute to the body of knowledge in the domain of prevention of vehicular traffic accident.
« Towards Self-Optimization of a Pervasive Computing Task »
par Manolo HINA
Elsevier Procedia Computer Science , vol 10 , Niagara Falls, Ontario, Canada , 2012
Liste des auteurs : Manolo Dulva Hina, Amar Ramdane-Cherif, Marietta Sicat, and Rodolfo Vargas
Abstract
In pervasive computing, our aim for the end user is continuous computing anytime, anywhere whenever possible, in a way that the computing system adapts to the user's context. An integral component of pervasive computing is pervasive task, a computing task that follows wherever the user is, adapting to the computing needs of the user and self-optimizing on user's behalf. In this paper, a task is represented by a set of computing files which then are mapped to their corresponding applications and suppliers, and optimal quality of service dimensions for each application. Here, we present how pervasive task can be realized, implementing selfconfiguration and self-optimization to adapt to the user's preferences. This work is a contribution in the ongoing research towards adaptation of computing system to suit the user's context and preferences in a pervasive computing system.
« Patterns Architecture for Fusion Engines »
par Manolo HINA
Springer- Verlag Lecture Notes in Computer Science, Proceedings of ICOST 2011, 9th International Conference on Smart Homes and Health Telematics , Montreal, Canada , 2011
Liste des auteurs : Ahmed Wehbi, Manolo Dulva Hina, Amar Ramdane-Cherif, Chakib Tadj and Atef Zaguia
Abstract
This paper presents a design of an architecture that facilitates the work of a fusion engine. The logical combination or merging of input streams invoked by the fusion engine is based upon the definition of a set of patterns and its similarity with previously collected data from various modalities. Previous fusion engine designs had many weaknesses, among them their being specialized on a specific domain of application. The proposed architecture addresses such weakness and provides additional features, namely its ability to handle large number of modalities and due to its using knowledge base and standardization characteristics; it becomes suitable to various types of multimodal systems. The techniques used to achieve these features are discussed in this paper.
« Infrastructure of a Context Adaptive and Pervasive Multimodal Multimedia Computing System »
par Manolo HINA
Focus on Information, Intelligence and Knowledge , vol. 1 , Chapter 5, "Infrastructure of a Context Adaptive and Pervasive Multimodal Multimedia Computing System" , pages 99-126 , 2011
Liste des auteurs : Manolo Dulva Hina, Amar Ramdane-Cherif, Chakib Tadj and Nicole Levy
« A multi-agent based multimodal system adaptive to the user’s interaction context »
par Manolo HINA
Multi-Agent Systems - Modeling, Interactions, Simulations and Case Studies , Chapter 2, "A multi-agent based multimodal system adaptive to the user’s interaction context" , pages 29-56 , 2011
Liste des auteurs : Manolo Dulva Hina, Chakib Tadj, Amar Ramdane-Chérif, Nicole Levy
Abstract
Communication is an important aspect of human life; it is with communication that helps human beings connect with each other as individuals and as independent groups. In informatics, the very purpose of the existence of computer is information dissemination – to be able to send and receive information. Humans are quite successful in conveying ideas with one another and reacting appropriately because we share the richness of our language, have a common understanding of how things work and have an implicit understanding of everyday situations. When human communicate with human, they comprehend the information that is apparent to the current situation, or context, hence increasing the conversational bandwidth. This ability to convey ideas, however, does not transfer when human interacts with computer. On its own, computers do not understand our language, do not understand how the world works and cannot sense information about the current situation. In a typical impoverished computing set-up where providing computer with information is through the use of mouse, keyboard and screen, the result is we explicitly provide information to computers, producing an effect that is contrary to the promise of transparency and calm technology in Marc Weiser’s vision of ubiquitous computing. To reverse this, it is imperative that methodologies are developed that will enable computers to have access to context. It is through context-awareness that we can increase the richness of communication in human-computer interaction, through which we can reap the most likely benefit of more useful computational services.
« Le paradigme d’un système multimodal multimédia ubiquitaire sensible au contexte d’interaction »
par Manolo HINA
Thèse doctorale , Université du Québec, Ecole de technologie supérieure, Canada | Université de Versailles St-Quentin-en-Yvelines, France , 2010
Liste des auteurs : Manolo Dulva Hina
Abstract
Le contexte évolue dans le temps et l'acquisition de l'information contextuelle est essentielle. Nous tenons que le contenu de l'information contextuelle ne devrait être défini que par l'utilisateur. Ceci nous mène au concept de l'acquisition incrémental du contexte où des paramètres de contexte sont ajoutés, modifiés ou supprimés, un paramètre de contexte à la fois. Dans ce même ordre d'idée, nous élargissons la notion du contexte au contexte de l'interaction (CI). Le CI qui est employé pour se rapporter au contexte collectif de l'utilisateur, de son milieu de travail et de son système de calcul. Afin de maximiser les bienfaits de la richesse du CI dans la communication personne-machine, la multimodalité tient compte d'un éventail de modes et de formes de communication, choisis et adaptés au contexte de l'utilisateur. La multimodalité ubiquitaire et adaptative enrichit la communication dans l'interaction personne-machine et fournit les modes les plus appropriés pour l'entrée / la sortie de données par rapport à l'évolution du CI. Cette thèse - Le paradigme du système multimodal multimédia ubiquitaire sensible au contexte de l'itération - est une conception architecturale qui montre l'adaptibilité au CI. Il est intelligent et diffus. Il est conçu avec deux buts à l'esprit. D'abord, étant donné une instance de CI qui évolue avec le temps, notre système détermine les modalités optimales qui s'adaptent à un tel CI. En second lieu, nous avons conçu un mécanisme qui permet le choix automatique des applications de l'utilisateur, les fournisseurs préférés à ces applications et les configurations préférées de la qualité du service de ces fournisseurs.
« Autonomic Communication in Pervasive Multimodal Multimedia Computing System »
par Manolo HINA
Autonomic Communication , Chapter 10, "Autonomic Communication in Pervasive Multimodal Multimedia Computing System" , pages 251-283 , 2010
Liste des auteurs : Manolo Dulva Hina, Chakib Tadj, Amar Ramdane-Chérif, Nicole Levy
Abstract
Autonomic communication in a computing system analyzes the individual system element as it is affected by and affects other elements. In the human-machine communication aspect, the autonomic system performs its services autonomously, adjusting the behavior of its services to suit what the user might request implicitly or explicitly. The pervasive multimodal multimedia computing system aims at realizing anytime, anywhere computing. Its autonomic communication includes the protocols that selects, on behalf of the user, the modalities and media devices that are appropriate to the given interaction context. The modalities are the modes of interaction (i.e. for data input and output) between the user and the computer while the media devices are the physical devices that are used to support the chosen modalities. The interaction context itself is the combined user, environment, and system contexts. In this paper, we present the autonomic communication protocols involved in the detection of interaction context and the multimodal computing system’s corresponding adaptation. The heart of this paradigm’s design is the machine learning’s knowledge acquisition and the use of the layered virtual machine for definition and detection of interaction context.
« Interaction Context-Aware Modalities and Multimodal Fusion for Accessing Web Services »
par Manolo HINA
Ubiquitous Computing and Communication Journal , vol. 5, no. 4 , 2010
Liste des auteurs : Atef Zaguia, Manolo Dulva Hina, Chakib Tadj and Amar Ramdane-Cherif
Abstract
These days, a human-controlled multimodal system equipped with multimodal interfaces is possible, allowing for a more natural and more efficient interaction between man and machine. In such a system, users can take advantage of the modalities to communicate or exchange information with applications. The use of multimodal applications, integrated with natural modalities, is an effective solution for users who would like to access ubiquitous applications such as web services. The novelty of this work is that all modalities that are made available to the user to access web services are already found to be suitable to the user’s current situation. By suitablity, we mean these are optimal modalities – found to be suitable to the user’s interaction context (i.e. the combined context of the user, his environment and his computing system) and media devices are available to support them. These modalities can be invoked for the data input/output by the user to access web service using a semantic combination of modalities, called “multimodal fusion”. While current state-of-the-art uses two (on rare cases, three) predefined modalities, our approach allows an unlimited number of concurrent modalities. This approach gives user some flexibility to use the modalities that he sees fit for his situation and comfortable with it. The description of the detection of optimal modality as well as the fusion process, together with sample application are presented in this paper.
« Using Multimodal Fusion in Accessing Web Services »
par Manolo HINA
Journal of Emerging Trends in Computing and Information Sciences , vol. 1, no. 2 , 2010
Liste des auteurs : Atef Zaguia, Manolo Dulva Hina, Chakib Tadj and Amar Ramdane-Cherif
Abstract
In our days, the technology allows us to produce extended and totally human-controlled multimodal systems. These systems are equipped with multimodal interfaces allowing a more natural and more efficient interaction between man and machine. End users could take advantage of natural modalities (e.g. eye gaze, speech, gesture, etc.) to communicate or exchange information with applications. The use of multimodal applications in web services, integrated with natural modalities, is an effective solution for users who cannot use a keyboard or a mouse, on users who have visual handicap, on mobile users equipped with wireless telephone/mobile devices, on weakened users, etc. Our work presents an approach in which various input modalities (speech, touch screen, keyboard, eye gaze, etc.) are available at user’s disposition in order to access web services. While current state-of-the-art uses two (on rare cases, three) pre-defined modalities, our approach allows an unlimited number of concurrent modalities using semantic level called “multimodal fusion”. Such approach gives user the flexibility to use the modalities as he sees fit for his situation. The detailed description of the proposed approach as well as the application that has been developed that uses these multimodalities is presented in this paper.