Rüdiger Dillmann

What can a Robot learn from Human Demonstration?

Humanoid robot systems are designed to interact with humans in terms of cooperation and conversation about the task to be done and how to do the task and finally how to execute it. In addition such robots should act goal-oriented and be capable to react on disturbances or unexpected events in a competent manner. Such robot systems should  be able to operate in dynamic human centered scenarios which require capabilities such as adaptivity, perception, categorisation, action and learning. Examples for such systems are service robots or humanoids that interact in the immediate environment of humans in a context dependand goal-oriented manner and cooperate with humans. The behavior of such robots is characterized by active sensing processes, fusion of sensor data, perception, interpretaion and the selection of appropriate actions as well as their superimposed control systems. Thereby learning and shaping of sensory and motoric abilities as well as the active observation and interpretation of situations and actions are of major interest. 

 A probate approach for learning knowledge about actions and sensomotory abilities is to acquire examples of human actions by observation of the actions with sensors and to transfer these abilities in the sense of learning by demonstration to the robot. This requires human motion capture, observation of interaction , object state transitions and observation of spatial and physical relations between objects. By doing this, it is possible to acquire so-called "skills", situative knowledge as well as task knowledge, and can be introduced to new and unknown tasks. New terms, new objects and situations, even new types of motion can be learned with the help of a human  teacher or be corrected interactively via multimodal channels. The term of "multimodality" describes communication channels which are intuitive for humans, such as language, gesture and haptics (physical human-robot contact). These are to be used for commanding and instructing the robot system. 

 The field of programming by demonstration has been evolved strongly as a response to the needs of generating flexible programs for service robots and is largely driven by attempts of modeling human behaviour and to map it onto virtual Androids or humanoid robots. It comprises a broad set of observation techniques processing large sets of data from high speed camera systems, laser, data-gloves and even exosceleton devices. Some operate with precise a- priori models other use probabilistic approaches to approximate human behaviour. In any case observation is done to identify motion over space and time, interaction with the environment and ist effects, usefull regularities or structures and ist interpretation in a given context. With this goal, systems have been developed which combine active sensing, computational learning techniques, multimodal dialogues to enrich the semantic system level, memorisation techniques as well as mapping strategies to make use of the learned knowledge to control a real robot. One important paradigm is that objects and action representations cannot be separated and form the building blocks for cognitive robot systems. Thus, so called object-action complexes -OACS- can be derived to unify different sensor, actuator and object representations including language and allow the robot to understand ist environment (EU-6 .framework IP PACO-PLUS). 

 In the context of the SFB 588 "Humanoid Robots" a mobile two-arm system with dexterous five-finger hands, a flexible torso as well as a head with visual and acoustic perceptive capabilities has been developed, which appears to behave like a human. The locomotion system and the behaviour of the robot as well as the multimodal interaction are designed to cope with humans. For human-robot cooperation it is important for the robot to acquire the aim of humans, to remember already corporately accomplished actions and to apply this knowledge in each individual case in the correct way. The status of the Humanoid Robot Project is outlined and the achieved results are discussed.