R2M2 - Robot-Robot-Human-Mimic (BPSE WS18)
Bachelor PSE WS18/19 compact: R2M2 - Robot-Robot-Human-Mimic
Project description
Current robotic systems are becoming increasingly intelligent. The embedded systems used for this purpose have significantly better technical capabilities, so that more abstract programming of complex motion control combined with a variety of sensory capabilities can be achieved. The interaction between humans and robots as well as between robots themselves is therefore increasingly coming to the fore as a field of experimentation for future applications.
Human-robot interaction
As part of a measure to improve the quality of teaching, various robot systems are available with which we now want to create and test a field of interaction between robots and humans. This will allow us to compare the different characteristics and abilities of robots with each other. In particular, this also provides the opportunity to test the interaction of robots with each other and to control them using human gestures and facial expressions.
The robots
We have robots with very different characteristics at our disposal. With two humanoid NAO robot systems, which are already equipped with a large number of sensors (including two HD cameras), we can test the interaction with humanoid systems. We also have four Sphero robot balls and four Cozmo AI robots as small robot systems that can be controlled via Bluetooth.
Recognising gestures and facial expressions
The idea is to use additional stationary cameras or cameras mounted on the robots with wireless transmission to record the view from the robot's perspective or from another position and then process the moving images in an external computer. This allows us to use and test different algorithms for recognising gestures and facial expressions. The NAO systems themselves already have integrated cameras whose views can be processed by the integrated embedded system.
As a system project
In a kick-off meeting at the beginning, we will discuss different scenarios together in order to define the scenarios to be developed. During the project phase, selected procedures from SCRUM will be used to support you in the implementation and with which you will organise the collaboration as a project team. Furthermore, TES Electronic Solutions GmbH will support the project as an industrial partner, providing us with guidance on application-related issues and the usability of TES software. At the end, you will demonstrate the results of your project in a final presentation.
From the creative idea to the project
At the beginning, you can contribute your creative ideas about scenarios to be developed. From this, a project with a fixed project duration should be defined. Are the ideas and concepts realisable within the project duration? As a team, you define your project goal based on this. We then take on the role of the client who wants the project objective to be realised. As is usual with projects in practice, you will probably recognise the need for changes to the project objectives during the course of the project and can practise how to deal with them. It would be particularly nice to have a demonstration as a result, which we can set up again at a suitable opportunity and present to visitors to our faculty, for example.
Previous knowledge
Good programming skills, basic knowledge of graphical data processing, basic courses in computer science and software development for embedded systems are helpful.
We look forward to an exciting project with you!
The dates
Block event at the end of the lecture-free period in September
Kick-off event: 6.9.
Project phase: 10-21 Sept.
Final presentation: 26.9.
Lecturers of the system project
Prof. Dr Jens-Peter Akelbein - Head of the Embedded Systems Technology (EST) laboratory
Björn Frömmer - Lecturer in Computer Graphics, Image Processing, Natural User Interfaces, Virtual Reality
supported by
Mario Hoss - research assistant
Bettina Kurz-Kalweit - research assistant and laboratory engineer EST
Results groups 1 + 2
Group targets with humanoid robot NAO and Sphero SPRK+
- NAO recognises, locates and tracks Sphero
- NAO recognises Sphero's colours and gestures...
- ... and performs various actions based on these
Group 1 has the additional requirement that NAO's own camera cannot be used.
Group 3 result
The aim of Group 3 of the R2M2 project is to control the Sphero SPRK+ using an iOS augmented reality app. The Sphero SPRK+ should move to a place in the room after a touch gesture in the iOS app. The augmented reality feature is achieved using Apple's ARKit. This ARKit offers the possibility to create a coordinate system with planes and to recognise/track objects.
Group 4 result
Anki Cozmo as a courier
The idea is to simulate a town in which parcels have to be delivered. For this purpose, certain places in the city are marked as houses and one place is designated as the parcel depot. The robot's task is then to deliver the parcels to the correct house.
The assignment of parcels to houses is implemented using facial recognition: photos with faces are attached to both the cubes (the parcels) and the houses.
It turned out that it is not necessary to stick the faces on the cubes, but that the robot memorises the faces and can permanently assign them to a specific cube (parcel).