Students at Eindhoven University of Technology (TU/e) in the Netherlands have developed the world’s first interactive drone that can transmit emotions aimed at reassuring residents when guiding them to safety in the event of a fire.
TU/e student team Blue Jay has introduced an interactive drone at an event in the Evoluon conference centre in Eindhoven that guides elderly people to the exit during a fire in an aged care home, even before the fire brigade arrives.
The Blue Jay Aeden is believed to be the first interactive drone in the world that can transmit emotions and can fulfil an important function in helping save people’s lives.
On average, aged care homes in the Netherlands have to be evacuated about 600 times a year because of a fire risk.
“During an evacuation, elderly people often do not know what to do, which makes them anxious and can sometimes cause them to panic,” Paul van Kuppevelt, team manager of Blue Jay, expains.
“Because they panic, they often cannot find their own way out, which can lead to dangerous situations and even death.”
In 2020 alone, 25 people died as a result of these situations.
To prevent this, Blue Jay has developed the first drone that can reassure the elderly and help them find the right route.
Programmed to see emotions
The drone contains a small camera that is programmed to see and understand the emotions of the elderly.
If a resident looks happy or surprised, the drone will register this emotion as such.
A Convolutional Neural Network (CNN) is used for this, a form of deep learning that enables the drone to recognise emotions by analysing a large amount of visual images.
The drone can recognise five different emotions (angry, happy, sad, neutral, surprised) with an accuracy of 94 per cent.
After a clear emotional state is detected, Blue Jay’s invention will react by showing an emotion itself.
This makes the drone appear human-like, with the aim being that the resident feels more at ease and is more likely to trust and follow the drone.
According to Van Kuppevelt, this works best if there is a natural response to the resident’s emotion.
“For example, if a resident looks happy, it is obviously important that the drone looks happy in return, and not angry or sad.”
In order to make the drone show the appropriate emotions, the Blue Jay team worked together with a group of elderly people from the Joris Zorg nursing home in Oirschot, slightly north west of Eindhoven.
Via several tests in the nursing home, it was possible to find out how the elderly and the staff react to the presence and the emotions shown by the drone.
Live watch at fire station
The fire brigade can also use a live stream.
Through this stream, firefighters on their way to the location can watch via an app on their phones what the drone sees at that moment.
“This way, firefighters know exactly what is happening in the nursing home, so they can be better prepared,” Van Kuppevelt says.
“This saves precious minutes, which means more lives can be saved.”
A spokesperson for the fire brigade said they’re delighted with the innovation and are looking forward to eventually being able to use the drone.
When that will be will depend on a number of factors, including Dutch laws and regulations.
For example, the drone is not yet permitted to fly in public areas.
The drone is at present undergoing further development, including solutions that will ultimately allow the drone to fly completely autonomously.
The team is also looking at possibilities to ensure that the drone is not obstructed by closed or locked doors.
Paul van Dooren, Innovation Advisor at Safety Region Brabant Zuidoost, says the interactive part of the drone ensures that it can provide added value to the existing resources.
“Resources such as fire alarms do give a signal, but cannot offer specific help to every person in need.
“The fact that the drone can now recognise emotions is a good step towards making this possible,” van Dooren says.
“Hopefully in the future we can develop this into a full analysis of the situation, for example through speech or object detection.”