Brno scientists develop monitoring system to keep pilots from nodding off
Brno scientists are developing a system for monitoring pilot fatigue or any kind of health problem. Based on a camera with artificial intelligence, the device monitors the pilot's condition and, if necessary, responds with an alert or a more insistent wake-up call.
Long journeys –in the air and on the road-can get tedious and, after a few hours of monotonous activity, it is easy to nod off. According to an investigation by the American Transportation Safety Board, pilot fatigue is a probable cause or at least a contributing factor in nearly one in five aviation incidents. The British Pilots Association recently conducted a survey in which 56 percent of pilots admitted to having fallen asleep on a long flight. 29 percent admitted that, when they woke up, they found that the co-pilot had nodded off as well.
Experts from Honeywell's Brno branch, have developed an AI system aiming to detect the first signs of inattention, sleep or any kind of health problem that may prove to be a safety risk up in the air.
The basis of this safety system is a camera that closely monitors the contours and expression of the pilot’s face. Bohdan Blaha, one of the developers working on the project, explains:
"The dots you see here are detailed contours of the pilot’s face. The system monitors the slightest movement or changes to the face - the turn of the mouth, how the head is tilted, whether the eyes are closed or open, what the frequency of blinking is, or whether the pilot is looking in one direction for a long time without blinking," he says.
If the system detects that the pilot is tired or starting to have a medical problem, it will write a recommendation on the display about what he or she should do. If there is no reaction to the text message a sound message issues an alert or a more insistent wake-up call.
Bohdan Blaha says the whole system is based on artificial intelligence that has learned directly from humans.
"We invited volunteers and, together with experts from the sleep lab, we conducted the respective monitoring tests, monitoring minute changes to facial features during the process of falling asleep, which was recorded on camera as information for the software program".
The software has thus learned to distinguish an alert person from a tired or sleeping person by sight. The researchers then installed the system in various kinds of simulators and then in the first test aircraft.
"The entire system was successfully tested in an Embraer 170 aircraft, including detection of a problem and the subsequent alerting of the pilot," says Bohdan Blaha.
The researchers have already installed the camera in the first commercial aircraft, but without the wake-up call.The system is currently flying in an Airbus 321, but it is only used to collect data. It does not at this point communicate with the pilots in active mode. Before it can be put to use, Honeywell scientists will have to get it certified.
Later, it should be a key part of the Darwin project, which is co-funded by the European Union, and would allow even large transport aircraft to be flown by just one pilot instead of two.
