A system that tracks eye movements and gaze patterns of pilots-in-training has been installed in No. 79 Squadron’s Hawk 127 Lead-In-Fighter simulator at RAAF Base Pearce, courtesy of Canberra-based company Seeing Machines.
The technology uses IR lights and a camera to track where trainees are looking and for how long, allowing instructors to gain an insight into how trainees are thinking. It emerged through a commercial investment from Plan Jericho and Defence reportedly wants the technology implemented across all Hawk simulators and ‘the rest of the flying training system.’
“Our engagement with RAAF was extended to a two-year contract and enabled full visibility of the capability of the technology and the utility of the output data and visualisations to all or most of the flying training units across the ADF,” Patrick Nolan, General Manager of Aviation at Seeing Machines, said to ADM.
One of the challenges of installing the system was ensuring that the addition of new equipment did not disturb the mature training system or distract new pilots.
“This involved some creative 3D printing, and efficient collaboration outside of simulator sessions,” Nolan said.
Seeing Machines worked closely with 78 Wing, 76 and 79 Squadrons, TFSPO, the Air Warfare Centre and the Institute of Aviation Medicine, as well as the simulator OEM, CAE. When travel to WA was restricted by Covid-19, Seeing Machines quickly adapted to video conferencing and secure data transfers to upgrade the system.
“We were fortunate to have strong partners in CAE to enable us to achieve this in line with Defence requirements,” Nolan said.
For Seeing Machines, the main takeaway from their success has been understanding the importance of integrating new technology into the military training environment.
“The overarching lessons are not purely about the tech or the signals, but rather more about ensuring that the capability integrates into the training environment and can be seamlessly incorporated into the Instructor workflow,” Nolan said.
Seeing Machines is also looking to build on this success to grow their footprint in aviation and look to possible international opportunities.
“The technology is the same but the use case is slightly different as we change the focus from precise gaze tracking in the training environment to fatigue and attention in the cockpit,” Nolan said. “Similar to our Guardian Fleet and Backup driver Monitoring System solutions for freight and automotive respectively, we’re working on delivering an airborne solution that supports pilots from fatigue and incapacitation.
“We also continuously invest in developing features and applications that rely on a range of signals collected through eye, head, and face-tracking; gaze and attention being the more obvious ones, but also fatigue, distraction, operator condition, and cognitive workload signals.”