Defence is investing $5 million towards simulation technology that will improve outcomes for troops in combat situations.
Software engineering and systems integrations company elmTEK has been engaged by Defence to deliver 'Human-in-the-Loop' simulation technology under Phase 2 of Defence’s Joint Project 500 (JP500).
Human-in-the-Loop is a type of simulation that immerses a human operator into a conflict environment and collects data on how they interact and respond to different operational scenarios using Emulated Military Equipment.
The scenarios enable equipment to be tested for efficacy, and tactical and countermeasure approaches to be optimised, resulting in better real-life outcomes for troops.
elmTEK Director of Strategy, Bjorn Wharff, said the company’s Human-in-the-Loop simulator technology was the most advanced of its kind, with 'cutting-edge precision and fidelity' improving the accuracy and usefulness of the data collected.
The simulator consists of a 270-degree immersive dome, coupled with visual and audio scene rendering technology expected to deliver highly realistic effects for the human operators.
Wharff said the enclosure is capable of creating life-like physical effects such as vibration, weapon kickback and realistic battlefield sounds.
“We want to measure and assess true behaviours, so operators must be absolutely convinced of the scene realism and be ‘immersed’ in the scenarios,” Wharff said.
“When the operator pulls the trigger on a weapon, they need to feel the right resistance and experience an immediate response. If there’s even a 10-millisecond delay in sound, feel or sight when the trigger is pulled, the operator can notice and start to change their behaviour, either consciously or sub-consciously.
“Latency and correlation are critical things to get right to ensure the experience is compelling, so that we can get the most insight from the data.”
Wharff said the technology used in the project has been further bolstered by the recent merger between elmTEK and Consilium Technology, who bring specific simulation technology and extensive artificial intelligence expertise to the fold.
“The scene generator that is driving the visual experience for operators is Infinite Studio, a Consilium product. We are also exploring how their AI can be used to deliver high order insights in the areas of operator and system effectiveness for this and future projects of this nature.
“Through Infinite Studio we get cutting-edge visuals and graphics, best-of-breed creation tools, multi-spectral rendering and real-time performance. The result is world-leading real-time simulation.”
To assess behaviours within the simulation, elmTEK will track eye movement, the interactions of both the operator and equipment, their combined capabilities and vulnerabilities, and operator biometrics.
“Operators will be fitted with clear glasses with a camera looking forward and also back onto the pupils, which will be used to track metrics on what each pupil is doing.
“We can draw inferences about operator workload, stress and decision-making by monitoring eye movement, alongside other measures. Specific eye motion metrics provide a link to cognitive and emotional processing. By using these techniques in the way that we are, we can characterise operator behaviours and provide insights into how someone in the field might behave in certain scenarios.”
The results will enable Defence to develop and validate Tactics, Techniques and Procedures (TTPs), as well as alternative countermeasures in combat situations.
“If we can better understand how and when an operator will use a weapon, and how that weapon performs in those circumstances, we can improve the training protocols to account for those behaviours and improve outcomes in a combat situation,” Wharff said.
“For example, using the simulator we will be able to provide high confidence statistics around the effectiveness of both weapon and countermeasure effectiveness for given scenarios.”
“It’s a really powerful tool in developing and deepening the sovereign capabilities around technology, equipment and people for the Australian Defence Force.”
The Human-in-the-Loop simulator is currently in its second phase of development, with the build expected to be completed in August 2023.