thedrifter
09-30-07, 05:28 PM
Immersive experience
Live, virtual training combine to create 3-D world
By Gidget Fuentes - gfuentes@militarytimes.com
Posted : October 08, 2007
CAMP PENDLETON, Calif. — The days when Marines trained for battle with combat video games may be over, now that advanced technologies are propelling military simulators into the three-dimensional realm of “immersive training.”
In November, the Marine Corps and the Office of Naval Research plan to unveil a $1.3 million prototype Infantry Immersive Trainer at Camp Pendleton, Calif. A second simulator is planned for the Marine Expeditionary Rifle Integration Facility at Quantico, Va.
“We show what the art of the possible is,” said Navy Cmdr. Dylan Schmorrow, a program officer who is helping lead the project for ONR. “You’re able to operate temporally through three-dimensional space.”
Immersive training, a relatively new term, means a combination of live and virtual training, and eventually constructive, or networked, aspects as well.
Announced in April, the project incorporates several technologies developed by ONR, which has spent nearly $75 million and plans to spend another $50 million in the next five years in human performance, training and education programs. The Institute for Creative Technologies at the University of Southern California, meanwhile, has developed FlatWorld, which combines sounds and high-resolution imagery projected on digital flat-screen displays with real physical objects, movable props and reconfigurable rooms and buildings.
It’s a far cry from a decade ago, when the Marine Corps turned to “Marine Doom,” a military version of the popular two-dimensional personal-computer-based game, to help sharpen the battlefield tactics of its troops and fire teams.
A leap ahead
Fire teams or squads will participate within a 3-D video game of urban-battlefield streets in life-size combat driven by video-game simulations and interactive technologies that are more realistic and adaptable and easily incorporated into training facilities. A rifle squad could practice closer to home base while operating virtually in simulated play through a key firefight during the 2004 battle of Fallujah, for example.
The Infantry Immersive Trainer is being developed in a 30,000-square-foot former tomato-packing warehouse near Camp Pendleton’s northern infantry camp, San Mateo, Schmorrow said. Once it goes online, it could pave the way for broader uses for the Corps’ version of immersive training.
“We use the term ‘holodeck,’” said Tom Buscemi, director of the I Marine Expeditionary Force Simulation Center at Camp Pendleton, a technology featured in the Star Trek Franchise. “Certainly, technology doesn’t offer that kind of thing right now with holograms and that sort of thing. But we’ve had enough of the 16-inch, 21-inch-monitor flat screen. We need something that is more real, more interactive, more personally involved.”
The immersive trainer grew out of a requirement identified last year by Marine Corps Combat Development Command and Lt. Gen. Jim Mattis, then-commanding general.
“Mattis has been very visionary,” Schmorrow said. “He’s been a good bully pulpit.”
Mattis wanted a system “that was gymnasium-size, that provided for the inoculation, primarily for the guy that hadn’t done it before, into close-quarters battle, and the chaos, the confusion and the ethical problems that they experience,” Buscemi said.
Immersive training would challenge infantrymen ethically through scenarios and help prepare them for the more intense, larger-scale “Mojave Viper” exercises at the Corps’ desert training base in Twentynine Palms, Calif., he said.
Mattis wanted infantry simulations to get the kind of money and attention spent on flight simulators, and he wanted more than the traditional static trainers, such as indoor marksmanship trainers.
“People had asked, ‘Why aren’t we doing this with the infantry, for the Marine Corps?’” Schmorrow said.
Mattis wanted a trainer that didn’t tie up Marines with wires and extra gadgets, that allowed them to wear their standard combat gear and carry and operate their weapons.
“He’s not going to be hooked and wired with ... gear,” Buscemi said.
About 1,000 acres will be developed into a larger training area so a platoon could get off a helicopter and assault or operate in the simulated town, he said.
The new simulator will be produced by ONR, the Corps’ Training and Education Command and its training systems program office.
The trainer will showcase emerging virtual technology with live training environments. An initial conceptual drawing of the Infantry Immersive Trainer prototype shows a series of buildings with courtyards, interior rooms and multiple levels. The street would be wide enough to operate tactical vehicles such as Humvees.
Members of a fire team or squad will dismount and patrol with their standard weapons and communications and combat gear, searching shops, homes and other buildings — some with virtual interactive life-sized images projected on walls. Rooms and buildings will be tailored and fashioned to a specific combined-arms operating or combat environment.
Buscemi said he was stunned when the Institute for Creative Technologies showed off a countersniper trainer with life-size images projected on flat screens.
“You opened a door and there was an insurgent there who fired. And he fired, and bullet holes appeared on the other wall. I jumped through my skin when I saw that,” said Buscemi, a Vietnam veteran and helicopter pilot.
“How they do it was not too technically difficult,” he said. “The realism, the size definitely got your attention, because you walked into a room that had furniture strewn about from some combat action and you didn’t know what to expect.”
The concept will use live people and things: role-playing civilian villagers and insurgent fighters as well as furnishings, doors and stairwells. Those will be combined with virtual effects — explosions, sounds and smoke — and interactive flat-screen projected images that show, for example, images of a room’s interior or a fighter standing by a doorway. All eventually would be tied into a constructive play that can be tailored toward particular missions, roles and environments.
“It’s going to be able to leverage all that underlying simulation technology ... so they’re in the game,” Schmorrow said. “One of the biggest bangs for the buck is we are better able to prepare people to have that live experience. It’s even more effective training.”
The training will stretch beyond shoot/don’t-shoot scenarios; they will test infantry squads mentally as well as tactically and physically. A fire team, for example, could burst its way into a clinic and be confronted with loud noises and a foreign-speaking medic.
“Maybe that’s a virtual projection in the room. Do you shoot them? Do you communicate with him?” he said. “These afford us very complex and cognitive training platforms.”
By tapping into existing training aids and simulation technologies to train Marines to operate and handle stressful, chaotic situations well before they reach a real combat zone, the Corps is increasing the chances those Marines will get through their first combat experiences, whether an enemy attack, ambush or roadside bomb.
“A lot of studies have shown if you survive your first firefight, then the likelihood of you surviving the rest of your tour is getting increased,” Schmorrow said.
Cubic Corp. is also working on a live-virtual-constructive prototype for an immersive training system, said John Lewis and John Schmader, retired Army infantry officers who work for Cubic’s Training and Education Division in Leavenworth, Kan.
“We’ve got the constructive piece finished, the live [piece] is in and we’re working now to bring the virtual piece all together,” Schmader said.
Can a Marine or soldier step into the play and participate in an event in, say, the battle of Fallujah — and, more importantly, do his part to drive the progress of the event — with realistic and accurate reactions and feedback while fully intertwined with his squad and other “players” in that virtual environment? Taking training beyond the 3-D environment into 4-D immersive training is doable, Lewis and Schmader said, but may take some time to perfect.
“The real challenge now is to take multiple players and put them in a realistic environment so they interact,” said Schmader, a retired brigadier general who served as deputy commanding general of the Army’s Combined Arms Center at the U.S. Army Training and Doctrine Command at Fort Leavenworth.
“We are on the verge ... of programs that [place] a person into a scenario and environment that’s realistic, and everybody comes into a game and actually plays it out,” he said. “There’s a lot of people out there that have the individual pieces.”
Existing technologies can meet 80 percent to 90 percent of the current needs. “What we’re looking at is, ‘What is the availability of the technology that is in place today [to] put it into a package that is user-friendly and training friendly,’” Lewis said.
Cubic’s prototype will involve no proprietary software, Schmader said.
“We’re trying to use software that has commercial applications” so users can alter the product to meet their specific needs, he said.
Part of the challenge is taking real events — for example, Fallujah in all its house-by-house fighting intensities — and tying that into a virtual storyline.
“You turn a lesson learned into a story. You make that story digital, and then you make it interactive digital. You turn it into a real immersive type of learning,” Schmader said.
“It’s a systems approach to training. It’s not just buying a game. Games are just one component of immersive training,” Lewis said. “Ultimately, what you’re trying to do is enable for the training that’s needed — the learning that’s needed — to be delivered 24/7.”
Schmader said immersive trainers will do to individual squad-level training what high-tech simulators such as the Battle Command Training program have done for battle staff training and planning. “We want the individuals to determine the outcome via the game,” he said.
“My goal is to go out to the live-fire range and for two guys to get into a fistfight,” he said. “We want to really get these young soldiers involved in their own learning.”
With an immersive training system, the “walk-crawl-run” mind-set of infantry training would start with building individual skills and small-unit competencies at the fire team and squad levels with systems that support home-station training and portable adaptable systems to support deploying units. Then, he said, you work up the levels of command to support small-unit and command-level training across a larger battle space.
“What you’re looking for is a solution that provides an enabling training tool for that crawl stage,” Lewis said. An immersive training system may enable such things as podcasting a training support package in a gaming format that tests a fire team’s decision-making, for example.
At the walk stage, for example, a squad could come together using 3-D games, a Virtual Battlefield System or perhaps 4-D holographic images to support team learning, he said. It could support the “run” piece by synchronizing an LVC environment and team at a higher level of command and control.
Lewis is optimistic about what immersive technologies will represent to today’s infantry training programs.
“If you build it, they will come, because that’s the way they learn today,” he said. “It’s a shift in how we learn, developed by the operational commanders and driven by the tactical guys.”
Schmorrow, who has worked for the Defense Advanced Research Projects Agency, said it may take three to five years for immersive training technologies to mature.
Existing technologies already provide high-resolution graphics and interactive simulations that help keep some video games as popular training tools. Among those is the Close Combat series of video games, created by Atomic Games, which includes the 3-D tactical shooter game “Close Combat: First to Fight,” designed for infantry fire teams.
“For years, when a company developed a virtual training system for a military customer, it developed its technology from scratch, and it did not update them very quickly,” said Peter Tamte, president of Destineer, a developer of computer and video games. Users in time quickly noticed the lag in outdated images and lower-quality graphics.
But military simulation has caught up with the gaming world.
The emerging simulation systems are benefiting from higher quality and lower costs, and with the advantage of injecting more realism with the help of combat veterans and other experts, he said, “we can use them to make our commercial video games more authentic.”
Tamte said military immersive training systems and the video-gaming industry will benefit from technological advances, including improved graphics that can help drive decision-making, networking that allows more users to train together either on a local area network or on the Internet, and artificial intelligence that can play the role of friendly or opposing forces.
“It allows users, the students, to train multiple times in environments they might not be able to train in in the real world,” he said. “It’s repetition that really builds instincts.”
Ellie
Live, virtual training combine to create 3-D world
By Gidget Fuentes - gfuentes@militarytimes.com
Posted : October 08, 2007
CAMP PENDLETON, Calif. — The days when Marines trained for battle with combat video games may be over, now that advanced technologies are propelling military simulators into the three-dimensional realm of “immersive training.”
In November, the Marine Corps and the Office of Naval Research plan to unveil a $1.3 million prototype Infantry Immersive Trainer at Camp Pendleton, Calif. A second simulator is planned for the Marine Expeditionary Rifle Integration Facility at Quantico, Va.
“We show what the art of the possible is,” said Navy Cmdr. Dylan Schmorrow, a program officer who is helping lead the project for ONR. “You’re able to operate temporally through three-dimensional space.”
Immersive training, a relatively new term, means a combination of live and virtual training, and eventually constructive, or networked, aspects as well.
Announced in April, the project incorporates several technologies developed by ONR, which has spent nearly $75 million and plans to spend another $50 million in the next five years in human performance, training and education programs. The Institute for Creative Technologies at the University of Southern California, meanwhile, has developed FlatWorld, which combines sounds and high-resolution imagery projected on digital flat-screen displays with real physical objects, movable props and reconfigurable rooms and buildings.
It’s a far cry from a decade ago, when the Marine Corps turned to “Marine Doom,” a military version of the popular two-dimensional personal-computer-based game, to help sharpen the battlefield tactics of its troops and fire teams.
A leap ahead
Fire teams or squads will participate within a 3-D video game of urban-battlefield streets in life-size combat driven by video-game simulations and interactive technologies that are more realistic and adaptable and easily incorporated into training facilities. A rifle squad could practice closer to home base while operating virtually in simulated play through a key firefight during the 2004 battle of Fallujah, for example.
The Infantry Immersive Trainer is being developed in a 30,000-square-foot former tomato-packing warehouse near Camp Pendleton’s northern infantry camp, San Mateo, Schmorrow said. Once it goes online, it could pave the way for broader uses for the Corps’ version of immersive training.
“We use the term ‘holodeck,’” said Tom Buscemi, director of the I Marine Expeditionary Force Simulation Center at Camp Pendleton, a technology featured in the Star Trek Franchise. “Certainly, technology doesn’t offer that kind of thing right now with holograms and that sort of thing. But we’ve had enough of the 16-inch, 21-inch-monitor flat screen. We need something that is more real, more interactive, more personally involved.”
The immersive trainer grew out of a requirement identified last year by Marine Corps Combat Development Command and Lt. Gen. Jim Mattis, then-commanding general.
“Mattis has been very visionary,” Schmorrow said. “He’s been a good bully pulpit.”
Mattis wanted a system “that was gymnasium-size, that provided for the inoculation, primarily for the guy that hadn’t done it before, into close-quarters battle, and the chaos, the confusion and the ethical problems that they experience,” Buscemi said.
Immersive training would challenge infantrymen ethically through scenarios and help prepare them for the more intense, larger-scale “Mojave Viper” exercises at the Corps’ desert training base in Twentynine Palms, Calif., he said.
Mattis wanted infantry simulations to get the kind of money and attention spent on flight simulators, and he wanted more than the traditional static trainers, such as indoor marksmanship trainers.
“People had asked, ‘Why aren’t we doing this with the infantry, for the Marine Corps?’” Schmorrow said.
Mattis wanted a trainer that didn’t tie up Marines with wires and extra gadgets, that allowed them to wear their standard combat gear and carry and operate their weapons.
“He’s not going to be hooked and wired with ... gear,” Buscemi said.
About 1,000 acres will be developed into a larger training area so a platoon could get off a helicopter and assault or operate in the simulated town, he said.
The new simulator will be produced by ONR, the Corps’ Training and Education Command and its training systems program office.
The trainer will showcase emerging virtual technology with live training environments. An initial conceptual drawing of the Infantry Immersive Trainer prototype shows a series of buildings with courtyards, interior rooms and multiple levels. The street would be wide enough to operate tactical vehicles such as Humvees.
Members of a fire team or squad will dismount and patrol with their standard weapons and communications and combat gear, searching shops, homes and other buildings — some with virtual interactive life-sized images projected on walls. Rooms and buildings will be tailored and fashioned to a specific combined-arms operating or combat environment.
Buscemi said he was stunned when the Institute for Creative Technologies showed off a countersniper trainer with life-size images projected on flat screens.
“You opened a door and there was an insurgent there who fired. And he fired, and bullet holes appeared on the other wall. I jumped through my skin when I saw that,” said Buscemi, a Vietnam veteran and helicopter pilot.
“How they do it was not too technically difficult,” he said. “The realism, the size definitely got your attention, because you walked into a room that had furniture strewn about from some combat action and you didn’t know what to expect.”
The concept will use live people and things: role-playing civilian villagers and insurgent fighters as well as furnishings, doors and stairwells. Those will be combined with virtual effects — explosions, sounds and smoke — and interactive flat-screen projected images that show, for example, images of a room’s interior or a fighter standing by a doorway. All eventually would be tied into a constructive play that can be tailored toward particular missions, roles and environments.
“It’s going to be able to leverage all that underlying simulation technology ... so they’re in the game,” Schmorrow said. “One of the biggest bangs for the buck is we are better able to prepare people to have that live experience. It’s even more effective training.”
The training will stretch beyond shoot/don’t-shoot scenarios; they will test infantry squads mentally as well as tactically and physically. A fire team, for example, could burst its way into a clinic and be confronted with loud noises and a foreign-speaking medic.
“Maybe that’s a virtual projection in the room. Do you shoot them? Do you communicate with him?” he said. “These afford us very complex and cognitive training platforms.”
By tapping into existing training aids and simulation technologies to train Marines to operate and handle stressful, chaotic situations well before they reach a real combat zone, the Corps is increasing the chances those Marines will get through their first combat experiences, whether an enemy attack, ambush or roadside bomb.
“A lot of studies have shown if you survive your first firefight, then the likelihood of you surviving the rest of your tour is getting increased,” Schmorrow said.
Cubic Corp. is also working on a live-virtual-constructive prototype for an immersive training system, said John Lewis and John Schmader, retired Army infantry officers who work for Cubic’s Training and Education Division in Leavenworth, Kan.
“We’ve got the constructive piece finished, the live [piece] is in and we’re working now to bring the virtual piece all together,” Schmader said.
Can a Marine or soldier step into the play and participate in an event in, say, the battle of Fallujah — and, more importantly, do his part to drive the progress of the event — with realistic and accurate reactions and feedback while fully intertwined with his squad and other “players” in that virtual environment? Taking training beyond the 3-D environment into 4-D immersive training is doable, Lewis and Schmader said, but may take some time to perfect.
“The real challenge now is to take multiple players and put them in a realistic environment so they interact,” said Schmader, a retired brigadier general who served as deputy commanding general of the Army’s Combined Arms Center at the U.S. Army Training and Doctrine Command at Fort Leavenworth.
“We are on the verge ... of programs that [place] a person into a scenario and environment that’s realistic, and everybody comes into a game and actually plays it out,” he said. “There’s a lot of people out there that have the individual pieces.”
Existing technologies can meet 80 percent to 90 percent of the current needs. “What we’re looking at is, ‘What is the availability of the technology that is in place today [to] put it into a package that is user-friendly and training friendly,’” Lewis said.
Cubic’s prototype will involve no proprietary software, Schmader said.
“We’re trying to use software that has commercial applications” so users can alter the product to meet their specific needs, he said.
Part of the challenge is taking real events — for example, Fallujah in all its house-by-house fighting intensities — and tying that into a virtual storyline.
“You turn a lesson learned into a story. You make that story digital, and then you make it interactive digital. You turn it into a real immersive type of learning,” Schmader said.
“It’s a systems approach to training. It’s not just buying a game. Games are just one component of immersive training,” Lewis said. “Ultimately, what you’re trying to do is enable for the training that’s needed — the learning that’s needed — to be delivered 24/7.”
Schmader said immersive trainers will do to individual squad-level training what high-tech simulators such as the Battle Command Training program have done for battle staff training and planning. “We want the individuals to determine the outcome via the game,” he said.
“My goal is to go out to the live-fire range and for two guys to get into a fistfight,” he said. “We want to really get these young soldiers involved in their own learning.”
With an immersive training system, the “walk-crawl-run” mind-set of infantry training would start with building individual skills and small-unit competencies at the fire team and squad levels with systems that support home-station training and portable adaptable systems to support deploying units. Then, he said, you work up the levels of command to support small-unit and command-level training across a larger battle space.
“What you’re looking for is a solution that provides an enabling training tool for that crawl stage,” Lewis said. An immersive training system may enable such things as podcasting a training support package in a gaming format that tests a fire team’s decision-making, for example.
At the walk stage, for example, a squad could come together using 3-D games, a Virtual Battlefield System or perhaps 4-D holographic images to support team learning, he said. It could support the “run” piece by synchronizing an LVC environment and team at a higher level of command and control.
Lewis is optimistic about what immersive technologies will represent to today’s infantry training programs.
“If you build it, they will come, because that’s the way they learn today,” he said. “It’s a shift in how we learn, developed by the operational commanders and driven by the tactical guys.”
Schmorrow, who has worked for the Defense Advanced Research Projects Agency, said it may take three to five years for immersive training technologies to mature.
Existing technologies already provide high-resolution graphics and interactive simulations that help keep some video games as popular training tools. Among those is the Close Combat series of video games, created by Atomic Games, which includes the 3-D tactical shooter game “Close Combat: First to Fight,” designed for infantry fire teams.
“For years, when a company developed a virtual training system for a military customer, it developed its technology from scratch, and it did not update them very quickly,” said Peter Tamte, president of Destineer, a developer of computer and video games. Users in time quickly noticed the lag in outdated images and lower-quality graphics.
But military simulation has caught up with the gaming world.
The emerging simulation systems are benefiting from higher quality and lower costs, and with the advantage of injecting more realism with the help of combat veterans and other experts, he said, “we can use them to make our commercial video games more authentic.”
Tamte said military immersive training systems and the video-gaming industry will benefit from technological advances, including improved graphics that can help drive decision-making, networking that allows more users to train together either on a local area network or on the Internet, and artificial intelligence that can play the role of friendly or opposing forces.
“It allows users, the students, to train multiple times in environments they might not be able to train in in the real world,” he said. “It’s repetition that really builds instincts.”
Ellie