All aboard! San Diego, California’s premier aircraft carrier museum takes AR development to the next level with “The Battle of Midway AR Extreme.”
Augmented Reality (AR) is a little understood and quite magical technology that has a lot more to offer than most folks in the education industry realize. When I first got into AR in my Learning Design Tech Masters program back in 2016, the research indicated that AR was going mainstream and that the potential for AR as a tool for learning was impressive, but posed many difficult challenges, like device compatibility, availability of models, and overly complex development software. Now, in 2021, the barriers for AR have been broken down, and the opportunities for development have been unlocked with the simplicity and agility of CoSpaces.
When the pandemic of spring 2020 left educators scrambling for distance learning options, CoSpaces became a refreshing break from the assigned readings and writing prompts in my class. As my students were scouring the web for 360° photos to add to their Virtual Reality (VR) projects, I stumbled across some 360° images embedded on the USS Midway Museum’s website. I had attended some of the MIT professional development workshops for history teachers onboard the Midway over the previous summers, so I figured it couldn’t hurt to reach out and see if they would be willing to share their library of files. Just as I was scrambling to rethink the types of learning experiences I could pull off through distance learning in my own courses, the Midway Museum was simultaneously having to rethink what sorts of exhibits they might be able to offer during the pandemic. Indoor restrictions, sanitation and distancing had created new challenges for them as well. This serendipitous encounter got the wheels in motion, and before we knew it, we had plans to start making a virtual experience together.
Unfortunately, with the limited number of weeks left in the school year and the looming sense of uncertainty for the indoor shutdowns, the idea of connecting a US History class project with the needs of the museum became unrealistic with the little time we had left in the semester. Yet, we both agreed that my experience as an instructor of both Extended Reality (XR) development and history, combined with the Midway’s need for new, hands-off, cutting edge exhibitions for the 75th anniversary of the end of WWII, meant that there was an opportunity to embrace here.
I decided to turn to a student in my Computer Science Principles class to see if he might be interested in working on this project. Since the history class was pretty much over, I had a brilliant student in mind who was looking to gain some tech industry experience. Reggie Estrella (now a senior graduating in 2021) had been a student in my classes for six straight years, oftentimes taking either STEM or Computer courses while also enrolled in my History classes. It would not be much of an understatement to say, “I’ve taught him everything I know.” From engineering to CAD, and programming (although he usually is the one teaching me the code), to the lessons of industrialization, Reggie has more than proven his tech skill set, and he also happens to be a darn impressive social scientist too!
As I had anticipated, Reggie was not just willing to help, but approached this project with tenacity and an ambitious drive to create something really impressive. Despite the fact I am a Learning Design Tech instructor and veteran history teacher, I was still only somewhat confident I could develop this experience on my own to the level I was envisioning. However, with a student like Reggie, who can program in multiple languages and develops 3D games in Unity (he can truly code at a level comparable to a CompSci major), I became emboldened and believed we would actually surpass the expectations of the Midway with our respective experience and combined skill set.
As the plan started to come together, it is fair to say we encountered a number of hurdles and challenges that slowed down the project and added to the already complex development task. The Midway remained uncertain whether a VR or AR experience would be best suited, and the clock was ticking as we approached the deadline for the Midway’s 75th anniversary of the end of WWII event. As the summer months began, Reggie and I started prototyping both VR and AR examples to share with the Midway. Although a VR experience of the temporarily off-limits hangar bay was a cool option, we all agreed to push forward with an AR experience that commemorated the history of the Battle of Midway, arguably the most notorious naval battle in US History!
With a development plan determined, the Midway was able to secure funding from GEICO Military to sponsor the project, covering the cost of development, materials and 3D models. With just about four weeks to make this AR concept a functional exhibit on the flight deck of an aircraft carrier, Reggie and I began working long hours to design, build, test, and iterate over the course of the first few weeks of August. Once we had the first scene developed, we immediately began to recognize the need to develop multiple stations for the experience, as we had to break up the scenes to prevent load times from bogging down the app and optimize the cloud performance across an array of mobile devices we were expecting visitors to use (budget Android devices, top of the line iPhones, tablets, etc).
Fortunately, Reggie was able to use more advanced coding, using variables, functions, and conditionals that improved the flow of the experience and sped up processing and visual registration of models. While Reggie was debugging and optimizing our first scenes, I was working with artists and historical resources to make certain every plane, every ship, and every bomb, matched the WWII vintage from the 1942 Battle of Midway. Fortunately, the museum was able to consult on the historical narrations and provide some of their available writing on the aircraft involved in the battle. Needless to say, the whole thing was buttoned up and packaged to impress pretty quickly. Once the audio and models were recorded and licensed, we were able to finalize the experience with some light testing onboard over just a few weeks. Initially, I thought we might have enough time to create many of our own 3D models, but the already available options for download in SketchFab proved to be exceptional. Huge thanks to ThomasBerens3D (Belgium), PaintedTarget (Provo, Utah – US) and Heljiah (France) for detailing every model and donating and discounting their tedious and immaculate artwork on the WWII ships as well as the planes we included!
Despite the varied and complex challenge of developing this AR experience, CoSpaces, which is a K-12 cloud-based tool designed for classroom use, did an amazing job of providing accessible and quite impressive AR visualization with pro-level interactivity and media integration. Including click interactions, animation, and narrated audio with sound effects, the Battle of Midway AR Extreme is not only comparable to what other professionally developed AR experiences offer, but it far surpasses what any other museum is currently offering as an AR exhibit. The folks at CoSpaces and Merge Cube were kind enough to agree to special licensing agreements that allowed for the use of their products in industries outside of the classroom.
In completing this AR experience, we have done something pretty remarkable. We have executed development with a novel technology that was learned in the K-12 classroom, and extended that potential to another learning industry in our community, bringing rich history and groundbreaking technology together with Learning Design principles to enhance the experiences of visitors on board the USS Midway. For those interested in checking out the experience, it can be viewed from the flight deck of the USS Midway Museum (circumstances of access may vary based on COVID restrictions). As for those who are interested in checking out the experience from home or from your classroom (must use paper print foldable or an actual Merge Cube to experience this in true AR), the whole experience can be viewed from the CoSpaces Virtual Museum Site with any mobile device that can run CoSpaces.
