In February of 2019 I was interviewed by the New York Times for an article about augmented reality in museums. My quotes did not end up making into the article, but I came across the e-mails recently and feel the answers are worth preserving. I’ve made minor edits for grammar and clarity.
How are museums using it now?
At the moment AR is often used for adding labels or little animations to existing objects, and about showing three-dimensional captures of additional objects that were not physically in the exhibit.
What are pros and cons of its use?
Even in its simplest form, AR can be used to convey far more information that what you fit on a typical plaque, providing a much deeper context and understanding. But a brochure does this too, and to experience an exhibition through a tiny window held in your hand is not optimal. The true potential of the medium lies in its ability to do things preceding mediums could not achieve.
Which types of museums is it most appropriate for?
A modern artwork is made with a gallery or museum exhibit in mind; that is the context for it to be experienced in. Augmenting that to provide more context can certainly be worthwhile, and there is already physical art that its creator considers incomplete without its virtual or augmented components. But a historical artwork or artifact in a museum today is seen far outside its cultural, geographical, and historical context, and we're left to bridge that gap with our imagination and preexisting knowledge. AR can close that gap considerably, allowing us to see the object being made or used, seeing and hearing its author, and even for a moment inhabiting the very time and space where it was created. No brochure can do that.
How do you see its use evolving?
AR will become an increasingly more integral part of any exhibition, to the point where its absence is as noteworthy as a person without a cellphone. Waving our phones around for AR is not the end game, and once we're in the era of consumer AR glasses we will quickly come to expect augmentation as a part of every day life, not just in museums.
It's safe to assume most, if not all, natural history museums are hard at work digitizing their collections, not just as photos but as complex 3D models. This will allow us to see the vast treasures that can't fit on the exhibition floors. It also promises a future where we're there's a lesser dependency on physical presence, where instead any object or artwork can come to me where I am.
But for that to feel worthwhile, the virtual representations need to move much closer to the real experience. Thankfully the tools for digitizing physical objects are rapidly improving, allowing the preservation of objects to an increasingly fine degree of fidelity, including very precise information about its materials and how they interact with light and so on. A historical dress saved as a 3D model with a flat texture is neat, but inferior to seeing the real thing. Seeing its virtual incarnation move naturally, its materials deforming and shimmering in an entirely physically correct manner, observable so close you can see individual fibers, will instead give the viewer a greater understanding of the piece than a static display could. While this is cumbersome and costly to create now and for quite some time forward, here lies the potential of the medium: To provide context, insight, and understanding beyond what any museum is able to convey through other means.
What are long-term prospects of AR use by museums?
Just as special effects in cinema are increasingly imperceptible as synthetic, there is a point on the horizon where technology for both capture and display is so good we will see no meaningful difference between a physical and a virtual object, or a real sound versus a synthetic one made to appear as though it emanates from somewhere in the room you're in. But beyond the technology lies the science of how we perceive, interpret, and understand the world around us. As the sophistication of AR climbs every upward, an important component is to alter our sensory input, perhaps even to a personalized level, to convey understanding more efficiently. This challenge deals less with ones and zeroes than with human perception itself, and the most successful AR companies will be those who can solve both.
What challenges do museums currently face in its use? How will these challenges change going forward?
Bottlenecks abound, as there's massive room for improvement across museum's understanding of the medium's potential, visitor's familiarity with—and appreciation of—AR, production cost for custom apps, access to digitized content, optimization of 3D scans etc to perform well on today's phones, production of custom content, bandwidth, and more. But all of these change dramatically year over year at the current pace. Rather than looking at the obstacles of today, I'd recommend considering how inevitable the rapid evolution of all this is. When any object can be digitized in great detail, using commodity hardware in minutes, augmented with any text, audio or animation one can imagine through intuitive interfaces, and distributed to anyone in the museum through open standards, who would choose to not make AR content available? And all of that is either possible today or very soon.
Which demographic does museums' use of AR appeal to? Do you see this evolving over time?
Phone based AR today is often a bumpy user experience that can often feel like novelty, attracting mainly the curious and tech savvy. But what if it's in the browser with no app download requirements, no perceptible download times for content, visual fidelity that makes it genuinely difficult to tell the virtual from the real, and a more expansive and ambitious production value for the content itself driven by better authoring tools—I think at that point there's no more a demographic divide for AR adoption than there is for using your phone's camera or looking something up in a search engine.
Will "David Bowie Is" be a prototype for museum exhibitions of the future or do you consider it a one-off, developed under unique circumstances?
There's plenty here other curators could draw inspiration from, and it's scope is impressive. It is ultimately however a physical exhibit in virtual form, and while deeply valuable as a means to democratize access, I believe the ultimate potential of AR is to provide something beyond what a physical space could.