I’m back from AWE 2024 in its new home in Long Beach, and what a trip it has been.

The conference was awash in discussion and demos of the newest elephant in the room: Generative AI. After all, who can resist the allure of a technological genie that can grant all your digital wishes? (Though precious few people acknowledged the contributions of all the artists, authors, and other creators whose data was scraped to conjure said djinn.) Here are a few of the most dazzling gems from the show. (Note that you can find the videos for all AWE sessions on the AWE YouTube Channel.)

Generative AI Accessibility

First, allow me to 🌬️ my own 🎺: our panel on How Generative AI Can Make XR Creation More Accessible went great! Huge thanks to my panelists Sean Dougherty of Lighthouse SF, Stefanie Hutka of Sendfull and UC Berkeley, and Ash Shah of Magic Leap. (Shoutout to Molly Bloom of Cisco who had to cancel for a family emergency.) We discussed:

1. How GenAI can empower disabled creators to bypass inaccessible platform tools to share their creative visions and innovations.

2. How we can ensure that all content created by GenAI is accessible and free of ableism and other algorithmic bias.

Rest assured this could fill an entire blog post of its own and probably will soon. You can watch the panel on YouTube; for further reading, I’d suggest:

1. Art, AI, and Disability Futures

2. We are exploring Accessibility, XR development tools…and getting lost!

3. “They only care to show us the wheelchair”: disability representation in text-to-image AI models.

Wishing for More Wishes: How GenAI Will be Used and Abused

There were plenty of panels on how best to use generative AI, as well as forecasting how it will be abused and what we can do to mitigate that.

• GenXR: Creating AI Powered XR (Terry Schussler, Deutsche Telekom; Leslie Shannon, Nokia; Dominique Wu, XRealityPro)

  • GenAI enables people to spin up entire worlds simply by talking to an AI agent. That said, users are often not very imaginative or descriptive, and it may be necessary to “prompt engineer” users in order to get them to elaborate on what they want to see.

  • One of AI’s strengths is that it enables powerful data visualization with natural language, empowering nontechnical users to ask questions directly to the machine instead of waiting for a data scientist to whip something up. That’s how the city of Wellington, New Zealand ended up with a 3D map of vomit locations!

  • GenAI could support reading by enabling readers to “ask the author” in realtime by training an AI specifically on an author’s work and notes. They demonstrated with a convincingly creepy Edgar Allan Poe answering questions about his lost Lenore in “The Raven.”\

• Using AI Image Generation in Production Pipelines (Lauren Morrison, Google)

  • GenAI can quickly generate hundreds of pieces of concept art, but it’s easy to get lost in the details - make sure you can articulate what you like about an image and the core concepts it embodies that are relevant to your project.

  • One of the weaknesses of GenAI right now is that it’s hard to make small adjustments. Since there’s no underlying layers or 3D models, you can’t make small tweaks to camera angle, lighting, or objects. For that reason, it’s still better to create your own models and images once you’ve gotten past the brainstorming phase.

  • AI can level the playing field for low budget creators, but it’s no substitute for experience and an artistic eye.

• The New Holodeck: A Vision for Spatial Problem-Solving (Valentin Heun, PTC)

  • Star Trek’s famous Holodeck is ultimately a place where you can put data in specific physical contexts. Now we can do the same by translating video and scans from the factory floor and viewing them in XR.

  • We can teach GPTs to understand space and reason about it, annotating events like the steps to operate a piece of machinery, or evaluating scan data from the surface of the moon. (I really can’t do this one justice but it’s REALLY cool - wait for the video!)

• Impact of XR, AI, & Emerging Tech on Journalistic Integrity (Melanie Hogue, Microsoft; Mary Matheson, ASU; Adam Rose, Starling; Robert Hernandez, USC JOVRNALISM)

  • AI makes it easier to fake content than ever before, so we need to find new ways to authenticate our news, such as Glass to Glass Authentication.

  • Misinformation isn’t new; plenty of “news” organizations already willfully present things out of context, no AI required. Preventing abuse is just as much about a culture of informed skepticism and media literacy as it is anything technological.

  • XR has the power to break through disinformation bubbles by presenting a “raw experience” that we’re more likely to believe is real. Of course, this can equally be used to reinforce an existing narrative.

The Real Growth of XR

XR has always suffered the ups and downs of the hype cycle, but over time, several rock solid use cases have emerged and continue to grow. As XR matures, so too must our processes and regulations grow with it.

• Deploying XR for Veteran's Care & Transition (Stephanie Montgomery, XR Association; Anne Lord Baily, Veterans Affairs)

  • VR has shown great power for empathy-building and training - for example, simulating sexual harassment, or teaching gun safety.

  • Much of the focus is now on improving deployment within clinical workflows. In a perfect world, a clinician could prescribe VR as easily as medication!

• Welcome to the Elderverse (Robin Raskin, Virtual Events Group; Jeff Pohlman, HTC Vive; Richard Robinson, AgeTech Collaborative from AARP; Ted Werth, Mynd Immersive; Trent Hermen, Select Rehabilitation)

  • Elders represent a huge and growing percentage of the US population and spending

  • VR can be incredibly useful for socialization, physical and emotional therapy, and safety evaluation. For example, can an elder in VR identify hazards like a towel on a stove in a virtual kitchen, or successfully shop for themselves at a grocery store? The answer can help determine whether it’s safe to let them live alone.

  • We need increased focus on how nurses, therapists, and caregivers can support their charges in setting up and using VR. They should be able to launch programs and monitor status from a phone or tablet with minimal friction.

• Helping the Blind to See (Charlie Fink, Forbes; Caspar Thykier, Zappar; Soobin Ha, Benvision; Michael Freeman, Ocutrx)

  • Accessible QR codes can both help blind users access product information in a store, and provide data directly to manufacturers about interest in their products. (While protecting data privacy, one hopes!)

  • HMDs like the Ocutrx AR Headset can use passthrough AR to automatically correct the vision of people with macular degeneration (central vision loss), for example enabling them to read by identifying the dead zone in the center of their vision and automatically shifting text to the peripherals. (Rad!)

  • Applications like Benvision use AR and machine vision to create a musical experience that can inform and guide blind users around a space. (Full disclosure: I’m on their advisory council.)

• Deploying Sustainable Enterprise XR Solutions (Angelina Dayton, Virtual World Society; Dan Blair, BSD XR; Brian Nevinsky, Lockheed Martin; Nathan Thomas, CN Railway)

  • Many enterprise companies are exploring XR for training and evaluation, but have conflicts working with smaller studios. Best practices include:

    • Advocate for an internal council with key stakeholders, so you don’t run afoul of silos.

    • Allocate time and budget for quality assurance up front, as it is often the victim of budget cuts and delays.

    • Ensure there are failsafe policies in place, so there’s no confusion about ownership of data and WIP.

    • Make sure your solution can run independently of the network, as many work sites won’t have reliable internet.

  • Since there are no commonly accepted accessibility standards for XR, common practice is to have a screen-reader accessible companion app in 2D. (That’s terrible for both companies who have to make the app twice, and disabled folk that can’t benefit from XR!)

• Identifying & Overcoming Regulatory and Legal Roadblocks (Brian Wassom, Warner Norcross + Judd LLP; Tony Liao, University of Houston)

  • Social science in XR is vital for understanding impact on difficult topics and vulnerable communities. We need to invest in it via organizations like the Critical Augmented and Virtual Reality Research Network (CAVRN).

  • VR still faces critical privacy risks - for example, some studies have been able to identify individuals by their motion in VR with a 95% success rate. Motion can also encode personal data like gender, disability, and even substance use.

  • Regulation is still lagging behind when it comes to physical safety, mental safety, and societal stability. We can look towards regulation on biometrics as a source of insight.

Bonus Innovations

Finally, AWE wouldn’t be complete without a few fun things from the expo floor.

Another One for the Books

Overall, I had a fantastic time at AWE this year; the level of innovation on display is fantastic. Though my concerns for the welfare of artists and other creators whose work fuels the Gen AI revolution is not abated, it is great to see how it can be put to use as part of the artistic process, not just as a shoddy replacement. XR continues to grow, and I can see accessibility becoming a growing concern for those who want to see the technology mature properly and inclusively. Hopefully next year is even better!