When Google and Apple launched their mobile augmented reality (AR) platforms last summer, they effectively shifted the center of gravity for 3D application design and development.
Before these launches, the main hardware platform focus had been virtual reality headsets and use cases that tended heavily toward gaming, filmmaking, architecture, and engineering.
Post-launch, the focus is squarely on the far more diverse world of mobile devices and applications.
“The first great mobile AR experiences will be built by people who already know 2D design tools and workflows.”
One of the major impacts of this shift was to radically increase the importance of traditional 2D UX design patterns and workflows in the development of immersive applications for both mobile and web. Anyone familiar with mobile app development could start building immersive applications and experiences instantly, with the potential to reach a half billion devices—a vast increase in market size and device type compared with what was previously possible.
Related: 7 things to know about designing for virtual and augmented reality
The only problem: While 2D mobile and web UX patterns, workflows, and tools were relatively mature and well-integrated, their 3D counterparts were fractured, if they existed at all. Take prototyping, for instance.
Simply designing, building, and sharing a mobile 3D AR prototype—a task that might take a few hours for a traditional 2D app—regularly consumes weeks and often months.
And no single step in the process is to blame.
At Torch, we documented this issue in a white paper based on over 150 interviews with designers. We found that complexity and lack of tools for every step of the 3D prototyping process produced far more frustration that usable prototypes. In fact, so byzantine was the resulting workflow that we dubbed it, only half jokingly, the Wall of Pain.
The 3D Wall of Pain
There are five major challenges that make up the Wall of Pain. These groupings make it easier to both understand this phenomenon and, more importantly, begin developing solutions.
1. Learning new tools and developing workarounds.
According to the designers we interviewed, the single biggest challenge in developing new ideas was the need to either learn Unity, a powerful but complex game development SDK, or resort to paper and pencil.
As with other complex 3D tools such as Blender (used to build 3D models), Unity has a steep learning curve and is geared toward game developers. Those who opt not to undertake the considerable time and expense of learning Unity are left to rely on 2D renderings of concepts that fail to convey the power and impact of immersive applications.
Designers have developed elaborate paper methods to make 3D prototypes. Image: Saara Kamppari-Miller.
2. Cumbersome communication between designers, developers, and other stakeholders.
Another commonly reported challenge is the difficulty of collaborating to make real-time changes on a prototype. When a designer does not know Unity, they must first develop prototypes on paper or in a 2D illustration tool, which makes it impossible to convey depth, scale, and spatial relationships. This leads to friction and frustration when the 2D design is handed to the developer—or any time the designer is trying to convey interactions, animations, multi-scene concepts, and other complexities.
Designers often basically resort to miming as a means of conveying animations or changes to developers. They’ll also capture and mark up screenshots of the prototype in order to communicate design changes, which is a sadly ineffective way to fully convey a designer’s careful intentions.
3. Obtaining and creating 3D assets.
Anyone who has attempted to develop in 3D quickly learns how good we have it in the 2D world when it comes to file inter-operability and support standards.
First off, there’s an enormous number of vendor-specific formats—more than 40 at last count. And attempts to standardize (like the COLLADA and gLTF formats) have only been a partial success. It’s not uncommon to find that a design tool can open one 3D model of a specific format, but not others.
Creating high-quality models is expensive. A single, well-rendered figure can easily cost $5,000. In addition, online model warehouses are growing but present inter-operability problems and are still somewhat limited in what they offer. For example, one 3D modeller reported budgeting at least one hour a day to the task of repairing files that designers have downloaded from online sources.
Related: Incredible animated 3D illustrations by Lorenzo Etherington
The good news is, the situation is improving. Google’s Poly, Sketchfab, and the TurboSquid’s PixelSquid are all trying to tackle this problem, and making some important strides. But we’re still a long way from the usability of common 2D formats.
4. Rapid prototyping that’s anything but rapid.
According to nearly every designer we talked to, time is the first casualty of 3D prototyping. Days can be lost to a single iteration, which is not surprising when you factor in the frustration of learning new tools and workarounds; the difficulty of conveying even simple 3D concepts in 2D; and the challenges of finding or building 3D assets and importing them into a work environment.
5. The challenges of sharing work.
Even if, through some freak alignment of skills and luck, you were able to overcome the above agonies of the 3D Wall of Pain, you would still face the giant challenge of sharing your work. In fact, it’s the most commonly reported cause of 3D projects getting a red light.
The reality is that anyone with less technical know-how than you will not be able to easily view the prototype. That’s because each version of a prototype built in Unity is packaged as its own app, which means any change needs to be packaged and shipped to the client for viewing.
Viewing itself then becomes the problem. Designers frequently describe flying across the country with VR headsets to show a client prototypes and gather feedback. That feedback, even if expeditiously gathered, then takes time to implement (see #4 above).
The practical impacts of this torturous process can be crippling. Not only is each iteration expensive and time consuming, but clients often don’t see a prototype to provide feedback on until much later in the 3D workflow. With fewer opportunities to course correct, the client is less likely to approve the project. You spent all this time scaling the Wall of Pain, and in the end the project is scrapped.
Dioramas, while more effective at representing spatial relations, are difficult to share. Image courtesy of Cat Schmitz.
Not to be forgotten: the 2D UI prototype
If all this sounds like cruel and unusual punishment for designers, there’s more. We haven’t even addressed the 2D portions of the mobile AR prototype yet. Imagine, through sheer determination, you have overcome all of the above challenges to build a 3D prototype and get it approved. There’s still an entire 2D UI prototype to create. After all, mobile AR apps are more than just AR.
You still need to connect the 3D elements to the 2D UI elements that are locked to the screen. But there’s no way to quickly connect interactions in a 2D prototype to their 3D counterparts. Nor is there any straightforward way to combine the two parts of the prototype into a single unit for sharing and gathering feedback.
“Prototyping for mobile AR is like dragging yourself up a huge mountain only to reach the top and find another mountain looming in front of you.”
Getting the picture? Prototyping for mobile AR is like dragging yourself up a huge mountain only to reach the top and find another mountain looming in front of you.
Don’t give up just yet
If all of this makes you want to quit, bear in mind the following.
First, people are succeeding in building interesting 3D applications. Lego AR Studio is delightful and well conceived. Even simple line drawing applications, of which there are many, show a clever merging of 2D and 3D design concepts.
Second, and more importantly, help is on the way. New apps and tools are being built to fill the gaps in the 3D workflow, and more 2D designers are getting involved to help. Over the last year, we have watched experienced mobile UX teams improvise parallel workflows that converge with a single, well-executed prototype.
In a future blog post, we hope to share what we have learned about the current best practices for combining tools like InVision with existing 3D prototyping tools. In almost every instance, these efforts are being led by 2D UX designers excited by the prospect of building mobile AR applications.
Building a mobile AR prototype is really tough, but it’s getting easier. Opportunities abound for 2D designers to pioneer the workflows, tools, and interactions of an entirely new platform. And it’s worth a little struggle.
More posts on VR and AR design
Next to studying Graphic Design, I started working as a Digital Editor at Dupuis and then I became an Interactive Designer at Emakina — that’s where I met Vincent Battaglia (Ludus CTO). Quickly after, I co-founded my own creative bureau with Vincent (1MD or One Million Dollars) as well as WooRank (which became a market-leader SEO tool).
Paul Reynolds is CEO and co-founder of Torch, creating immersive prototyping and design tools for professional creatives. He has over 18 years experience in the video games and interactive 3D industry. Prior to starting Torch he was at Magic Leap for nearly 3 years as Senior Director of SDKs and Applications. Interested in early access to Torch’s 3D prototyping tool? Sign up for the beta at https://torch3d.com.
