AI and spatial computing promise a paradigm shift in product design
Spatial computing and artificial intelligence (AI) will facilitate a paradigm shift in the product design processes and will give product designers more tools to shape user interfaces and experiences.
Dane Howard, head of design for Samsung NEXT, talked about the future of user interfaces (UI) during a recent InVision webinar. Product designers, he said, have the opportunity to embrace these changes sooner rather than later.
“There’s always going to be, right in front of us, new technologies, new business models, and new consumer habits that create change,” Dane said. “And with that change comes opportunity.”
Masters of the 2-D interface
While web and mobile design are both relatively mature, change is on the horizon as ubiquitous screen-based UIs begin to be increasingly ‘self-monitored’ by users. Increasingly, users are realizing that they are “too engaged” with screens and they are self-monitoring their own screen time.
As a result, they are finding new ways of interacting with ambient user interfaces. Voice assistants, such as Alexa and Bixby, can already look things up, send messages, set reminders, change your music, and take other action based on voice commands.
The future is going to bring a whole new UI experience, Dane said. For example, at the Coachella Festival this year, festival-goers could see an AR-based virtual extension of the stage on their phones while they watched, filmed, and photographed the performances.
The physical and digital realms blending together. Screens won’t disappear anytime soon, but they will lose their dominance as this trend continues. And that has huge ramifications for design. “What happens when the information is all around us and the compute layer is all around us,” Dane asked. “What will that mean to the user interface?”
As interfaces move to real spaces, context becomes key. In the future, Dane suggested, information will be organized and accessed in real-world layers and authored in context. This is spatial computing, and designers and technologists will be responsible for setting the design and interaction patterns that define how it works.
A new 3-D canvas
Software designers traditionally define their canvas according to screen dimensions. But in order to create digital content and design interfaces for a real space, they have to capture it first.
This process is called photogrammetry. It involves photographing a space to make measurements of it and translating that data into a point cloud that can then be assigned the appropriate context or logic. In spatial computing, that point cloud is the designer’s three-dimensional canvas.
Dane likened the use of photogrammetry to making a video game or film, except that what is created doesn’t diverge from the real world. Instead, becomes a virtual layer in a real-world context.
For example, one of Dane’s colleagues at Samsung NEXT built a storytelling rig in the Unity game engine to help work through and communicate to others the complex interactions that might happen in such a layer. This allows him to mock-up scenarios and storyboard sequential narratives in context — to tell stories that inform design decisions.
The rise of AI
As spatial computing evolves, Dane said, artificial intelligence is infiltrating every part of computing and business. According to a report in the MIT Technology Review, AI startups and investment have grown rapidly over the past 20 years, and key techniques, such as object recognition and machine translations, are quickly approaching human-level performance.
Moreover, an Economist Intelligence Unit report found that three-quarters of executives expect AI to be “actively implemented” in their company within the next three years.
Actually benefiting from the power of AI requires data scientists who can harness and analyze the data it generates. Dane pointed to the huge growth in demand for data scientists. Moreover, he said, there is a growing need for better-designed tools and interfaces to get the job done.
AI is a solution when it comes to new toolsets. For example, Dane discussed a machine learning technique that uses algorithms to teach itself to do a specific task by parsing a large dataset. Machine learning, he said, will help data scientists be more productive because the algorithms can be refined and trained to learn from experience.
AI-powered machine learning can help data scientists analyze massive amounts of information, Dane explained. It can save time and reduce the tedium of a job. Topaz Labs, for example, built a tool that uses machine learning to upscale an image without losing quality.
In addition to eliminating tedious tasks, Dane added, AI provides tools for creative assistance. It can be taught to brainstorm variations on a concept or to mock-up an idea. An AI tool powered by machine learning, Dane explained, is “basically a very studious student that is iterating for you, to allow you to then curate and … build off of that.”
Similarly, the user interfaces of the future could be personalized through algorithmic models that learn from user input to adapt. For instance, constant iteration and automated A/B testing can be used to adapt to subtle differences, such as the user’s hand size.
Skills for the future
The rise of spatial computing, Dane said, requires new skillsets. The first of these is that product designers need to get comfortable thinking in 3D. He said this is essential because it brings time and space into the thought process, which then leads to “time training.” This is “the ability to actually think and act technically and sequentially,” he explained.
Get a handle on this, Dane added, and designers can grow their skills in storytelling, which is essential for figuring out the scenarios that lead to the context-based interfaces and interactions of spatial computing.
But the most important thing in designing user interfaces, he added, is empathy and emotional intelligence — the ability to effectively manage both your own and other people’s emotions.
“If you think about your strongest opportunity to connect with customers,” Dane concluded, “to connect with them in a moment and empathize with them, this will be really a superpower from here, for a long time. It’s very difficult to train a computer how to have empathy, thus… your strongest opportunity to connect with customers, to empathize with them, will be your superpower, for a long time.”
For more of Dane’s insights on the future of UI design and how to embrace and prepare for new developments in AI and spatial computing, watch the full webinar here.