The future of user interface – the potential of post-screen design

The future of user interface – the potential of post-screen design

Published on April 8, 2016

We engage with multiple digital touchpoints each day, with companies competing to create the most innovative and interactive experiences. But, no matter how immersive the experience, we are still curbed by a limitation so integral that we barely consider it - the screen.

If we look at the timeline of new tech over the past few decades, mass adoption of the personal computer took off in the 1980s, the smartphone in the 2000s – it’s time for a paradigmatic development that will spell the beginning of a new stage of interaction design.
The way we interact with the screen is already changing. Virtual reality, wearables, projection technology and haptic UI are beginning to crystalize into this nascent phase of ubiquitous UI – one that goes beyond the screen. Look at the recent launch of the highly anticipated Oculus Rift; virtual reality (VR) is rising to prominence, alongside its less immersive cousin, augmented reality (AR). With myriad devices making the technology practically accessible to all, AR and VR are gaining traction as we move into the age of the post-screen user interface, where the boundary between digital and physical realities is becoming increasingly blurred.

The evolution of user interface

Ubiquitous computing demands ubiquitous interfaces. We live in a progressively connected world. The ever-connected nature of modern life demands a centralized method of control that has fewer limitations than the current form – the smartphone.
We have progressed from Graphical User Interface (GUI), allowing users to control information on screen with external apparatus (keyboard, mouse), to Tangible User Interface (TUI), more commonly referred to as the touchscreen, which allows users to physically interact with digital information. But, if the rapid rise of virtual and augmented reality is anything to go by, we may not be far from a future where we merge AR with TUI to create Tangible Augmented Reality (TAR), a technology that would bridge the gap in the transition between how we interact with the screen now, and the fully immersive world of VR.
The academic world is no stranger to TAR, with MIT and Carnegie Mellon conducting research and development in the space. But it’s when this technology hits the mainstream that we’ll see the most interesting developments.

TAR application in industry

As brands move into this field, they will have greater scope to redefine themselves through the experiences they offer.
TAR applications will differ from industry to industry, but it has real value to offer in areas broadly ranging from entertainment to tourism to architecture, to name only a few. AR and VR are already making movements in industries like education – take a look at Magic Leap and Google cardboard. So, this idea isn’t new, but TAR is promising to bridge the gap between the way we currently interact with screens and the fully immersive experience of VR, offering a semi-immersive medium.

But what does this mean for the way we design digital products?

Designers need to start thinking about this technology now. The key is to anticipate potential use cases and develop them so that you are ready to test your hypotheses and design with TAR as the technology becomes accessible. 3D interactions are set to proliferate, so designers need to take this into account and experimenting with different software for prototyping.
To offer the best experience to users we need to prepare for the impact this technology will have on new interaction models, which will shape the future of brand engagement and communication.

The major players in TAR

Alongside the big academic players, tech companies are getting involved with post-screen interfaces. As well as their work with Oculus, Facebook is delving into AR, as is Sony. Microsoft’s HoloLens is already making a splash, alongside the aforementioned Magic Leap, which is backed by Google.

What’s next?

We need to define what TAR is – as it stands, Tangible User Interface currently refers to a whole range of interaction models and the verbiage around AR is vague. To start efficiently designing for interactions with this new tech, we need to make sure there is standardization around the way we talk about it.
Technology like TAR gives brands even more scope to create meaningful connections with their users. But with immersive experiences, it’s imperative to offer experiences that are valuable, be it through personalization or otherwise, rather than contributing more noise to our already tech-crowded surroundings.
It’s important to remember that although AR and TUI are longstanding pieces of tech, it’s hard to predict the reality of combining the two. The practical application of tech like this has historically failed (think Google Glass), so it’s important to keep an eye on failures as well as successes to truly understand how to make a use case compelling enough for it to infiltrate our everyday lives in the same way the smartphone has.