Apple Vision Pro Spatial Design 101 - (Part 2)
Learn Spatial Design from scratch
This is part 2 of an ongoing series, catch up on part 1, part 3, part 4, part 5 here.
How do we interact with apps and objects in a 3D immersive experience? Interestingly, our eyes and hands are the new spatial inputs to perform interactions and this article will explore them in detail.
I. Eyes
Focus & Hover effect
When people focus on an interactive component with the intention of interacting with it, a subtle visual highlighting occurs, known as hover effect, which is a visual feedback (to confirm the object that they are targeting).
Long Focus
When an element such as button, tab bar, etc. is focused on for some time, it reveals more information, maintaining a clean UI.
Focusing on the microphone glyph in a search field triggers Speak to Search, revealing this layer and allowing to perform a search using just eyes and voice.
Dwell Control Feature — Assistive Technology
People can select content just with their eyes, by activating the Dwell Control feature. In this example, focusing on a button for a short time will show the Dwell Control UI and will select the button without needing to perform a tap gesture with hand.
II. Hands
1. Indirect Gestures
An indirect gesture is primarily done from a distance. When people bring focus to a button and it highlights with hover effect indicating that the button is targeted, they can activate or select it by quickly tapping a finger to their thumb to make the indirect tap gesture.
Indirect gestures can also be used to scroll, zoom and rotate.
Using eye direction combined with hand gestures can create precise and satisfying interactions.
For example, when zooming an image, the origin point of the zoom is determined by where within the image the eyes are focused at that moment. This results in that particular area to be magnified and centered as it zooms in.
Another example of this behavior is pointer movement in Markup. To draw, people can control the brush cursor with their hands, similar to a mouse pointer, but then if they look to the other side of the canvas and tap, the cursor jumps there landing right where they are looking.
Standard gestures are available by default in visionOS to perform interactions.
2. Direct Gestures
A direct gesture affects the virtual object that people are touching (bringing a finger close to an object). For direct gestures, virtual objects are placed near the people to be able to reach them and use fingertips to interact with them, such as scrolling a page, typing on a virtual keyboard, manipulating 3D content using fingers, and more.
Direct touch is to be used for those experiences that invite up close inspection or object manipulation, any interactive mechanic that builds on top of the muscle memory from real-world experiences, and when physical activity is at the center of the experience.
When components in virtual space are interacted with direct touch, there must be visual and audio feedback to compensate for missing tactile information, and to make direct interactions feel reliable and satisfying.
For example, while a finger is above the keyboard, buttons display a hover state and a highlight that gets brighter as it approach the button surface. When a button is tapped on, the state change is quick and responsive, and is accompanied by matching spatial sound effect.
Other familiar inputs
Devices such as keyboard, trackpad and game controller can also be connected to interact with the apps in the spatial experience.
Using eyes and hands to interact with virtual content is new for most people. That’s why it’s so important to guide them by providing clear feedback and to rely on familiar interaction patterns and gestures where possible.
In my next article here, we focus on how to design great UI for spatial computing apps.
Credits: Apple
Let’s work together on an interesting project. Ping me at hello@luxanza.com.
Get the Best Seller Professional Resume that has helped dozens land great jobs, now available here.
