Design overview

What is Fiori Android AR annotation

AR Annotations components are designed for exploring objects in a facility. They can be used to explore a warehouse, machinery, or a business object to enhance the user experience by leveraging augmented reality.

My role

I am one of the designers for the Fiori Android AR annotation experience. The Fiori Android AR annotation is part of the Fiori android design system. Before I joined this project, we had two designers who explored the Fiori AR user case, based on their foundation we started this project.

Challenge

Based on the design for the Fiori IOS AR annotation experience, we need to cooperate with the developer to create an Android AR annotation experience based on the ARcore.
Understand iOS ARcore's capabilities & limitations.

Product related

Department  | SAP mobile
Duration | 4 months
Designer | IOS research: Suhaib, Naomi ; Android: Mianying, Mina
User | SAP product designer / developer
Toolbox | Figma, After Effect, Android Studio

Final design overview

Fiori IOS AR exploration

Afte Mina and I took this project, our first thought is to connect with the AR IOS research team to understand why they picked AR annotation as the user case for the initial SAP enterprise AR experience.

Background acquisition

The AR IOS research team went through a consumer-facing AR experience they conclude on the market for us. These include home goods online retail experience, AR education experience, AR navigation experience, and wearable/immersive AR experiences SAP did previously.​

IKEA consumer-facing AR experience

Google navigation experience

AR education experience

SAP Enterprise AR  experience

Design constrain

Based on the research they did above, they thought the constrain of the AR project on the market would be.
•  3D assets not widely available, unless core to sales process.
•  Lack of clarity with regards to who (OEM, customer) should develop experiences​.
•  Localization techniques still require visual anchors (images, object detection under some conditions) for stable, robust experience​.
•  Consumer-grade Mixed-Reality hardware not widely available​.

Design opportunity

But there are also many opportunities in the AR field now.
•  Modern features of occlusion, persistence & collaboration provide a functional basis for advanced experiences.
•  All modern iOS devices support AR experience; latest iPhone series includes LIDAR​.
•  ARKit is a focus technology for Apple, and thus relevant for SAP in delivering best-of-breed apps.

Fiori for IOS ARkit scope

After they finished the initial round research, they defined the scope of the Fiori for IOS ARkit.
•  Screen-based (iPhone, iPad)​​.
•  Annotation-centric​.
•  Indoor Navigation with ARKit​.

Fiori for IOS AR annotation

The first area Suhaib and Naomi focusing on is the AR annotation experience. Through the example below you can find nowadays we already use annotation to annotate the world, but there are many problems there for the real world annotation (Annotation text not clear, Text faded...), so, is there any possibility to use AR technology to improve the annotation experience?

We already annotated the world

Problems

Design options

3D world space (plane anchor)

Pro: Realistic, highly engaging, advanced rendering capabilities ​
Cons: Only static content, hard to implement interactivity, significant jitter​​

2D projected world space (plane anchor)

Pro: Realistic, highly engaging, advanced rendering capabilities ​
Cons: Only static content, hard to implement interactivity, significant jitter​​

2D projected world space (plane anchor)

Pro: Swift UI, very fast to develop, interactive, dynamic out of the box, still provides context.
Cons: Hard to read (jitter and movement), hard to place convincingly, no occlusion, won’t take scene properties.

2D Hybrid (Image anchor)

Pro: Balances discovering content in context with readability.
Cons: Complicated (but we do the work for you)​​.

The plane anchors only can place content vertically or horizontally. The​​accuracy and precision of persistence with plane anchors are highly variable. Based on the design options above, the IOS research team decided to use the 2D Hybrid design as the final design.

Fiori IOS AR components.

Initialization / Coaching

Coaching users to help them find the right image to start their AR experience. Users can expand the minimized instructions for the unobstructed view on demand.

Annotation markers / Transitional connector

The annotation marker has select and unselects states.

Annotation card

The annotation card contains the information related to the marker. If the user types the card, it will direct the user to the card detail page. But if the user types the marker, then the bi-directional interactivity helps in discovery, and the transitional connector directs attention to the right marker/card​.

Design for Fiori Android AR annotation

Based on the foundation of IOS AR annotation's design,Mina and I are tried to explore the Android AR annotation experience. Since the AR tech IOS and Android are different, we first explored the Android ARcore's documents.

ARcore exploartion

Mina and I cooperate with the developer to went through the document and tech for the ARcore guideline, then we found compared with IOS that can place the AR elements in the 2D environment, all the elements in Android needs to place in the 3D environment which means we need to consider in z latitude (Depth) when we create the design. I made a very simple prototype to explain if we want to make a 3D annotation similar to the IOS experience to help our developer know what it may look like.

Design options with design review

Card

The card design under the research phase only provided a short description related to the resources behind it. We want to provide designers more flexibility for the new card design to help their users better understand the information behind the card.

Annotation

We want to redesign Annotation since the ARcore only places 3D elements in the AR environment. How to help AR annotation indicate the element more accurately and provide a better experience is the thing we want to verify.

Our Android team decides to use the card with preview image, title, and description to help users preview the content through the team internal design review. For the annotation, we combine a small dot with the annotation to avoid the 3D jitter problem.

Design verification and research

After we finalize our design, we start user testing with the research team to verify our design goal approach user's expectations, which is optimistic. Our next step would be the AR authorization flow to help the business user build their AR experience for their user.

Stencils with auto layout

At last, we use this checklist below to finalize our design and put them into our stencil.
• Link every text style.
• Link every color style.
• Link universal components (buttons/icons).
• Apply auto layout, constraints, resizing.
• Create variants.
• Proper naming.
• Dark and light mode.