Approaching a ‘small’ problem
Microsoft Native Ads Template Design
During the summer of 2021, I worked with Studio 8, a multi-product and multi-disciplinary Design team in Microsoft STCA. My project is on creating new hypotheses and designs for native advertisement templates in the News Feed section on the Edge Browser. Coming from a data science background, I find the most rewarding part of the internship the process of gathering and analyzing different data to inform design decisions.
Context
Microsoft Advertising Network(MSAN)
Jun 2021 - Sept 2021
Key Contribution
Concept ideation
User and product scope research
Hi-fi mock-ups and engineering handoff specs
Internal tools for design efficiency
Team
Designers, PMs, SWEs in U.S. and China
What is Native Advertisement?
Native Ads are ads in a format that matches the function and appearance of the context. In this case, we are designing for native ads in the News Feed section on the Edge Browser.
Design Goal & Metrics
Based on a monthly sprint, we want to design new native ads templates to grow revenue and improve user trust and affinity for both the advertiser and the Microsoft platform.
Original designs
Click-Through Rate(CTR)
Conversion Rate
Helpfulness
Trust
Visual appeal
Design Process
The team started with a hypothesis-based approach. Each designer comes up with multiple new hypotheses and has ownership of the greenlighted ones.
I owned multiple hypotheses and high-fidelity designs. In this case study, I will only showcase the design process for one ungreenlighted hypothesis without NDA constraints.
When I joined the team, various high-level hypotheses were built by UX designers, UX researchers, and PMs. I decided to dig into the idea of Sustainability. It is such as big topic and I approached it with multiple research methods such as Competitor Studies and User Persona.
How might we design ads templates that highlight sustainable practices of the business or the product to Bing ads viewers?
Competitor Studies
Differentiate
2. Analyze
3. Organize
4. Connect with Business Values and Persona
Among all ads viewers, whose perceptions are most related to sustainability?
How can ads resonate with the lifestyles of these groups?
What are Microsoft’s missions on sustainability?
How to establish a clear information hierarchy of mission and the commercials?
https://www.microsoft.com/en-us/sustainability/approach
User persona for millennial and Gen Z shoppers - a group with lower perception of Microsoft
Analyzing data from the first iteration, we found that the new templates have an inconsistent performance on the click-through rate(CTR), even if they all received positive UX feedback in the user interviews before launching.
What makes CTR unpredictable?
Ad click depends on relevance, affordability, brand preference, visual interactivity, etc. We only control the bottom layer - information displacement and interactions.
If a template is designed only for a specific user group(e.g Gen Z), or an ad type(e.g sustainable shipping products), it can receive positive UX feedback from a small sample of user testing, but might not work for a larger audience.
How Might We design templates that highlight sustainability while remaining…
1. Relevant to users with different levels of interest in sustainability
2. Visually accessible and appealing with different ads image and text
The universal design needs lots of data-driven decisions. In my design process, I was inspired by Jason’s sharing “Design is [Data]”, a case study in designing Ads on Google Search.
1.Universal design for different levels of interest in sustainability
First Iteration - Specific brands
I did some quick sketches and explored multiple visual treatments and interaction models using Reformation, a brand that advocates sustainability practices. However, it is too specific on brands with sustainable practices and stories.
Second Iteration - Universal patterns
Sustainability Score Tag
Explanatory Visualization
A/B testing feedback
Product data - A/B Testing
For each hypothesis, the team ran A/B testing on a slight variation of the visual elements or interactions. I focused on 2 parts: facts and assumptions. A made up example:
Fact: Progressive disclosure performs better than other interaction models. Assumption: It performs better because
Users experience information fatigue with high information density at the bottom of ads card.
Users are curious about the disclosed information.
For some hypotheses, I worked with engineers to design the new rounds of A/B testing variants to test out the assumptions.
User Data - Interview & Usability Testing
User feedback
A small sample of user interviews along with usability tests are conducted by the User Research Team for each design direction. Feedbacks on the UX Metrics(Helpfulness, Trust, Visual Appeal) are collected from 3 viewing scenarios: ad concept alone with no comparison, ad concept along with a controlled card, as well as ad concept on pages.
Backend Data - Development Process
Development feedback
What type of data is available? What graphic editing requirements are achievable? Many previous concepts were blocked because of the unmet technical requirement. Therefore, I reached to the engineers to ensure my design feasibility.
Iterate with Data
Final Design - Interaction
2.Universal design with different ads image and text
During my internship, I found that designers would design templates with mock-up ad images from big brands with clear visual communication. But when engineers run the templates with real assets, the templates and ad images can be incompatible in many ways. This leads to many back-and-forth communications and fixings.
Ideal image
Actual Image
Testing Design’s Accessibility
With my mentor’s support, I took initiatives in refining this flow between engineers and designers.
[Designer] I gathered data requirements from different templates and reached out to engineers
[Engineer] Engineering team filled out the real ads data
[Designer] Applied data to the new template with Figma plugin Google Sheet Sync
[Designer] Identified problematic edge cases and proposed solutions
[Engineers] Evaluated the requests
With this pipeline, the template designs becomes more efficient and universally accessible. I also gained a deeper understanding of cross-team collaboration.
This direction did not proceed to the spec session. I created engineering hand-off specs for some hypotheses and designs under NDA. The process includes specifying text and button sizes, providing edge case solutions(full bleed image, image with white background, etc), defining tab order, etc.
Some of my designs are already flighting with a positive Click-Through Rate(CTR) lift. My self-initiated project is in practice for future ads template design and received positive feedback from multiple engineers and PMs on how it improves communication efficiency.
Given more time, I would love to go more in depth on the research part. Although most of the research was conducted by the User Researchers for my project, I want to be able to answer the following [How Might We] questions as a designer.
A thankful note
Starting my day with the daily scrum had become a habit. I feel lucky to have the opportunity to work with wonderful designers and thinkers at Studio 8, Microsoft STCA, and to enrich my knowledge on accessible designs in the cross-cultural business & advertising domain. I really appreciate the unconditional support from my mentor Shu Zhang and manager Qiao Li. Also a big thanks to my colleagues for the insightful conversations and the happy hours:)
With a deeper understanding of why and how we innovate our way through ambiguity with an inclusive mind and data-driven decisions, I can't wait to carry the spirits to my next journey.