Case Study

Navigating Complexity to Successfully Launch a New Outlier-Detection Service

Client Industry: Web Service & Cloud Computing
Client Revenue & Footprint: Big 5, Worldwide
Project duration: 10 months

Overview

Situation: A global web services company needed an intuitive user experience for their new machine-learning service designed to detect outliers in business and operational metrics.

Task: The product design team aimed to create an easy-to-follow guide for users to set up and activate the detector, and develop an outlier dashboard featuring real-time insights and detailed visualizations. The goal was to enable users to quickly and effectively act on the detected anomalies.

Action: As a digital strategy and design expert, I collaborated cross-functionally with client teams to drive the design process. This included gathering business requirements, developing proto-personas, conceptualizing designs, and facilitating the handoff to engineering. Additionally, I provided coaching on tool usage to ensure the design team’s proficiency. My involvement in this project required rapid acquisition of machine learning (ML) knowledge, breaking down complex workflows, and leading a team within stringent constraints.

Result: Despite encountering challenges and a significant pivot in requirements, the team successfully delivered the project on schedule, resulting in a smooth launch and a well-received user experience.

DISCOVERY

Due to an accelerated timeline, we were unable to conduct new user research. Instead, internal interviews with stakeholders provided sufficient insights to create proto-personas.

Users

The Mechanic (Developer/IT)

Responsible for overseeing the functionality of the detector, this role involved setting up and managing the detector, cleaning data, connecting data sources, defining metrics (KPIs for outliers), and configuring alerts. The Mechanic required an intuitive setup wizard with clear instructions, as the Help Panel feature was out of scope. Therefore, detailed instructions needed to be integrated directly into the UI.

Business User (Marketing Manager/Analyst)

Focuses on reviewing results and providing feedback to refine the detector. This user needs concise, intuitive outlier results that offer at-a-glance insights, helping them quickly identify potential issues and take appropriate action.

Use Cases

Scenario 1

The customer supplies historical data to train the model and sets up live data collection on a recurring schedule for ongoing outlier detection.

Scenario 2

Only live data is provided, with no historical data. Outlier detection begins once enough live data has been collected to train the model.

Scenario 3

The system is used for single-use experimentation and testing.

Definition

Terminology & Mental Model

Initial conversations revealed confusion stemming from the use of interchangeable terms without clear definitions, which led to communication gaps. To address this, I developed diagrams to visualize the relationships between key elements and compiled a comprehensive list of terms and their variants. After several iterations, we reached consensus on the mental model, selected preferred terminology, defined each term clearly, and established maximum/minimum limitations (e.g., outlier versus anomaly, detector versus model, measure versus metric). This exercise quickly and effectively aligned the team with a shared understanding of the terms and their relationships.

User Flow

I developed a basic wireframe flow diagram to offer a comprehensive view of the user experience. This diagram served as a valuable reference for discussions with stakeholders regarding user needs and goals, product scope, and business requirements. It went through several revisions to refine and enhance the user experience.

Blue: navigation. Grey: wizard. Magenta: needs clarification. Red: out of scope.

“This project convinced me that Laurel is indeed magic, and quite possibly even a wizard… but at the very least, a wonderful designer with an innate ability to make sense of chaos.”

— Senior Project Manager, projekt202

“She’s kick-*ss! I really love her design thinking, both mechanically and heuristically. She has a good sense for logic and mental models, which translates well to IA concerns. And she is a Dragon Master at Sketch… holy sh*t.”

— Principal Information Architect, Client

Design

efficient design and validation

The client’s design system predetermined the visual design, allowing us to swiftly develop layouts for review with the project team. These prototypes were instrumental in clarifying questions and validating the user experience during a full-day "Wall Walk" review with leadership and key stakeholders.

Simplifying Complex Data Concepts for Users

A key challenge was effectively communicating the concept of “metrics” to users. Users don't create metrics directly; instead, they assign measure and dimension fields that the system combines to generate metrics.

The complexity arises because there is no dedicated resource page for users to view metrics or the individual measures and dimensions. Instead, these elements are shown on the resource page of the datasets they come from, while detection results are displayed by metric. Users are billed by metric count, so it is crucial they understand the total count when assigning and mapping data fields.

To address this, we first established a hub to track user progress in the setup process and provide quick access to available actions. We then used this same visual language to create collapsible instructional panels in each Create Flow. These panels explained relevant concepts in context, ensuring users had the information they needed right where they were working.

The “Assign and Map Fields” step proved most challenging, requiring numerous revisions. We tested various approaches, including stacked vs. tabbed datasets, separate vs. combined field tables, and integrated assignment and mapping actions. Ultimately, the most refined solution, depicted on the right, was selected.

To streamline field assignment and prevent users from losing track while tabbing between datasets, we implemented a summary section that tallies the total number of assigned fields, keeping users informed of their progress. Since users must assign a field type to make a selection, we removed the redundant checkbox selector to simplify the process.

Explorations included using color and/or icons to differentiate between field types, allowing users to easily track their assignments at a glance. This approach enhanced usability and ensured users could efficiently manage their field selections.

Fortunately, the scope was later narrowed to a single data source and dataset, significantly simplifying the flow. However, the solution was designed to be scalable for future expansion to multiple resources.

Users can assign fields from any dataset up to a specific max. To prevent users from losing track as they tab between datasets, we provided a summary section to tally the total number of assigned fields. — Tabbed datasets with a field types summary for tracking assigned fields.

To make a selection, users must assign a field type. So we removed the redundant checkbox selector. — A selected field type for assignment.

Explorations included using color and/or icons to distinguish between field types, so users could track the tally at a glance. — Explorations for enhanced usability, including color and icon differentiation.

Constraints

Due to API limitations, creating datasets, connecting to data sources, and defining metrics could not be done individually. Instead, these tasks needed to be handled simultaneously, resulting in a complex "wizard-within-a-wizard-within-a-wizard" solution. While this was not ideal for usability, it sufficed for the MVP implementation.

Additionally, because the MVP could not utilize the standard Help Center from the design system, we embedded explanations directly within the UI and made the “How it Works” wizard collapsible to provide necessary guidance.

wiz within wiz.png — Complex "wizard-within-a-wizard" UI and collapsible "How it Works" wizard.

Curve Ball

The team had to pivot swiftly when the product's scope expanded significantly mid-project, increasing from 2 data connectors to 23. We quickly assessed the impact on user experience and development, adjusted the business requirements, and implemented effective changes to the features and functionality. An example of this impact is illustrated below, showing the increased complexity of the UI due to the expanded menu.

Increased menu complexity due to the expanded list of data connectors.

Collaboration & Efficiency

With design and development timelines running nearly in parallel, quick iterations and timely hand-offs were essential. The UX underwent a rigorous internal review process and multiple executive approvals before being handed off to engineering. Maintaining close communication between all teams was crucial to meet stringent deadlines. The UX team designed the complete setup flow for the detector, including create, edit, delete, and error states for each screen.

Summary

Outcomes

Successful MVP Launch:
- Achievement: Despite facing API constraints and a significant mid-project scope expansion, the team successfully launched the Minimum Viable Product (MVP) on schedule.
- Impact: Met critical deadlines and delivered a streamlined and contextualized setup flow, which led to improved user engagement and satisfaction. This success ensured a smooth onboarding experience for users.
Enhanced User Understanding:
- Achievement: Implemented embedded UI explanations and a collapsible “How it Works” wizard to guide users through the complex setup process.
- Impact: Provided clear, contextual assistance, which significantly improved user comprehension and navigation, leading to a more intuitive setup experience.
Adapted Design for Scale:
- Achievement: Adapted the UI to accommodate a significantly increased number of data connectors.
- Impact: Demonstrated flexibility and the ability to pivot design solutions to meet evolving requirements, ensuring scalability and continued functionality as the product grew.
Complex Workflow Analysis:
- Achievement: Analyzed and deconstructed a highly complex workflow, creating a detailed visual diagram.
- Impact: Improved process clarity and streamlined workflows, serving as a common language for aligning team members and stakeholders, which facilitated effective communication and coordination.
Team Development and Design Leadership:
- Achievement: Guided the multi-disciplinary client team through design methodologies and best practices. Coached a senior architect on using Sketch and provided guidance to enhance their design skills.
- Impact: Enhanced the team's understanding and application of design methodologies and tools, leading to more effective and efficient design processes and fostering skill development within the team.

Key Learnings

Reflecting on the project, several key learnings emerged:

Importance of Clear Terminology: Establishing a shared understanding of terms and their relationships early in the project is crucial for effective communication and alignment among team members.
Flexibility in Design: The need for a complex "wizard-within-a-wizard" approach highlighted the importance of designing solutions that balance complexity with usability, especially under API constraints.
Effective Communication: Regular, close communication between design, development, and executive teams was essential for meeting tight deadlines and adapting to scope changes.
Rapid Learning and Technical Acumen: Rapidly acquired deep technical knowledge in machine learning (ML) to effectively contribute to the project. Developed a comprehensive understanding of the technology stack, which was crucial for making informed decisions and providing valuable input.

Conclusion

This project highlights my capability to manage and lead effectively, even from an IC position. It underscores my ability to rapidly acquire technical knowledge, break down complex workflows, and lead a team while adhering to design constraints. I guided the multi-disciplinary client team through design methodologies and best practices, created a responsive component library, and rapidly iterated under tight deadlines to ensure a successful MVP launch. Additionally, the effective use of in-UI guidance and iterative design ensured that users could navigate the setup process with ease. The experience underscored the value of clear communication, flexibility, and user-centered design in driving project success and meeting organizational goals.

← Back to Case Studies

Prev / Next