THE PROBLEM

We had just launched an API-first platform, but only a fraction of our users had coding experience. We wanted them to use our API to build and deploy applications for their non-coding colleagues.

THE USERS

Material Scientist (ME) - User running scientific experiments with deep domain knowledge but mediocre technical skills. Needs to decide which experiments to run to find "the next big thing".
Data Scientist (DS) - Python-savvy user who trains ML models and needs to graphically explain their work to materials scientists.

THE REQUIREMENTS

• Users don't need to install any software to get started.
• Data Scientists can build and deploy Python-based applications using our technology.
• Services and Customer Data Scientists can collaborate with each other on the code.
• Applications are securely stored within the customer's platform deployment, with no chance of data leakage.
• Material Scientists can interact with the applications with zero Python knowledge.
• The solution is tightly integrated with our existing B2B SaaS offering.

HOW WE BUILT IT

We worked in 2-week sprints and hit milestones extremely quickly.
Sprint 1: Shipped a functional prototype where we ran Python code remotely.
Sprint 3: Built a UI to allow users to sign in and work with a ready-to-go environment.
Sprint 6: Created a landing page where Material Scientists could review user-built apps, load one, and interact with it without seeing a line of code.

THE RESULTS

The product worked.

Really well, actually.

Internal Data Scientists loved it.

But the business needed to pivot to self-serve customers, and we didn't trust customer organizations to have the same level of Data Science experience. An API only wasn't enough.
‍
This kicked off a major engineering effort to build a tailor-made UI to our target user segment.

THE PROBLEM

I was the Product Manager of a legacy product we were planning on deprecating. With only a few engineers, we needed to support several six-figure contracts. But how could we prioritize that work?

THE USERS

Services Team User- My power-user colleagues who need to build accurate AI models using customer data and deliver convincing results to the customer.
Customer User - Customers, often AI skeptics and unskilled in making data-driven decisions, who need to have a basic understanding of the technology and act on its recommendations.

THE REQUIREMENTS

• Services Team users need to be able to take advantage of any feature on the platform, even if there isn't a purpose-built UI.
• Services Team users need insight into why their AI models aren't training successfully and how to fix them.
• Customer Users need a simple, straightforward interface for answering key questions and understanding how the product works.
• Services Team users need to be able to quickly perform their work on the platform.
• Services Team users needed rapid turnaround on high-priority features that were blocking their projects from being successful.

HOW WE BUILT IT

On the first day, we focused on identifying the core problem at hand:
I gathered my Scrum Team and a group of Services Team users and asked them "How can we make our primary workflow [of delivering AI-based recommendations] seamless?"
I took screenshots of the current experience, and led a critique of that area (Rose, Thorn, Bud)
Then, we identified trends by clustering together similar feedback (Affinity Clustering)
This gave us a set of clear, actionable themes to consider

On the second day, we switched to focus on the solution:
We brainstormed potential solutions for the key themes using a variety of levers. This included documentation, Services Team work, and feature development.
We voted on the key solutions we wanted to pursue and estimated their ROI by asking users for how valuable a feature would be and engineers how expensive it would be to implement.

With this relationship between the developers and services team built, I put developers directly in contact with the people they needed to support when it came time to build something and they would find the right solution.

With the burden of finding the solution off my mind, I could focus on finding the next problem to solve.

THE RESULTS

Our team earned a stellar reputation within the company for consistently meeting the needs of its users.

So much, in fact, that we were consistently compared to another team with 20 members and told that we were more effective than them and "why can't it be more like [my team]".

We sent out a survey asking how we did over the course of a year.
‍
70% of Services Team respondents said the product "improved significantly over the last 12 months"

Immediately afterwards, we hit our quarterly goals a month early.

THE PROBLEM

I was the Training Program Manager responsible for teaching a set of student users how to use a software application to manage combat aircraft. Users typically took almost a year and required a ton of instruction to achieve their training goals. However, the organisation didn't want to spend the same level of resources on the students of my program. The existing model of 90% in-classroom training without even interacting with the product wasn't going to cut it.

The process workflow below shows a complex interplay among the three user personas during graded events, but minimal interaction outside of graded events. This is where a majority of the waste occurred.
- Instructors spent significant time preparing single-use lesson plans.
- Students didn't know how to effectively study.
- Pilot simulators lacked the training to effectively aid the instructors.

As a result, our students would consistently struggle with actual application usage despite being able to clearly articulate how to use it in the classroom.

I hypothesized that we could simultaneously improve quality while reduce the training cost and by giving them more hands-on experience with the product and have each instructor teach more students.

THE USERS

Student - Military members from other countries with wide variation in experience, motivation, and English speaking skills.
Pilot Simulator - Junior military members with high school education and limited real-world experience. Good at following directions and reading from a script. Responsible for simulating the role of pilots during training events.
Instructor - Senior military members with combat experience. Very skilled, but expensive.

THE REQUIREMENTS

• Students can interact with the product during all stages of training (was previously only during graded training events).
• Any instructor could deliver the lesson with <15 minutes of preparation (was previously 4 hours).
• No more than 2 training staff members are required during lessons (was previously 4 members).
• Time from login to training start <5 minutes for all users (was previously 30 minutes).
• Students require no more than 8 hours of product onboarding to begin their first graded training event (was previously 16 hours).
• Students could complete advanced training lessons (was previously extremely simplified lessons).
• Curriculum changes could be continuously delivered (was previously 1-year cycles).

HOW WE BUILT IT

Our first goal was to get students interacting with the product, which required a different teaching environment.
‍
There were 3 major items on the first quarterly roadmap:
1. We started off by installing a podium and projector in the classroom, which enabled an instructor to demonstrate how to accomplish the tasks.
2. Then, we synchronized all the training material on a SharePoint drive and delivered it automatically to the teaching podium. At this point, any instructor could log in, access the right training material, and teach the students.
3. Next, I automated 80% of the login process so users could immediately begin interacting with the system.

At the end of the quarter, there was significantly less waste in preparing lesson plans and non-instruction work, plus the infrastructure to rapidly deploy changes to courseware or product configuration.

Since changes were cheap to implement or roll back, we could rapidly experiment and immediately on stakeholder feedback. This accelerated the training program's evolution.

We continued iterating on the program and dramatically changed the lesson preparation workflow. We integrated 3 parallel, often ad-hoc processes into an interactive practice session.

hover over the workflow diagram below to compare before and after!

In the following 3 quarters, we dramatically changed the user experience by rewriting all the content from scratch, doubled the number of hands-on training hours, and continuously reduced the overhead required to deliver product training.

THE RESULTS

An Austrian student with 20 years of military experience declared my training program the "best course I've ever taken".

Users provided consistently positive comments in end-of-course critiques.

I earned an organisation award for "Best Instructor" among a group of 64 of my peers.

Lastly, our methodology caught the attention of the larger organisation, which was still stuck in theoretical instruction, 3-year syllabus cycles, and a extremely slow student progression.