Understanding Education’s Role In Healthcare
As a team, we began the project without clear expectations or priorities. Stakeholders had varied and conflicting assumptions, and there was no cohesive product vision. This left an opportunity for me to influence the product strategy by bringing user-oriented evidence to the discussion.
Discovery Research Goals
- Learn more about how people use LMS products
- What are people using Knowledge Tracks for?
- What business purposes do learning experiences serve?
- Where do people spend time in our tools, and where is time wasted?
- Understand the larger context of use
- What are the roles involved with designing, creating, and managing education content?
- Discover specific issues and needs
- What are the motivations behind existing feature requests?
- Identify confusing concepts usability issues
Research Methods
My research focused primarily on remote interviews and user testing of the existing product. This was supplemented with customer requests that had been gathered from feedback forms embedded in the existing product.
When conducting research interviews, I approached education administrators at both our major clients who operate at scale, as well as administrators at smaller companies who represented the long tail of our customer base. These administrators are not only the users of the content authoring tools that comprise the bulk of the software suite, but are often the purchase decision makers for their company. Including them in research served a secondary purpose of reassuring frustrated customers that progress is being made.
Following the interviews, a knowledge board was created to track evidence and synthesize insights.
Managing Education At Scale
Primary Insights
LMS = Content x People x Outcomes. Each learning experience requires creating content, selecting learners, and tracking their outcomes. No part of the process stands in isolation, and a deficit in any part quickly renders the learning experience meaningless. All aspects of the LMS ecosystem must be considered in product decisions.
The product equation: Every use case comes down to combining content, people, and outcomes
Unsupported processes require time-consuming and error prone manual work. Based on expert interviews and user testing, we know that savings in time are highly valued. Several tasks like scheduling and followup are not supported and must be manually completed using offline processes like spreadsheets and calendar reminders. There is a tremendous opportunity to create savings in time for customers with large teams by addressing our assignments and outcomes tools.
Knowledge Track Creator usability issues create a perception of low quality. Our content creation tool is buggy and tedious to use, but fundamentally sound in its structure. As the tool our customers spend much of their time in, usability issues lead to perceptions of low quality that have an outsized impact on subscription renewals. MedBridge’s urgent desire to rebuild this feature is largely based on remedying perceptions of poor quality and retaining customers, rather than on creating user impact.
Deeper insights
While some conclusions quickly became apparent, synthesis and followup interviews revealed deeper insights around organizations’ behavior.
Chain of Command: Administrators typically identify learners who require followup. This is often due to missed deadlines or low test scores. Administrators will forward this information to the learner’s direct manager, who then follows up either in person or by email. This is a time consuming, manual process that does not scale as more learners are added to the organization.
Supporting Scale: Manual tasks that work for 25 learners can’t scale to 1000 learners. At scale, it’s not possible to manually check whether each participant is on track to complete their training on time. Policies for alerts and reporting could automatically identify students who need followup and send them appropriate reminders.
"It's hard to police 150 new grads." followup has to be delegated to another clinician. "It should be automated."
-P.I.
The student groups tool helps manage complex assignment tasks, but is not enabled for most administrators. An updated groups tool could help administrators quickly assign relevant training to students and analyze their performance.
Assignments: Having too much assigned work overwhelms learners. This causes administrators to stagger assignments, a manual process that is coordinated with pen and paper or spreadsheets. Basic scheduling tools could automate this work, saving time and decreasing errors.
Two Needs: Two primary use cases for Knowledge Tracks emerged: Single courses and onboarding. The LMS was designed for one-time learning experiences that are created, assigned once, and discarded completely. For organizations onboarding new employees weekly, Knowledge Tracks are continuously updated and assigned to new learners. This rolling update schedule created issues when tracking which employees had completed a given version of the content.
“We have to spend a lot of time trying to go back and figure out [completions] when hopefully there's a way for the system to just produce that for us.”
-P.M.
Recommendations
From these insights came a set of high level recommendations.
- Holistically address the entire process of creating, assigning, and following up on Knowledge Tracks.
- Support manual processes like scheduling and auditing to reduce the most time-consuming activities.
- Modernize existing tools with better usability and clear workflows.
Reframing the problem
The redesign of the platform was reframed holistically using a “How might we…” statement that provided a high level design prompt.
How might we help businesses create, deliver, and assess their educational content?
A Major Change In Direction: Redesigning Knowledge Track Creator For Content Authoring
A product strategy decision
Given the evidence that assigning Knowledge Tracks was the primary pain point and highest impact opportunity for the effort, the team investigated this challenge first. But several weeks into the redesign, a major technical dependency was discovered that required engagement from another team. Because of roadblocks in obtaining this resource, we decided to find another opportunity to impact this space.
While the assignment aspect of the LMS was causing pain for administrators, some of that difficulty came as a result of the Knowledge Track Creator. While its usability issues were highly visible, smaller process issues like lack of reusability or ability to manage updates made the system inflexible and created manual tasks for administrators.
We chose to pivot to a rebuild of the existing Knowledge Track with three main goals in mind: refactor the underlying technology, rebuild existing functionality with improvements, and improve overall usability and visual design.
What are knowledge tracks?
Our customers want to create learning experiences for their employees. Knowledge Tracks are containers for educational content. Education administrators can add content from MedBridge’s own library of training videos, as well as include their own custom content in several formats.
Addressing user pains in the old Knowledge Track builder
Several issues were apparent and needed to be addressed in the rebuild.
- The old Knowledge Track builder used a monolithic code base, requiring a complete technical rebuild before making significant changes.
- Content was created with a use-once mentality. In the old Knowledge Track builder, as content was created, each section was given a due date. This worked for one-time content, it was impossible to have multiple cohorts using the same Knowledge Track on different schedules. This led to administrators creating multiple copies of the same knowledge track that could be assigned to different cohorts.
- It was difficult to manage changes and their impacts. In the old tool, a Knowledge Track was a record of completed content. Any change made to that content marks previous assignments as incomplete. While sometimes desirable, many administrators either did not understand this change or were not adequately communicating the change to their learners.
- The old quiz builder couldn’t reuse content, so quizzes needed to be rebuilt manually, question by question, when reused in a different knowledge track. Usability issues made it difficult to create and organize content.
