NMSI Online Learning Experiences
To comply with my non-disclosure agreement, I have omitted confidential information and assets in this case study. The information in this case study is my own and does not reflect the views of the National Math and Science Initiative.
Organization
The National Math and Science Initiative is a national nonprofit dedicated to transforming math and science education.
The Challenge
In an effort to expand the organization’s offerings to reach more students, we needed to develop online learning experiences that were comparable to (if not better than) the organization’s face-to-face experiences. We also needed to build these experiences within the limitations of older technology systems.
A screenshot of a presentation I led illustrating the issues with our various technology platforms
My Role
I joined NMSI in 2018. As the Manager of Digital Learning and Instructional Design, I was responsible for setting the vision for digital design and online learning delivery of nationally-recognized K12 programs.
This was a cross-functional project that involved different teams:
Program Design: responsible for designing and developing the overall learner experiences
Coach Development: responsible for training our coaches who would be developing and delivering content
Program and Delivery Operations: responsible for the logistics of registration and onboarding
Technology: responsible for ensuring our systems were connected and configured correctly
Data Analytics and Evaluation: responsible for collecting and evaluating information about our learners and programs
Program Management: consulted on the design and provided valuable insights on our schools, they were in the field and closest to the end users
MarCom: consulted for the overall branding and messaging of our content
Finance: consulted to ensure the solutions we were developing captured key information needed for grants
The Approach
This project would be implemented in phases, starting with pilots in select schools. (When the pandemic happened, it moved up our timeline significantly. The vast majority of our offerings are online now.)
User Research - Stakeholders
My first step was meeting with key stakeholders across the organization to learn about their experiences using our existing tools and technology systems, as well as their attitudes toward online learning.
User Research - Audience
My next step was gathering information about our end users: educators and students.
Online learning was a new endeavor for the organization, and I wanted to hear from the users who were already participating in our face-to-face learning experiences. I also wanted to learn more about their experiences with online learning and technology.
In addition to analyzing existing information about our users, I developed surveys and conducted focus groups to gather additional insights about our end users. Both educators and students were quick to offer suggestions on the types of online learning experiences they’d like to see the organization develop.
A sample user persona I developed
Building Team Knowledge and Capacity:
One of my initial findings was there were varying levels of comfort using our technology tools, and limited standardization and alignment in our processes. I developed instructional resources and standardized templates to support adoption of our different tools. I also facilitated cross-functional brainstorming and collaborative design sessions that leveraged design thinking to create and maintain a shared view of users among various stakeholders. I made recommendations on how our platforms could be improved and better leveraged to provide a better user experience.
A screenshot from a presentation I led recommending our digital platform strategy
A screen shot of a web page I developed highlighting our different tools
A screenshot of a website I developed with instructional resources on using our learning management system
Prototyping and Developing
After sharing user insights with the rest of the design team, we worked within our design constraints and guiding principles to develop several prototypes that provided a combination of synchronous and asynchronous learning opportunities for our users.
User Testing
We presented our prototypes to other teams across the organization for their input. We conducted additional focus groups with users to capture their input on our prototypes and/or proposed design changes. We further refined our designs based on this feedback.
Iterations
We surveyed users before, during, and at the end of our online experiences. We used this information to refine our experiences year after year. Overwhelmingly, the data suggested users found our resources and support valuable, but our technology systems were negatively impacting their overall experience.
We also held debriefs at the end of every project to discuss how our internal processes and workflows could be improved.
Results
With our team’s recommendations and user feedback data, our learning management system was upgraded from Blackboard Original to Blackboard Ultra, which provides a more user-friendly and intuitive experience. Courses and content will be revamped to fit the updated system. We have also set up recurring meetings with our technology platform vendors to make continual improvements to our systems.
This project also highlighted the importance of improving cross-functional collaboration and communication. Teams and processes have been restructured to better execute the online delivery of our programs. The SharePoint website I developed for our team has been adopted as the organization’s template for team knowledge management.
More people have the understanding and confidence to use our online tools. I have also observed an increase in departments across the organization leveraging design thinking methods, such as creating user personas.
Additionally, our work in developing online learning experiences laid the foundation for other products and services to move online. When the 2020 pandemic happened and schools closed down, the organization was uniquely positioned to move all face-to-face events online and offer support to millions of educators and students nationwide.
Our Online Student Supports were credited for increasing students’ Advanced Placement exam scores and increasing college readiness. In some instances, educators and students felt they learned more from online sessions compared to their classes.
Our Online Summer Series Conference for educators was the largest and most attended online event for the organization. More than 10,000 educators registered, resulting in about 40,000 unique enrollments in over 100 courses.
The organization will be hosting its second online summer series in Summer 2021, with the advantage of additional planning time and concrete lessons learned from this experience.
If you’d like to view a gallery with additional assets from this project, click the button below.