Both Ruth Colvin Clark and Myra Roldan are evidence-based learning designers who combine a nuanced understanding of learning theory with years of hands-on experience developing solutions.
Ruth is principal and president of Clark Training & Consulting and has spent her career as an instructional psychologist, working with diverse organizations. She’s the author of many articles and books, including Evidence-Based Training Methods, and she co-authored e-Learning and the Science of Instruction.
Myra Roldan is a technologist and learning professional who currently serves as chief cloudification officer at Amazon Web Services. She does a bit of everything, from research and data analytics, to implementing new technologies, to creating full-blown learning experiences. Much of her personal work is focused on increasing access and opportunity for underserved groups, including women and minorities, so they can get jobs that will earn them a livable income.
In this fourth episode in our seven-part series on learning science’s role in a learning business, we speak with Ruth and Myra individually to get their perspective and expertise on designing effective learning experiences. We cover a variety of topics including research-based principles and their implications on learning, tips for working with subject matter experts to ensure learning objectives are met, how to avoid cognitive overload, and the importance of knowing your audience.
To tune in, listen below. To make sure you catch all future episodes, be sure to subscribe via RSS, Apple Podcasts, Spotify, Stitcher Radio, iHeartRadio, PodBean, or any podcatcher service you may use (e.g., Overcast). And, if you like the podcast, be sure to give it a tweet.
Listen to the Show
Access the Transcript
Download a PDF transcript of this episode’s audio.
Read the Show Notes
[00:28] – Intro/preview of episode.
Part 1: Ruth Colvin Clark
[01:44] – Background information about Ruth.
The Value of Evidence-Based Practice
[02:19] – You’ve done a lot of work to bridge the gap between academic research on instructional methods and then practitioner application of that research. How do you explain the value of evidence-based practice?
A lot of money is invested in learning events, and there is limited time to deliver training events, so you need to maximize the value of that training. One of the best ways to do that and to get a return on investment is to draw on instructional methods that have actually been empirically researched and proven.
[03:23] – How would you describe the value for the learners in participating in learning that is developed according to evidence-based practices?
Learners can get more value from a well-designed and empirically based set of instructional methods. Often in traditional types of training, the learners are passive; learners be more engaged by appropriate instructional methods.
[04:14] – Do you see more practitioners making sure that their decisions and designs are backed up by solid research than you did a decade or two ago?
The answer is yes and no. In the 1990s, evidence-based medicine emerged. As a result, practitioners involved in training the allied health sciences and in medical schools began to attend more to evidence-based learning. That gave evidenced-based practice a boost in that professional arena. However, in many cases, organizations have high turnover in training and limited learning professionals to provide guidance.
In some ways we’ve moved forward, but we still have large gaps and probably an ongoing challenge with promoting and disseminating the concepts of evidence-based learning.
Ruth Colvin Clark
Keeping Up with Learning Science
[05:30] – How do you personally keep up with new research and developments in learning science and the implications on learning design?
Every month, Ruth checks several journals (mostly academic) that publish fundamental research. There are 10 to 20 themes that she monitors and then files to reference when she’s preparing a chapter or a book. Most practitioners don’t have time to find and read original academic research, so she’s hoping to fill a gap by doing that reading for learners and summarizing the information.
[06:38] – What advice do you have for learning businesses looking to stay on top of learning science and learning-related research?
Ruth recommends that people look at books, online conferences, Web sites, and discussion forums on LinkedIn that are grounded with evidence-based practice.
She also suggests checking out businesses that help in making these kinds of translations, such as the International Society for Performance Improvement (ISPI) and the Learning Development Accelerator.
Designing for E-learning Versus Other Modes
[08:02] – You co-authored (with Richard Mayer), e‑Learning and the Science of Instruction. How different is it to design effective learning for different modes? What are the salient differences when designing for e-learning versus classroom instruction, for example?
For all media, there is a common body of research and guidelines that apply, whether you’re in the classroom or whether you are designing for e-learning.
As for differences, one of the main differences in e-learning is learner control. For learners using asynchronous e-learning, they can usually go at their own pace and go back and review something. However, in the classroom, typically learners have to go at the instructor’s pace. Also, with technology, it’s a little easier make use of simulations or immersive virtual reality. The classroom has great opportunities for social presence.
All media have strengths and weaknesses, and perhaps sometimes the best solution is a blended solution, where you combine some asynchronous e-learning with some synchronous classroom-led training.
[10:07] – Is there a gold standard in terms of mode?
Research over the past 20 years or so has shown that e-learning has steadily grown in terms of the proportion of its use, but there, of course, continue to be examples of classroom training. Ruth thinks we should try to blend the best of all worlds and involve classroom as well as e-learning. Note the term “classroom” doesn’t always have to be the physical classroom since we now have technology for virtual classrooms too.
Sponsor: SelfStudy
[11:53] – If you’re looking for a technology partner to help you effectively engage learners online, check out our sponsor for this series.
SelfStudy is a learning optimization technology company. Grounded in effective learning science and fueled by artificial intelligence and natural language processing, the SelfStudy platform delivers personalized content to anyone who needs to learn either on the go or at their desk. Each user is at the center of their own unique experience, focusing on what they need to learn next.
For organizations, SelfStudy is a complete enterprise solution offering tools to instantly auto-create highly personalized, adaptive learning programs, the ability to fully integrate with your existing LMS or CMS, and the analytics you need to see your members, users, and content in new ways with deeper insights. SelfStudy is your partner for longitudinal assessment, continuing education, professional development, and certification.
Learn more and request a demo to see SelfStudy auto-create questions based on your content at selfstudy.com.
The Coherence Principle
[13:01] – What is the coherence principle? Any suggestions for how to avoid the pitfall of adding extra material?
A lot of times the information and the skills and knowledge that we are charged with creating learning around are not the most stimulating or exciting. Yet we’re all used to being immersed in high-intensity media, games, etc., so it’s tempting to try to spice up or elaborate on content. Below are some common dimensions that can make it harder for learners to learn:
- Graphics
Often a simpler graphic is better, easier to produce, and ultimately more effective instructionally compared to a high-end graphical interface. - Stories
Stories are often memorable, but, if they are not directly related to learning outcomes, they can become distractions and disrupt learning. - Subject matter experts
Because they know so much, SMEs often want to provide everything there is to know about a certain topic. Instructional professionals need to narrow down “need to know” versus “nice to know.”
[14:34] – Do you have any suggestions for how to work with subject matter experts to whittle away the non-essential and get to the core content?
- Give the SMEs the learning objective, and then ask if the story (or other content they’re suggesting) supports those objectives.
- Have groups of people review the material.
- Act as an editor yourself, and cut out extraneous materials.
The Redundancy Principle
[15:50] – Would you tell us about the redundancy principle?
The redundancy principle refers to a situation when you have a screen or slide with a graphic of moderate to high complexity and you use text and an audio narration of that text to explain the graphic. This is redundant because the graphic goes into the visual center of a learner’s brain, and the on-screen text is competing with that limited visual resource. It can also be out of sync with what the learner’s natural reading rate is. All of this creates a higher cognitive load.
Strategies for Chunking and Segmenting Learning
[18:17] – What advice you have for how to effectively chunk or segment learning?
A huge advantage of asynchronous e-learning is that learners can control their own pace. This makes it possible for them to also segment what you have provided. It’s much more difficult to do this for something instructor-paced.
Keep your target audience in mind. If your learners are experienced in the content being taught, then the chunking principle is not quite so crucial because they themselves can manage greater amounts of instructional content. However, when you have novice learners, you should employ a variety of techniques to chunk and segment. For example, use a lot of white space. You can include just one or two words on the screen and use narration to elaborate on it. For complex graphics, you can build them gradually, explaining key points as the build happens.
I always would recommend err on the side of having shorter amounts of content or slides or screens in a topic. I myself, even in my pleasure reading, I’m more and more drawn to books that have short chapters. I just find it a whole lot easier and more enjoyable to read and review it. I think in learning, when you have relatively short little topics, you feel a sense of achievement, and you don’t feel so overwhelmed by the long, lengthy lesson.
Ruth Colvin Clark
Outlining and Storyboarding
[20:40] – Are outlining and segmenting related concepts in getting clear on the content you’re covering?
Outlining is one very powerful tool, and it’s a good place to start because it’s a relatively straightforward technique. Start with the learning objectives, next the content, and then use the outline to decide how to break it down into modules, lessons, or other divisions.
Storyboarding is another useful tool. Start with the outline, and then sketch out storyboards to decide where to show a graphic, etc. Start at a high level first, and then break out more detail.
[21:33] – A reader can look at the table of contents in e-Learning and the Science of Instruction, and it serves as a review session because it lists all the topics and subtopics.
That was an evidence-based technique. If you start a lesson or a chapter with a quick outline, that serves as an advance organizer and it helps with the reading or the learning process.
A Tenet of Effective Learning
[22:05] – What aspect of effective learning do you wish was more broadly understood and supported by those designing and providing learning to adults?
One of the major understandings that would help learning designers and providers is to appreciate the limits and strengths of working memory. We have two memories: working memory and long-term memory. Leveraging these two memories effectively leads to a lot of the instructional techniques already discussed. Understanding the fundamentals of the mental processes involved in learning and how to accommodate those would be a useful design understanding.
[23:37] – Is there anything else that you’d like to share?
The more we can promote evidence-based practice, the better off we, as a profession, will be, and the more we will grow. Also, keep in mind that research evolves, and hopefully we will continue to have people disseminating that evolving information for practitioners.
Part 2: Myra Roldan
[24:41] – Background information about Myra (who we heard from in our last episode on market assessment and needs assessment).
Working with SMEs to Incorporate Learning Design
[25:38] – What advice you have for working with SMEs who don’t necessarily know anything about effective learning design?
Subject matter experts don’t need learning design experience—that’s your job, as a learning provider and designer. Myra suggests doing the following when working with subject matter experts:
- Ask good questions, such as, “What does someone who is new need [to the field, profession, industry, role, etc.] to know?” or “What are the [X number of] things that someone who is new needs to master first?” Then break those steps and points down.
- Value the expertise of SMEs, but also help them narrow the content down to very specific areas.
- Take control of and guide the conversation. Start by asking questions such as, “If I didn’t know anything about this or if I was new, what things do I need to know in order to be productive immediately?”
- Have some back-up subject matter experts you can go to in case the one you’re working with doesn’t have the bandwidth to give you the time you need.
[28:11] – Are there other specific techniques, tactics, or questions that you like to use when engaging subject matter experts to help you get at that most essential content?
It varies from subject matter expert to subject matter expert, but, overall, Myra suggests:
- Educate SMEs, and help them put themselves back in the shoes of a beginner.
- Build trust with them.
- Make them feel like they are the subject matter experts, and that you’re trying to gather nuggets of knowledge from them.
- Let them know you value their time and the expertise they bring to the table.
Common Mistakes in Learning Design
[29:48] – What’s one of the common mistakes that you see in designing learning for adults?
One of the most common mistakes Myra sees is huge learning solutions that are drowned in theory and scenarios with too much information.
You end up throwing your learners into cognitive overload, where they don’t have sufficient time to actually process the information that they’re being given…. We tend to throw everything at people and then expect them to know what they’re doing. And we all know that there is a forgetting curve…. Even though we know that, we see learning being designed as so dense and thick…. And you get bored. You get lost. You don’t know what you’re supposed to be learning, and it’s not engaging. To avoid that, you should take a step back and put yourself in the shoes of the learner.
Myra Roldan
Design thinking can help. Ask yourself, “What are the obstacles that learners encounter when they’re using our solutions? And how does it impact their daily operations? How much work do they have to do on a daily basis, and how do we help lower the cognitive load?”
The Impact of New Technology on Instructional Design
[31:27] – What’s the impact of new and evolving technology on instructional design? Does new tech change how we design learning—or how we should design learning?
Myra explains that it does, and it doesn’t. You do need additional skills to work with the new technologies. But instead of learning how to create or program in the new technology, you should learn how to design for it—which is very different. It’s understanding how the tech works, but you’re not going to build it. To build, bring in an expert who knows how to program in that technology.
A main issue with a new technology is the cost. A lot of organizations want to integrate new technologies without really considering the cost of development and maintenance and the feasibility of integration into a learning space. Myra recommends that you do a feasibility study before you decide to use any of these new technologies.
Listen to Myra below where she shares how virtual reality has evolved and how it’s transforming the way we live, work, and learn.
Learning Science and DEI
[35:14] – What role do you see for learning science and learning design in the realm of diversity, equity, and inclusion?
Myra thinks learning science and learning design play very significant roles in diversity, equity, inclusion. Learning design is about how you’re going to help someone learn and what obstacles learners encounter and have to overcome to gain access to a specific training option.
When you think about learning science, you have machine learning, personalization, and AI. But how do you ensure that people are able to gain access?
I think where we fail is that we’re always trying to build new and shiny, the latest and greatest, but then we’re…widening the gap—the equity gap, the gender gap, the access gap, the opportunity gap gets wider when we start to integrate these new and exciting technologies that may not be accessible to someone who doesn’t have access to a computer and Internet.
Myra Roldan
Keeping Up with Learning Science
[37:43] – What do you do personally to keep up with new research and new developments in learning science and the implications that those might have on how you design?
Myra spends a lot of time doing labor market research to understand the competencies that employers are looking for in new employees. She also focuses on new tech developments. She doesn’t do a lot of research around learning science and learning design because she feels it’s too narrow of a perspective. Instead, she likes to reach out into other realms and explore what’s going on in psychology, technology, behavioral therapies, and other fields to see if there are things learning professionals should be looking at that are being applied effectively.
Advice for Incorporating Learning Science
[40:26] – What advice do you have for a learning business that’s looking to make good use of learning science in their offerings?
Myra’s number one advice is to know your audience. Don’t build a solution looking for a problem. Understand the problems that people are grappling with, and then align yourself to create solutions that will solve those problems. You want to build solutions around the needs of your audience.
[41:52] – Wrap-up
Ruth Colvin Clark founded Clark Consulting & Training and has written many articles and books, including Evidence-Based Training Methods and e-Learning and the Science of Instruction, that focus on translating research into practical guidelines for creating adult learning.
Ruth is generous with her translation work, and so you can find a number of free, one-page summaries she’s done of academic articles and their implications on the Learning Development Accelerator site. For example, she’s done one on the role of emotions in learning. Her short summaries may be sufficient, but she always provides the full citation, so you can also dig into the original research too.
Myra Roldan is a technologist, a learning professional, and chief cloudification officer at Amazon Web Services. You can connect with Myra on LinkedIn and at myraroldan.tk, where she has some free micro courses available on topics including using slides as a virtual background in Zoom, and she posts information there when she teaches courses that are accessible to the general public.
To make sure you don’t miss the remaining episodes in the series, we encourage you to subscribe via RSS, Apple Podcasts, Spotify, Stitcher Radio, iHeartRadio, PodBean, or any podcatcher service you may use (e.g., Overcast). Subscribing also helps us get some data on the impact of the podcast.
We’d also appreciate if you give us a rating on Apple Podcasts by going to https://www.leadinglearning.com/apple.
We personally appreciate your rating and review, but more importantly reviews and ratings play a big role in helping the podcast show up when people search for content on leading a learning business.
Finally, consider following us and sharing the good word about Leading Learning. You can find us on Twitter, Facebook, and LinkedIn.
[44:19] – Sign-off
Other Episodes in This Series:
- Learning Science for Learning Impact
- Effective Learning with Learning Scientist Megan Sumeracki
- Needs, Wants, and Learning Science
Episodes on Related Topics:
Needs, Wants, and Learning Science
Leave a Reply