As an internationally recognized learning designer, speaker, and researcher, Dr. Patti Shank uses evidence-based analysis aimed at improving training, learning, and performance outcomes. She was an award-winning contributing editor for Online Learning Magazine, research director for the eLearning Guild, and is a regular contributor to eLearning Industry.
Patti is also author of the Deeper Learning series – Write and Organize for Deeper Learning, Practice and Feedback for Deeper Learning, and Manage Memory for Deeper Learning – which offer practical tactics for improving outcomes based on training and other research.
In this episode of the Leading Learning Podcast, Celisa talks with Patti about what it means to design for deeper learning, how to effectively use feedback and practice, and what learning designers need to keep in mind when it comes to memory and cognitive load. They also discuss the pros and cons of multiple-choice questions and tips for navigating research initiatives.
To tune in, just click below. To make sure you catch all of the future episodes, be sure to subscribe by RSS, Apple Podcasts, Spotify, Stitcher Radio, iHeartRadio, PodBean, or any podcatcher service you may use (e.g., Overcast). And, if you like the podcast, be sure to give it a tweet!
Listen to the Show
Read the Show Notes
[00:18] – A preview of what will be covered in this episode where Celisa interviews Dr. Patti Shank.
Reflection Questions
[01:27] – You might consider the reflections questions below on your own after listening to an episode, and/or you might pull the team together, using part or all of the podcast episode for a group discussion.
- Think about the feedback and practice opportunities you offer learners. Are you devoting the same time and energy to those aspects of learning as you are to the content when you develop educational offerings?
-
- Is the feedback you’re giving learners appropriate? Are you taking in to account the learners’ proficiency and the context?
[02:22] – Introduction to Patti and some additional information about her background and work.
Patti shares that it’s important to know that most of what she’s doing right now is writing articles on easily applicable, evidence-based tactics to get better outcomes from training and instruction. She’s also teaching online courses for people in the workplace learning practitioner audience to help them apply what the research says.
Deeper Learning
[04:04] – All three of your books include “deeper learning” in the title. What do you mean by “deeper learning”? How do you define “deeper learning”?
Patti explains that it gets confused sometimes because when you’re talking about AI or machine learning, that’s a specific subset. And she’s not talking about that, although they’re related.
She defines deeper learning as the research, which describes a continuum of learning approaches and learning outcomes, from shallow to deep, based on the intent of the learning and the strategies that are used.
As an example, shallow learning tends to be learning disconnected facts and concepts such as terminology, locations, or meanings of certain concepts. But it doesn’t go very far—it’s needed. And Patti talks about how shallow learning is where we start in many cases, using the example of learning medical terminology.
If you consider this as a continuum (not shallow and deep, rather shallow to deep), Patti says deeper learning is where you’re using all the things that have been taught (the facts, terminology, concepts, etc.) and putting the pieces together to help people make sense and understand how it applies to what they’re learning. So deeper learning is teaching for application.
And Patti stresses that we have to get to this point of application/deeper learning, but that a lot of times, we just don’t. She also says there are different strategies for shallow versus deep.
Feedback and Practice in Educational Offerings
[06:54] – Providing learners with the opportunity to practice, and offering feedback to learners, are two activities that I think our listeners would agree are important (and would seem to be a strategy on the deep end of the spectrum, helping people apply what they’re learning). But not all practice and not all feedback is equal. What issues do you typically see with feedback and practice in educational offerings?
Sponsor: Community Brands
[10:04] – Before we get to Patti’s answer, we want to pause to thank our sponsor for this quarter.
Community Brands provides a suite of cloud-based software for organizations to engage and grow relationships with the individuals they serve, including association management software, learning management software, job board software, and event management software. Community Brands’ award-winning Crowd Wisdom learning platform is among the world’s best LMSes for corporate extended enterprise and is a leading LMS for association-driven professional education programs. Award-winning Freestone, Community Brands’ live event learning platform, is a leading platform for live learning event capture, Webinars, Webcasts, and on-demand streaming.
And now back to Patti’s response about typical issues she sees with feedback and practice in educational offerings.
[08:49] – To put it in context, Patti says it takes a long time for people to become productive at workplace skills and core work tasks. And when you add non-core tasks (things outside the norm), it takes even longer. The problem is that before someone is productive, they typically cause problems/make errors/take a really long time to do their work.
Patti asserts that our role in learning and development is to create a path to proficiency. She cites the book, Learning Paths: Increase Profits by Reducing the Time It Takes Employees to Get Up-to-Speed by Jim Williams and Steve Rosenbaum, which discusses this notion of creating faster paths to needed proficiency. And that’s what deep learning is about—proficiency is about application but is also about application with accuracy and application with speed.
We spend far too much time just throwing content at people, actually making it harder for them to learn, making memory be challenged by what they’re trying to learn. So practice is one those essential and deep elements needed for proficiency. Yet Patti asks, how many courses have you seen that are 95% content and 5% practice? And the practice isn’t even realistic.
She says we worry about developing content for people. But we don’t worry as much about developing adequate and accurate practice at the level of proficiency they’re at so we can move them up from non-proficient to partially proficient to adequately proficient.
We deliver far too little practice for any level of proficiency, in fact no proficiency. And we generally deliver the wrong types of feedback to help people become more proficient. So we actually create problems for proficiency rather than creating proficiency when we do this wrong.
Patti notes that one of the things she hears on a regular basis is that this is all academic stuff and not applicable. But that’s simply not the case—the actual research might be academic and it may be hard to understand, but this is why she’s doing the work she’s doing. This is how we get people to where they need to go, faster and with better results.
Related to practice, Patti adds that it’s generally too generic and not in the context of work. But we have to teach and have people practice in the context of work because if we don’t, it’s hard or impossible to remember.
Memory as a Constraint for Learning Designers
[14:12] – In your article, How To Design To Help Working Memory, Part 1, you assert, “One of the largest constraints that learning designers and developers don’t realize that they have when working with instruction is the nature of our memory.” What do you mean by that statement, that memory is a constraint for learning designers?
Patti discusses how memory is what John Sweller – one of the leading researchers in the field – calls human cognitive architecture. And our human cognitive architecture is how we think, how we problem solve, how we learn, etc. Our memory has different parts and each part has attributes and limitations. The major aha here is that we have to work within the constraints and attributes of our memory parts; otherwise we make it harder to learn, not easier.
She uses the example of working memory, which is the part of memory that does processing of new information. The most important to thing to know about this is that it has extraordinarily limited capacity. This is why we get overwhelmed so easily when people are going on and on using fire hose training methods. We simply can’t learn that way—it’s not that our brain doesn’t prefer to learn that way, we simply cannot.
So we have to design all instruction according to the attributes and limitations of working memory. This has huge implications, one of them is known as cognitive load and we have to design with this in mind—so what overloads us and how we prevent that.
The other big part of memory to keep in mind is long-term memory. This is what’s called in research, prior knowledge, or what you already know, and how it’s organized in your brain. We have to design so we don’t overload working memory and we have to develop accurate and adequate organizational structures over long-term memory so that we can use what we know to learn and apply.
Patti stresses this is our job and we must do these things or we are actually becoming a bottleneck in people’s learning, which is a terrible thing for L&D professionals to be.
Efficiency and Effectiveness Tradeoff
[17:28] – Sometimes there ends up being sort of a tradeoff between efficiency and effectiveness. And you’ve written about this before, that some of the approaches you’re talking about do make for more effective outcomes/better educational results but often maybe get short shrift because people either believe or because they are less efficient or take more time?
Patti notes that one of things she regularly says is that doesn’t matter if it’s efficient if it’s not effective. So we have to look for the sweet spots. And being effective doesn’t mean it has to take longer. In fact one of the ways that we can be both efficient and effective is by giving people the broad foundational knowledge they need to keep learning on their own.
This is one of the strategies that we must take with people who are brand new to a given topic or given set of knowledge and skills. We need to figure out what it is they have to understand that will make it easier for them to learn on their own on the job because we’re not going to teach anyone everything.
With people who are brand new, you need to look at what the foundational concepts, terminology, facts, etc. are and how to use those for foundational skills so they can achieve learning on their own. And Patti argues this isn’t rote knowledge—it’s the foundation. She relates learning medical terminology to illustrate the importance of this.
Patti reiterates that we can’t teach people everything and uses an example of taking six weeks to teach people to take customer service calls. She explains that if we don’t follow certain well-known processes, they will have forgotten most of what they’ve learned at the beginning of those six weeks—and struggle at the end because of it.
So there are better ways to do this. You can give people what they know so they can become minimally proficient, let them start working, and then bring them back. And again, this isn’t just related to academics, it’s that people inacademics are the ones who have studied this.
What is the Most Helpful Feedback?
[22:23] – We briefly touched on feedback and the takeaway there was related to the wrong feedback. Can you expand a little bit on what that means and what is the type of feedback that typically gets given versus what would be more ideal or helpful in terms of learning?
Patti admits this is incredibly complex and interesting and that feedback is very nuanced.
She frames her discussion about feedback, limiting it to people who know less/have less prior knowledge. She also focused on electronic feedback (as opposed to one-on-one feedback) and shares the research related to 3 types of feedback:
- KR (Knowledge of results) – Just says whether you got the answer right or wrong.
- KCR (Knowledge of correct results) – This tells not only whether you got it right or wrong, but tells you what the correct answer is.
- Elaborative – tells you why (and there’s a whole host of things that can go into this).
The research shows (and again, this is very nuanced so doesn’t apply to every situation) that the best feedback for most people is knowledge of correct answer (KCR). The knowledge of results (KR) is not helpful. For people who arebrand new, the research shows that they can’t handle a lot of elaborative feedback.
It was also found that KCR was more effective when showing it in the guise of how it was shown in the questions. Patti discusses the implications of this related to choosing a multiple choice system. And for more advanced learners, she talks about how different it is for people who know less.
Pros and Cons of Multiple-Choice Questions
[27:57] – Multiple-choice questions are a particular flavor or subset of instructional writing where you’ve done some deep diving. What do you see as the major pros and cons of multiple-choice questions?
Patti talks about how it was never her intention to write courses on multiple-choice questions but because they are used so much and she realized people weren’t doing it well, she did. And she couldn’t really find other people teaching it.
She says the pros of multiple-choice questions are:
- They are extremely efficient as an assessment method. This is important because if it’s efficient and done well it means that we can assess far more content in a shorter period of time.
The cons of multiple-choice questions:
- Research shows that most multiple-choice questions are poorly written (because it easier to write poorly written ones). For example, recall only. And poorly written questions tend to be invalid for the assessment purpose, which starts getting into legal hassles. You can’t use assessments to make judgments about people’s learning or proficiency unless the questions and the assessments are valid. So they have to be written well. Patti stresses people must be able to do this and that it’s an essential and critical skill for anyone who designs instruction (and that’s why she got into teaching about it and designed the course about it).
Patti shares that her course is heavily hands-on so she limits the number of people to somewhere between 8-20. And she completely devotes herself and all of her time to helping the people over the duration of the course (which is several weeks long).
Multiple-choice questions are fine. What’s wrong with them is how they’re written.
Approaches to Research
[33:24] – You’re all about evidence-based approaches and looking at the research. Which is great for making sure we’re doing the right things as providers and designers of learning. But it can be slow—that is, conducting a well-run, broad study about any particular aspect can take time, and then there’s the time to write up findings, and then there’s time to get it reviewed and put out by a peer-reviewed journal, etc. So, given that lag, I’m curious to know what question about learning or learning design are you eagerly awaiting research on?
Patti admits that she’s never not found answers when she’s looked for them because research tends to not answer extremely specific questions. She uses the example of answering the question, is video good for learning?
The research answers a bunch of related questions but she says when you’re looking for specific research/answers, you have to go up in scale and down in scale and you have to know the words to use. In her eLearning Industry article, Does Video Improve Engagement and Learning?, Patti discusses how she found the answer and that it was related to what she hit on.
What’s interesting, she says, is that when you’re doing research reviews, you’re not looking for one study, you’re looking for a preponderance of studies on a specific topic. And the topic was attention span during learning.
So when you’re studying the research, the first thing to do is learn how to study the research. Patti notes that one of the major issues is related to key words and she shares a couple examples to illustrate this. She also talks about her experience writing her book, related goals, and what she learned.
[43:10] – What is one of the most powerful learning experiences you’ve been involved in, as an adult, since finishing your formal education?
Patti shares about her experience teaching her recent online course (related to writing assessments) and how the questions of the participants led her to have to do a great deal of research and learn so much (something she didn’t initially anticipate).
[46:57] – How to connect with Patti and/or learn more:
- Website: https://www.pattishank.com
- eLearning Industry Articles: https://elearningindustry.com/members/patti-shank-phd
[48:00] – Wrap-Up
Reflection Questions
-
- Are you focusing on practice and feedback as much as content when you develop educational offerings?
If you are getting value from the Leading Learning podcast, be sure to subscribe by RSS, Apple Podcasts, Spotify, Stitcher Radio, iHeartRadio, PodBean, or any podcatcher service you may use (e.g., Overcast).
We’d also appreciate if you give us a rating on Apple Podcasts by going to https://www.leadinglearning.com/apple. We personally appreciate your rating and review, but more importantly reviews and ratings play a big role in helping the podcast show up when people search for content on leading a learning business.
And we would be grateful if you check out our sponsor for this quarter. Find out more about Community Brands.
Finally, consider following us and sharing the good word about Leading Learning. You can find us on Twitter, Facebook, and LinkedIn. We also encourage you to use the hashtag #leadinglearning on each of those channels. However you do it, please do follow us and help spread the word about Leading Learning.
[50:05] – Sign off
See Also:
5 Pillars of Modern Lifelong Learning
Leave a Reply