Learning ecosystems, learning engineering, learning in the flow of life and work—these could be a list of top buzzwords. But each has gained attention for a reason. Each is important in the design and delivery of effective lifelong learning, continuing education, and professional development, and each gets its due in this episode, which features a conversation with Sae Schatz.
Sae formerly served as director of the U.S. Department of Defense Advanced Distributed Learning Initiative, and she was there when she first came on the Leading Learning Podcast. Though she no longer directs ADL, she continues to work at the intersection of human cognition and learning, technology, and data and is founder and CEO of the Knowledge Forge, which helps advance organizations through human performance and learning.
In this episode of the Leading Learning Podcast, co-host Celisa Steele and return guest Sae Schatz talk about learning in the flow of life and work, with Sae sharing insights from her work in Ukraine. Sae also shares her vision of how learning experience design, learning ecosystems, and learning engineering fit together to help organizations create and deliver effective products. Along the way, they touch on data analytics, competency-based learning, verifiable credentials, distributed identities, integration, privacy, artificial intelligence, and more.
To tune in, listen below. To make sure you catch all future episodes, be sure to subscribe via RSS, Apple Podcasts, Spotify, Stitcher Radio, iHeartRadio, PodBean, or any podcatcher service you may use (e.g., Overcast). And, if you like the podcast, be sure to give it a tweet.
Listen to the Show
Access the Transcript
Download a PDF transcript of this episode’s audio.
Read the Show Notes
Intro: [00:00:00]
Supporting Ukrainian Training and Education Planning
Celisa Steele: [00:02:04] I understand that you’ve been doing some work to support Ukrainian training and education planning. So would you tell us a little bit about what that work looks like?
Sae Schatz: [00:02:15] Sure. And I really just want to say I can’t take credit for this. I have been very impressed with what our Ukrainian partners have done during this really difficult time for them. As part of my official duties back when I was with the government, we collaborated with the National Defense University of Ukraine. And, as you might expect, when COVID happened, the military, as well as different universities and other organizations, had to shift a lot of the learning online. And then, of course, when the war broke out, there was this urgent need to leverage those online learning or other kinds of microlearning or just-in-time learning channels. For example, you have all these new people coming into the military or to do the homeland defense. How do you get education, training, or just-in-time updates to those folks? I’ve been really impressed with how they’ve been leveraging things—of course learning management systems, e-learning, YouTube, and also social media apps like WhatsApp or Telegram. And it’s a real excellent case study that I think we can all learn from about learning in the stream of work and life and, unfortunately in this case, in war as well.
Celisa Steele: [00:03:25] So, just out of curiosity, how did you get interested in or connected with this project at all?
Sae Schatz: [00:03:33] So in my previous life, I was in an official government capacity. The U.S. Department of Defense works closely with a lot of different allies and partners, and, because that role was related to technology-enabled learning, I was connecting to different technology-enabled learning folks from around the world. And then, since I’ve left government, of course, we’ve stayed in touch with various colleagues and with the Ukrainian work. There’s both this need to help support our Ukrainian partners in the technology infrastructure as well as in the courseware itself. But, then, also thinking about the future, and, okay, how do we find a silver lining here as far as leaping ahead and building infrastructure that is going to be even more modern, even more future-looking? Because there’s going to have to be a rebuilding effort anyway. Let’s not build a bunch of brick-and-mortar schools to recreate the old industrial age of learning in the future. What does the future of learning look like? And, if we’re going to have to rebuild anyway, how do we do that at a societal level that leverages the new technology, data-driven approaches, and a much more flexible, modern outlook on learning, development, and the workforce?
Learning in the Flow of Work
Celisa Steele: [00:04:45] You mentioned that you feel like you’re learning some lessons about learning in the flow of work and life. I’m curious what kind of takeaways you have from that work in Ukraine around how we might, in other situations that aren’t necessarily war zones, make sure that we are integrating learning into the flow of work and life?
Sae Schatz [00:05:08] That’s a great question. And I think that you really are asking the right thing, in the sense that we are all moving in that direction, if you look at some of the work coming out of, for example, Harvard on the 60-year curriculum or some of the others. We’ve heard buzzwords like “learning in the flow of work” or “just-in-time learning.” But it’s true—we’re all going to have to constantly be learning. And so I would say a few things to consider are, number one, learning experience design. How do we make it as easy as possible and as frictionless as possible for both instructors and staff and people who develop and deploy the learning, as well as the learners themselves, to access, find, and validate the learning? I think that is a really interesting and important area. I always see—and, of course, I think we’re probably all guilty of this—we get enamored by the new technologies. “Oh, my gosh, I can have virtual reality with augmented reality and all these other cool things.” Which is true. That has a place, and it’s really valid. But I think also just high-quality learning materials delivered in an easy, frictionless way can be really powerful.
Sae Schatz: [00:06:19] So learning experience design with high-quality pedagogy, or andragogy, instructional design components, number one. Number two, we are moving to a widely deployed ecosystem of learning. People are getting their instructional materials and their learning experiences from a wide variety of different places—YouTube, WhatsApp, learning management system, mentorship, a face-to-face thing. And so we really need to embrace that learning ecosystem approach as quickly as possible. What I mean by that is technology-enabled, data-driven learning that can work across a distributed series of different experiences and platforms. That really means integrating things like competency-based learning, verifiable credentials, distributed identities, and other components that will make that ecosystem work in practice. Number three, for all of this, is to embrace learning engineering. I think learning engineering is the functional, faculty-focused side of the learning ecosystem, which is more about infrastructure. We have this infrastructure that we’re able to connect across all these different learning islands and all these different learning experiences across time. But who is designing that content? Who is analyzing the data? How do we make sure that we’re doing that successfully? And that’s where I think, if we can all start to build up the bench strength in learning engineering, we’re going to be a lot more successful.
Celisa Steele: [00:07:54] Interesting. So three things: learner experience design, the ecosystem piece of this, and then the learning engineering. And so you already made the point that learning engineering is focused on the people delivering and developing the learning and really building up their capabilities in what they understand and what they’re understanding as they get feedback from the experience. The ecosystem being more infrastructure. And LXD would then, of course, be more of that focus on the learners themselves.
Sae Schatz: [00:08:22] Yes, exactly. I think all these need to go hand in hand.
We need to have that modern infrastructure that allows us to learn everywhere and to make sure that all of our learning experiences count and are integrated together, like a tapestry versus being like silos and disconnected from one another.
Sae Schatz
But, within that system, we can easily get overloaded. It can easily be discouraging if we don’t have good user experience. And then we obviously need to have quality experiences as well. Just because it’s easy to access a piece of instructional content doesn’t actually mean it’s good learning. So we need to have those learning engineers. Yes, I think that that’s probably the trifecta of what the future…and, of course, yes, it’s applicable in a high-stakes situation like what might be going on right now in Ukraine, but I think it’s very applicable just to the future of learning and development at large.
Sponsor: WBT Systems
[00:09:22] We’re grateful to WBT Systems for sponsoring the Leading Learning Podcast.
TopClass LMS provides the tools for you to become the preferred provider in your market, delivering value to learners at every stage of their working life. WBT Systems’ award-winning learning system enables delivery of impactful continuing education, professional development, and certification programs.
The TopClass LMS team supports learning businesses in using integrated learning technology to gain greater understanding of learners’ needs and behaviors, to enhance engagement, to aid recruitment and retention, and to create and grow non-dues revenue streams.
WBT Systems will work with you to truly understand your preferences, needs, and challenges to ensure that your experience with TopClass LMS is as easy and problem-free as possible. Visit leadinglearning.com/topclass to learn how to generate value and growth for your learning business and to request a demo.
Key Components of a Learning Ecosystem
Celisa Steele: [00:10:24] I know you have done a lot of work with learning ecosystems. You brought it up as one of the three pieces that you mentioned. But maybe we can just talk a little bit about what a learning ecosystem is because I think it’s one of those terms that gets thrown around a lot. It can be a little bit nebulous. So how do you describe or define a learning ecosystem? What are the key components?
Sae Schatz: [00:10:45] Sure. This is a good question, and I agree that the term has become kind of a buzz term. What I’m really talking about when I say “learning ecosystem” is that, if you think about somebody’s lifelong learning journey or, at the very least, their career-long learning journey, you have different experiences across time and different platforms and different subjects that somebody is experiencing. Ideally, we would like all of these different pieces to be connected together. And, from an organizational perspective, it would be great if we could see what people are actually learning and what their capabilities are, at least to the extent that privacy and ethics allow us to do so. But today what we usually find is, over there on the left side, we have a learning management system. Over here on the right side, we have face-to-face courses, maybe with a digital gradebook in Excel or something. Over here, we have something special with one virtual reality activity. Over there, we have some videos. And they’re all disconnected from one another. And so what ends up happening is that individuals have to interconnect the different pieces together themselves, mentally, which may or may not work depending on their self-directed learning or their metacognition. The different platforms themselves may collect data, but then it tends to be siloed, so we have to have analysts that try to glue it together to give us some insight into the workforce or our student body.
Sae Schatz: [00:12:12]
Basically, everything is, what I like to say, a learning island. They’re all separate from one another. They don’t talk to each other. So a learning ecosystem connects across these learning islands.
Sae Schatz
And, in addition to that syntactic interoperability, in addition to the technologies being able to talk to each other, it also has semantic interoperability. In other words, they speak the same language. In other words, using competencies and verifiable credentials so that we can understand what somebody learned in this space versus that space and are able to look in aggregate at somebody’s lifelong learning journey. Now, that somebody could be yourself, your self-sovereign identity, looking at your own learner records and doing your own learner planning as a person. It could be an organization. You’ve given them permission to see a slice of your experiences. Or it could be an institution. As an institution, you could be looking across the data of your learners, of your employees, and helping to optimize. So it’s really about the technology interconnecting, a data-driven approach, and something that’s at an enterprise or a more holistic level.
Celisa Steele: [00:13:18] So talk a little bit about how this might actually play out in the real world for an adult learner. I’m thinking, for example, that maybe my employer is providing some of the learning that I need. Maybe I’m part of an association that’s providing some other learning. Maybe I’m getting part of what I need through social media like LinkedIn and some of those resources. In your mind, are those multiple learning ecosystems, or is that still one learning ecosystem that that learner is part of?
Sae Schatz: [00:13:48] Well, I think it depends on your perspective. So who’s looking at it? I would say for, that learner, there is one learning ecosystem, ideally. This is the projection of what we would like the future to be. There’s one learning ecosystem, and so, whether you’re taking a MOOC, or you’re going to a face-to-face seminar or getting some education from the university, all of these different pieces are pushing data into your learner/worker wallet—so verifiable credentials that are going into your wallet that you can take to different places. And, again, we’re saying that this is the ideal future. So you take your learner record to these different places, and they can ingest it, with your permission, and then tailor those learning experiences to be most optimal to what you need. We can also imagine that in the future there might be third-party applications out there that you could feed your learner/worker record into, and it could recommend things to you such as, “Go get these credentials if you want to upskill into the next level of the profession.” “If these are your goals professionally, here are some different courses you could take.” Or “Here are some open jobs that we’re able to identify across society, across this larger learning and working ecosystem, that might be appropriate for you. Why don’t you apply to them?”
Organizational Involvement in a Learning Ecosystem
Celisa Steele: [00:15:05] And so, then, that’s the learner viewpoint in this ideal future where we’re headed, hopefully. If we think more about the individual organizational—I should say that’s confusing. We were talking about the individual, but now we’re talking about the organization and their viewpoint around a learning ecosystem. A couple of things occurred to me. One is that this idea of an ecosystem suggests nature. It suggests things that evolve organically and naturally. And yet it seems like there’s also a certain amount of the learning ecosystem that needs to be designed ideally for it to be interoperable and to really realize these benefits. So talk about it a little bit from that organizational standpoint and the level of control or thought that they might give to crafting their learning ecosystem.
Sae Schatz: [00:15:56] Well, what we’re seeing already is that organizations tend to have their own internal learning ecosystem, of course. And I would say that this is on a spectrum of less to more successful. On the least successful side, what I see are these systems that are probably not an ecosystem, in the sense that they are very hierarchical and very top-down. For example, you might have an HR IT system combined with a learning management system and maybe some sort of performance management system or digital gradebook. But they’re all very tightly coupled. You have one vendor that’s providing them. I would say that’s not really an ecosystem per se because the idea of an ecosystem is that you have different, disparate components that are able to interconnect together like Lego blocks, so you have a much more extensible, much more modular open systems architecture. So organizations can have that open systems architecture internal to their own organization, not intersecting with the outside world.
Sae Schatz: [00:17:01] And that is starting to get to a little bit more of the vision where you could have different suborganizations, different departments, each offering different components. Maybe individual workers can go out to that MOOC or that independent learning that they go do, and they can push back in the credentials that they receive from it, and that’s starting to get to that same vision. And then, at the optimal end, you still have that organizational control of those organizational learning opportunities, you still get the management of the data within your organization, but it intersects with the outside world, maybe pulling in official competency definitions from some central source or pulling in official job descriptions and looking at the available jobs across the society, those sorts of things. So you’re intersecting through application programming interfaces. In other words, having that open systems architecture that passes data back and forth. But you still have local control of your own systems and your own data, determining what is allowed to be shared and linking what is allowed to be shared to the outside world.
Common Organizational Misconceptions Regarding Learning Ecosystems
Celisa Steele: [00:18:20] What do organizations tend to get wrong when they turn their attention to a learning ecosystem?
Sae Schatz: [00:18:27] Well, it’s just a complicated thing. We’re talking about basically making the Internet for learning. It’s a pretty complicated, decentralized, and highly technical system. So there’s a lot to get wrong. But, at the same time, there’s a lot of promise. Some of the common misconceptions or challenges that I’ve seen are what I already mentioned—people building very tightly coupled, very proprietary, and not very extensible systems and then calling it an ecosystem just because it has different learning activities involved with it. Don’t think that makes it an ecosystem. And, worse than that, it makes it so that you tend to be very locked into what you currently have, technology-infrastructure-wise, and it’s difficult to upgrade or add components in the future, and you tend to get pretty locked into that one particular vendor or that one particular company that’s selling you software because it’s so specialized and tightly contained. The other thing that I see that people tend to get wrong is related to the data, and I think we all struggle with data at large. And, when we talk about human data, human performance data, or learning data, it can be really challenging to know what kind of data to collect. On the one hand, some people just don’t collect any kind of data, or they just collect completion data, which is not very useful for us. It doesn’t help us to know that somebody spent eight hours in a course if we don’t know “Did they actually learn something, or what gaps are in their knowledge?” Other times, I see people collecting all the data they can collect.
Sae Schatz: [00:20:00]
Sometimes I say it’s like a dragon sitting on its hoard of gold. It’s like, “I have all this data.” Well, what are you doing with it? “I’m sitting on it, and I’m admiring it.” Okay, it’s good that you have the data, but if you’re not using it for anything, then it’s just an expensive exercise in collection.
Sae Schatz
Of course, there’s a middle ground, which can also still not be very successful, it’s if you’re collecting some moderate data but don’t know how to analyze it in a way that’s meaningful. I also see this happen too. I see people say, “Oh, well, you know, I collected all this data about people’s preferences and what they scored on the quiz.” Okay, what does that tell you about the course or about how they learned? “Well, I’m not sure, but it’s 10,000 statements. So that’s pretty cool.” I think we just need to be really careful about making sure that we’re collecting the right kind of data, that we have good psychometrics, good rationale on how we’re collecting the data, and that we’re using it to drive decisions. Because that’s the only reason that we want to measure anything in this applied professional context anyway—so that we can make better decisions downstream, better decisions about what that learner needs, better decisions about future planning for a particular learner or a team, and better decisions about how to optimize a particular course or other learning activity.
Celisa Steele: [00:21:18] You mentioned learning engineering, and my understanding is that the data and making sure that you do something with this data that you’re getting back is a very integral part of learning engineering.
Sae Schatz: [00:21:30] Well, this is a great segue because that was the next thing I was going to talk about—making sure that you have the right team applied to the jobs. So learning engineering has a few different facets to it, and data is certainly one of them, but I would say it’s not the only one. Learning engineering is really about solving problems that can be solved through learning in some form or function, using learning science and human-centered engineering. Now, yes, that is usually data-driven. In fact, data is a core tenet of learning engineering, but it’s also about taking a holistic approach. And what I mean by holistic is really trying to say, “Okay, what’s my goal? What am I trying to solve? Am I trying to upskill my workforce because I need them to be more savvy about artificial intelligence?” If that’s the problem you’re trying to solve, then, yes, some of that might be solved through creating a curriculum around AI. But some components might be solved through just creating white space in the schedule so that they can go play with ChatGPT and Midjourney. There might be other ways to solve the problem or get to the goal besides solely taking a traditional instructional design approach. And then, again, like you said, we measure that to see, okay, are we being successful? And, if not, let’s tweak this, and let’s tweak that. So it also has this kind of continuous integration and continuous deployment approach, where it’s constantly being tweaked. You don’t just design a course, deliver it, and then be done, but constantly improve it every offering.
Celisa Steele: [00:23:00] Yes, I really appreciate that aspect of learning engineering. And we always like to talk about the fact that learning is a process and not an event. I think a lot of people, when they hear that, they think about the learner and that the learner needs to keep learning and all that. But I think the learning engineering viewpoint is also that the learning experience, what is helping those learners along the way, also needs to be revisited and to be improved upon over time.
Sae Schatz: [00:23:29] Exactly. And it needs to be thought about in context. If you’re delivering, let’s say, a seminar within an organization, that doesn’t exist in isolation. How does it connect to the other organizational goals or the other learning and development offerings? Are they supporting one another? Are they amplifying? Or are there places where there are gaps and we can connect them together better?
Where To Start When Building a Learning Ecosystem
Celisa Steele: [00:23:51] So, if an organization wants to get tuned in to its learning ecosystem and give that some attention and some thought, where would you recommend, or how would you recommend, that they get started? Are there points that are easier to get into or have greater leverage?
Sae Schatz: [00:24:10] I would say the first step in this is, number one, start to invest in your learning engineering bench strength. And there are a lot of excellent resources out there. I’m sure that some will be connected here in the show notes. I definitely recommend the Learning Engineering Toolkit [b Jim Goodell] as a good place to start.
Once you start to build up that bench strength, those folks will start to recommend different activities to better instrument learning activities, to collect data, to better use those data in learning analytics, and to better connect across different experiences. At the same time you’re doing that, I would say invest in doing better data-driven learning. Integration of the experience application programming interface (xAPI) so that you can start to collect interoperable data and just really starting to explore things like competency-based learning and verifiable credentials. That will get you started as you move down that journey. Starting to integrate some of the other infrastructure components is a good next step. For example, building out the xAPI capabilities, so you’re starting to collect data across different systems and being able to aggregate it together. Starting to move to interoperable catalogs of learning activities, so you’re defining the metadata of your e-learning offerings, your face-to-face offerings, or other kinds of learning activities. Making sure that you’re moving to a competency-based learning approach, defining those competency frameworks.
Sae Schatz: [00:25:36] And then starting to, at the very least, explore verifiable credentials. You’re going to continue to move down that pathway over time, and it can be gradual. Each of these different stops along the way will have benefits. So it’s not like you have to have that ultimate vision that we talked about a few minutes ago of this massive learning ecosystem that’s interconnected with the rest of the world. That’s going to happen sometime in the future, particularly for large organizations. But even just starting to interconnect across your own internal systems, with a more data-driven approach, with interoperable data, xAPI, competency-based learning, and so on, is going to get you a lot of value. As you start to move down that stream, you’re going to start to need to mature your internal infrastructure, integrating things like distributed identity, being very thoughtful and careful about your ethics, your privacy, and your identity management. Starting to do a better job at the learning analytics piece and really making sure that you’re making good decisions with the data and not just being that dragon sitting on your hoard of all this learner data and not doing anything with it.
Sae Schatz: [00:26:46] And then optimizing the systems over time. That’s where you can start to truly integrate things like machine learning because you’re going to start to have rich data. It can start to do things like path planning for individuals or even looking across your workforce, helping you do forecasting of what kind of jobs or development are going to be needed in the future. And, as you start to get to that, that’s where you can start to open up to some of these society-wide competency listings and connecting to them through linked data or starting to ingest verifiable credentials from third parties. That’s where you start to get a lot of power because then you can open up your learning and development to employees, letting them know, “Hey, you can go take courses anywhere. I just expect you to get to a three out of five in AI development expertise or a four out of five in this kind of program management leadership sort of thing.” And that’s where, again, you’re going to start to see so much flexibility and agility as an organization because you’re able to leverage not only your own offerings but the offerings across the whole ecosystem.
Personal Lifelong Learning Habits
Celisa Steele: [00:28:01] This is the Leading Learning Podcast, so one thing that we like to ask folks who come on is just a little bit about their own lifelong learning. And so we’d love to hear if you have specific practices, habits, or sources that you like to turn to help yourself grow personally and professionally.
Sae Schatz: [00:28:18] I am a big fan of social learning. What I mean by that is constantly coordinating with my peers and colleagues, looking at what they’re interested in, looking at what they’re writing, and talking to them. I find that some social and connective technologies—whether it’s LinkedIn, other kinds of meeting people online or at conferences—are really helpful. And then asking them, “Hey, what are you reading? What did you write? And what are you thinking about?” I found that LinkedIn is really a great resource. There’s so much content out there nowadays that it’s tough to stay on top of all of the trends, and it’s difficult to find quality a lot of times. There is a real information overload challenge, and that negatively affects our brains, too, by the way. It makes it really challenging to learn effectively when you’re under these effects of information overload. So trying to find the quality, the needle in the haystack, it’s challenging, I think, for everybody.
Celisa Steele: [00:29:27] I think your focus on social learning makes a lot of sense. And I heard in your response, too, that it’s a little bit like a recommendation. You have these trusted sources, and, if they can point you to a resource or an idea or share one of their own ideas, it helps, like you said, cut through that noise, and it’s coming from this trusted source so that makes it that much more valuable.
Sae Schatz: [00:29:50] Yes. And so I’m working on a book right now with a colleague of mine who you’ve had on the podcast before, Julian Stodd. We’re also working with Geoff Stead, and we’re working on a book related to learning science. So stay tuned for that. But applicable to what you just said, Julian works a lot with the notion of Social Age and how we are all moving to this more connectivist, social network approach to things, where somebody’s reputation, and, therefore, the trust and confidence you get in what they say, has more and more value because of this information overload, because of the way that the old hierarchical systems are breaking down and becoming much more of an ecosystem. I’m just going to keep saying “ecosystem.” So I think that’s really true. Whether you want to call it connectivist, whether you want to call it social learning, or the Social Age, it is this idea of being much more peer-to-peer and then leveraging those systems, whether it be things like LinkedIn, other kinds of communication technology, or some of these latest AI technologies, that help us skip past some of the informational pieces to get to the core of the answer.
Celisa Steele: [00:31:09] Is there anything else that has come to mind as we’ve been talking that you just haven’t had the chance to share, that you’d like to have the chance to air before we do wrap up?
Sae Schatz: [00:31:19] I think everyone right now is pretty enamored by generative AI and the promise that it brings, and I think that we don’t even know yet how those new technologies will change things. I’m a big fan of the SAMR model—the substitution, augmentation, modification, and redefinition model—that helps us think through how new technologies affect old systems. The punchline of that model is that often we just do substitutions of new systems into old systems, and not a lot changes. But, as we start to move forward and really advance, we start to rethink not only the technology but the whole processes around it. I’m not sure that we’re there yet with the impact of AI on learning. I think, going back to the beginning of what we talked about with the concept of the learning ecosystem and the concepts of data-driven learning and learning engineering, the latest kinds of AI, machine learning, and, even more particularly, generative AI are going to have profound effects on what we do.
As learning leaders, I think what we need to do is not only think about, okay, how do I update my infrastructure for this, how do I make sure my staff is competent at using generative AI, but really think about how this is redefining our systems, how this is redefining our processes, and to get ahead of that, not just from a learning and development perspective in the sense of instilling competencies or helping people develop learning—those kinds of shallow understanding—but at a comprehensive level. How do we rethink the way we do things? How do we rethink our systems?
Sae Schatz
Wrap-up: [00:33:14] Sae Schatz is founder and CEO of The Knowledge Forge, which offers independent consulting, applied research, keynote presentations, and custom workshops. Connect with Sae on LinkedIn here.
We’d also be grateful if you would take a minute to rate us on Apple Podcasts at leadinglearning.com/apple or wherever you listen. We personally appreciate reviews and ratings, and they help us show up when people search for content on leading a learning business.
Finally, consider following us and sharing the good word about Leading Learning. You can find us on Twitter, Facebook, and LinkedIn.
External Innovation and Learning Businesses
Leave a Reply