Tuesday, November 07, 2023

DevLearn 2023 - Las Vegas (Show Notes!)


DevLearn has always been my favorite L&D, elearning-nerd out conference. It's where I first hit the stage back in 2009 with my Kineo colleague with Stephen Walsh and then most every year until about 2019. And then we had a pandemic and I sat out a few years. 2023 was my first DevLearn since 2019 and it was really great to be back!

Photo of four smiling individuals in front of a conference registration desk with the Kineo logo behind them.
  • Kineo was a proud sponsor  of DevLearn this year and it was super cool seeing our logo at the registration stage and on the big screen.
  • Amazing to be on the ground with three of my Kineo colleagues: Joe Replin (Technical Consultant coming to Vegas from Chicagoland), Rodrigo Bolanos (Head of Kineo LATAM from Argentina), and Vicky Bartolacci (Kineo Managing Director all the way from Sydney Australia). 
  • So great to connect with so many amazing people - old friends and new connections made. 
  • Data and Analytics are top of mind for people and extremely low on the L&D team's list of skills. Orgs and individuals are eager to build this capability.
  • AI was the darling - and it's coming soon to a workflow near you if it hasn't already. AI is transforming how we work and what we'll be delivering to learners. Game changer. People are feeling a lot of uncertainty and fear and...yes, excitement. Hold on to your hats, the ride is just getting started.
  • Ooh - and I signed a whole bunch of copies of The Accidental Instructional Designer: Learning Design for the Digital Age. Always fun to connect with readers and fellow accidents.
  • And finally, Vegas has amazing food - no doubt. AND I got to experience the Sphere (we didn't splurge for U2 tickets, but we did see an amazing $90 movie! Hold on to your seats, it was awesome.)

Here's a run down of the sessions I attended (and some I led!).

Morning Buzz – AI and Your L&D Role

Josh Cavalier

Josh is a big AI player right now in the industry, definitely one to follow.

Don’t wait. Start now. Things are changing FAST. We don’t know where things are going to be in 3 months, let alone a year. Everything is changing at rapid pace, but we do know that AI is here and it’s disrupting everything.

Lots of fear in the crowd – “what’s this going to do to my job?”

Get a corporate policy in place.

Policy > Manifesto > Playbook

Manifesto – how your L&D team is going to use AI

Playbook – what tools you’re going to use and how – and then meet on that playbook monthly (this idea actually came from Marcus Bernhardt who was in the crowd – also an AI guy)

Opening Keynote: Phil Rosenthal

 

Phil Rosenthal was a producer/writer of the show "Everybody Loves Raymond" and now has a show on Netflix - 7 seasons - "Somebody Feed Raymond" where he travels the world and eats good food and makes people laugh.

 

Use your constraints - working within your box can unleash your creativity.

 

The show Raymond - they wrote it to be intentionally timeless. Keep out any time bound references (e.g., you don't mention Bill Clinton in the 90's) and you make the content RELATABLE.

 

The best stories are those that are relatable. As a storyteller, satisfy your own interests and be specific. When we are specific, people are more able to relate to your story.

 

"Do the show you want to do because we all get canclled."

 

Creating a Robust Company Learning Culture with Values-Driven Development

Jennifer Gilley, Anuvu; and Amy Wheldon, Go1

Building a values-driven learning culture at an org obviously needs to start with stated values. Then follow a methodology:


  1. Build an authentic FOUNDATION that centers on philosophy, values, and the employee experience.
  2. Execute meaningful form - uncover the opportunity, informed strategy, culture shift, feedback loops
  3. Leverage Effective tools - training and delivery, input and output

FOUNDATION

Anchor in your beliefs and values.

Your org's learning philosophy informs your L&D strategy.

  • Gather input from stakeholders
  • Map out what learning looks like
  • Write a philosophy statement (who, what, why)
  • This is high level - no perfect formulas for this
  • What does learning look like and what does it mean in your organization?

Core values:

  • How do you integrate these values into your culture?
  • Incorporate these values to inform training focus and to elevate the employee experience

The Employee Experience:

  • Journey mapping
  • What does it feel like to work here?
  • How can we create an environment where people can truly grow?
  • Where can we partner across the business as reinforcement?

L&D Program is really about change management - a form to drive change. Think about:

  • Opportunities
  • Strategy
  • Culture Shift
  • Feedback Loops

Uncovering Opportunities:

  • Use results from annual and pulse engagement surveys
  • Look at performance metrics - review SLAS - to identify gaps around skills/competencies
  • Get stakeholder feedback - informal and formal to round out data points

Create an Informed Strategy:

  • What are our biggest challenges?
  • Prioritize based on those that will have the greatest impact
  • ID groups that are most impacted
  • Build from competencies
  • Automate training
  • "This is how the new L&D program is going to support…"

Culture Stuff:

  • Model enthusiasm - leverage positive psychology, mirroring, cascade across and down all levels of leadership
  • Overt communications
  • Shared accountability - tracking with targets to meet

Feedback Loops:

  • Formal tools to get employee feedback
  • Leverage managers for informal feedback and formal
  • Leaders to demonstrate the new behaviors (modeling) and do check-ins with teams

Tools:

Your LMS, facilitators, surveys, what they called "elevators" (ways to elevate the experience - whiteboards, polls, gamification of sessions).

For compliance training - if you're trying to actually change culture, think about how these behaviors ladder up to your values.

For leaders/managers - provide them with a check-in template (for meetings, conversations)

When change happens, values usually don't change. But execution does chage. Ask "How do we define what it means to exhibit our values?"

The Hitchiker's Guide to AI

Matt Donavan, GP Strategies

Start first with what the work is you need to do (not the tools, etc)

Put up Enterprise Guard Rails:

  • Responsible Use of AI
  • AI Policy
  • Approved Tools

Guidelines for responsible use of AI:

  • Maintain privacy and security (how identifying data being used)
  • Ensure safety and well-being (provide guidance)
  • Embrace DEI (how well does the AI work in different regions of the world 0 does it worry the same in Japan and Brazil….
  • Drive accountability (is the content generated accurate, do you have the rights for copy and or images being managed. Does it meet quality standards?
  • Convey Transparency: let people know when you're using it - don’t be ashamed, but open.
  • Promote Fair and Equitable: does it bring people together. Does it drive equity?

GPT4 sitting in an azure environment. Put all of your past RFPs on your system - point it to meaningful data from historical RFPs. (but remember most org's data is going to be a hot mess - may require a lot of mopping up afterwards).

Your policy might be "don't put high-risk data onto that…"

Training Data Set Through 2020 (?)

Constitutional Training (how you set these ecosystems up).

Need both human guidelines in place and the how we're structuring machines

How can AI be used to support a manager in the timeline completion of quality of performance conversations?

Performance Management Conversation Workflow….where could AI support this workflow?

  • These are the tasks in the workflow: Remember (to do it!), schedule, locate prior documentation, prep or refresh, coaching or mentoring, conducting the conversation, complete the documentation
  • Could you have AI help with those sticky spots? AI could check if you've scheduled it. It could go out to your system and get feedback from people, then summarized the people - then it locates the docs for you, then brings in other data and info.

Morning Buzz – eLearning – what do people really want and need?

Cammy Bean, Kineo

This was an informal conversation about learner needs – and the reality of what’s going on in the real world.  Individuals and orgs are at all levels of maturity in terms of elearning. Lots of Storyline use. People leveraging YouTube videos into course content (why recreate what’s already out there and that people really want to see anyways – first thing most people do is Google It or go to You Tube to learn).

Thursday Keynote: The Five Mindsets of Innovation

John Linkner

Business leaders need to be like jazz musicians

Adapt, navigate uncertainty, situation awareness active listening, real-time collab and problem solving, recovering  quickly from setbacks.

Mundita - venture capital firm with a soul (this is his VC company)

Everyday Innovations

 

  1. Start before you're ready

Awaken the artist within

How have I always done it in the past. What would it look like if you did the opposite  the Judo flip.

Savannah Bananas

Reduce risk through experimentation. Lots of little ideas that you test - 15 minutes and 20 bucks fast. Cheap and fast.

Create a To-Test list - like a to do list. Keep a running list of things to try and test - the mere existence of the list will boost your innovation. Do a shared list. "What are we doing to test this week." Inject an experimentation mindset.

 

  1. Break it to Fix it.

Start with  a blank page - Jerry Seinfeld throws out all his material every year.

 

Let go to move ahead. A hallmark of innovators. Release an existing system to discover a new one.

 

At Ben & Jerry's when a flavor is no longer successful, they physically/literally BURY IT. They have a graveyard. They give it a champagne toast and read a eulogy. The let it go. It's ok to let go to move ahead. And they have a digital version on their website. It's ok to break it to fix it.

 

Try this….the business funeral. Look to a system, process, technique, approach that yo've been doing for decades. Give it a funeral and a cake and let it go. We spend too much time trying to save the past.

 

Speedo - break it to fix it - it used to be the fastest suit was smaller, smoother. Then they made a sharkskin suit…full body.

 

Try this…speedo did the judo flip. Instead of small and smooth it was large and not smooth.

 

Shatter conventional thinking

 

  1. Use every drop of toothpaste: MacGyver It

 

Try this….print an image of a tv on your bike box to reduce damage by 70%


4. SEEK THE UNEXPECTED

 

Before choosing A, B or C - think if there's an Option D or an Option X.

 

Option X is what are the obvious answers and what are those far out answers, the unexpected approach.

 

Pittsburgh Children's Hospital wanted to create a better experience - they outfitted the window washers in super hero costumes. Little innovations, big breakthroughs. This cost virtually $0 and made a huge difference.

 

Bring those ideas to the surface. Find the oddball idea that makes all the difference in the world.

 

  1. Fall seven times, stand eight.

 

Setbacks are part of the process, not shameful - a learning opportunity. Each time, try something new (don't keep doing the same thing). Try a different creative approach.

100% chance with jazz music that I'm going to screw it up. Confidence doesn't come from expecting to get it right, it's from knowing how to bounce back.

Downtown Boxing Gym (check it out)

 

Learning Architecture Design: Anchoring ID/LXD in Business Transformation

Frank Nguyen, Genentech (and formerly Amazon)

ID came about out of WWII.

Robert Gagne, 1956 The Conditions of Learning - nine events of instruction - established the beginning of instructional design.

Does training always results in learning? No - sometimes.

Training is a tool - sometimes a jackhammer - may be effective  but not always so efficient.

Walk out of a room, you'll forget 42% of what you heard; and then if in 3 months you haven't used it it'll go down way more. Ebbinghaus Curve was 1859 (?) - but results continue to be replicated.

Gagne's nine events have really served as the basis of ID.

You might sit in a class and follow all the nine events, it might still be really boring. Following conventional instructional design can be really dull.

  • Early 90s - world wide web!
  • 2005 Wikipedia and Web 2.0
  • 2005 YouTube
  • 2007 iPhone
  • "Mobile Learning" people don't even know what that is now - it's just part of the world.

So how do we design around this new world?

Instructional Design + User Experience Design = Learning Experience Design - ID has to evolve to encompass ALL these ways of learning and working.

So does Learning Experience Design always result in learning? No - maybe sometimes.

Do Learning Experiences always achieve Business Goals or Transformation?

Instruction Design > Learning Experience Design > are we due for the next evolution?

What is the problem with LD and LXD?

Are we focused on learning outcomes or business outcomes? Remember, training/learning may not always be the solution.

These don't move at the speed of the business - the business needs much faster solutions

Immediacy, application, tying it back to the learner.

Tenet #1:

Sometimes learning happens because of training, but learning happens all the time with or without us.

Learning produces business outcomes. Run a learning organization and not a training organization.

Jane Hart's list of top learning tools:

YouTube, Google Search, Teams, ChatGPT….then things like Zoom went down….there are no "training tools" on the top ten list - a user focused list. Where do you go to learn? Learning is happening all the time, with or without us.

How are you pulling in youtube etc into your learning architecture?

Tenet #2:

Learning is just one leg of the tripod for change management and business transformation

Change management model: ADKAR

  • Awareness (Communication - lighter weight, less frequency - training class is a jackhammer when you need a screwdriver),
  • Desire (marketing & comms),
  • Knowledge (training),
  • Ability (workflow learning and experiential learning),
  • Reinforcement (workflow learning, incentives, performance management, marketing and comms)

How do you bring value to people's daily jobs…to drive real business transformation, we have to think outside the training box.

Tenet #3:

Focus on learning outcomes not business outcomes

Working Backwards - Insights, Stories and Secrets from Inside Amazon (Colin Bryar and Bill Carr)

Solve real business problems through learning, not just learning objectives

Working Backwards:

  • Who is the customer?
  • What is the customer problem or opportunity? What problem does the customer need to solve?
  • What does the customer experience look like?
  • Is the opportunity compelling enough?
  • Build a roadmap, working backward from the end-state.

You may need to have a long perspective - it can take a long time to change.

"That's where we want to be and these are the iterative steps tto get there."

Instructional Design > Learning Experience Design > Learning Architecture Design (THIS is the current transformation)

Learning Architecture Design = Much more high-level.

Learning Architect Working backwards

  • Who is the customer (learner)?
  • What is the business problem (not the learning problem)?
  • What will be different in the business over next one, two, three years?
  • Design a learning architecture by working backwards from the end state
  • Anchor marketing, comms, and learning to business milestones or seasonality

Learning side will need to be agile because the business may change.

Amazon fulfillment center:

Jan - ramp down

Feb-May - invention/continuous improvement

June - ramp up

July Prime Day (created to be a test day for inventions)

Aug-Sept iteration

October ramp up

Nov-Dec peak

  • Align training/learning architecture with business seasonality
Think about:
  • Business Milestone
  • Marketing
  • Communicate
  • Workflow learning
  • Training (last resort)

Managers - consider creating a 15 minute meeting in a box for an experiential learning exercise in a team meting

If training/learning is tied to your job level, pay, etc. you may pay attention/learn

https://frankn.net/DL23-WBT.pptx

 

 

Getting Started with Learning Content Anaytics 

Cammy Bean, Joe Replin, Rodrigo Bolanos, Kineo 

 

A man standing in front of projected screen share, pointing at numbers on a mocked up data dashboard.

This was our session! Quick story - analytics is top of mind for L&D but it's a low skill set on most teams. Many orgs are just trying to figure out how to get started and there's a lot of overwhelm. We wanted to set the stage for why data & analytics are so crucial to L&D, share a framework for getting started with building these skills by looking more closely at learning content analytics, and share some examples and case studies from some of our work with different types of organizations.


Our deck is up on slideshare for your viewing pleasure.

 

Harnessing the Power of AI to Develop Scenarios

 

Jennifer Foster, Udutu

 

Using AI tools to jump start your workflow and generate branching scenarios:

 

  • Use ChatGPT4 ($20 a month) - to generate list of questions to ask your SME about X topic.

 

 

 

  • Scenario Builder Epic ChatGPT (based on Cristy Tucker's branching scenario work)

 

 

 

 

 

  • Zoom just came out with an AI tool last week - it transcribes your whole meeting for you and then throw that into ChatGPT - "hey give me my action items from that call"

 

 

  • Get around the uncanny valley feeling about avatars by just telling your learners upfront - "hey, this was created with AI".

 

  • Research show: AI created avatar videos match human presented ones in effectiveness

 

 

 

Making Your L&D Case by Storytelling with Learning Data

Eli Bendet-Taicher, Wix

Think about the story you want to tell - think about different ways to tell that data story - two to three ways (each version will vary depending on where you put your hook, your facts, etc.)

What do you want your audience to…?

  • Know - facts data
  • Think - perception
  • Feel - how do you want them to feel about the problem
  • Do - what do you actually want them to do after the session?

"Data in L&D is a lot like sex….everyone is talking about it, but not many are doing it, or even doing it well."

First party data:

  • What can I collect myself? (e.g., directly from your LMS if you're in L&D, etc.)

Second party data:

  • What kind of data do other departments collect within the organization that I can leverage?
  • How can I get my hands on second party data?
  • Second party data - sales, revenue, engagement surveys, performance reviews, business roles performance, quality control

Ideally, you want to correlate first party with second data to show impact

  • Reaction = first party data
  • Learning = first party data
  • Behavior Change =  first and second party data
  • Results = second party data

ROI (financial metric) vs. ROE (indicator of value

ROE - measurement of qualitative and quantitative benefits - did you meet expectations?

Correlating L&D Data with KPIS - the flow.

  • What are your organizational business KPIS
  • How does L&D support the business KPIS
  • Derive your own KPI from business KPIs
  • Collect data
  • Correlate all data points
  • What kind of story does it tell you?
  • Communicate that story to stakeholders

We remember numbers when the numbers serve the story (he showed a pitch video and the guy shared a lot of numbers) - when you tell a story with data, think about the story you want to tell (replace your adjectives with numbers --- a lot, again and again, any, not a lot-huge, game-changer, surprising - can probably all be replaced with numbers.)

Use infographics

Tell the data story and PR your work.

Closing Keynote: Michelle Poler

Michelle Poler, author of Hello Fears and survivor of her own “100 Days Without Fear Experiment”

This was a feel good session that literally had the crowd dancing by the end. Michelle shared stories of facing ones fears, being clear on what you want, and making a shift in your life to live more authentically and courageously.

The Expo Hall

150+ vendors – biggest year ever

BIG booths like DevLearn has never seen before!

·       Articulate – showing off AI power that’s coming soon to a Rise course near you (upload your content docs and press a button and voila – a course! Click a button to translate your content. Includes Video Avatar creator).

·       Regis – showed off an AI driven branching scenario builder.

·       Note – both Articulate and Regis were a bit smoke and mirrors as I suspect all developers were working until the day before the conference to get things together. This is all happening in REAL TIME.

·       Big LMS booths from Docebo, Schoox.

·       SmartCat – for AI driven translations

OTHER

And then there’s all the conversations in the halls and over dinners and captured in passing.

Lots of interesting things happening with AI driven chatbots and agents – this will change how people search for and access content – looking for answers when they need it in the workflow.

AI will change our workflow – how we do our everyday tasks. This will put huge pressure on content companies to reduce cost and speed to market. We need to pivot fast.

It will also change how learners search for and consume content – Learner Experience will be less about a big 45 minute course and more about that chatbot….of course, formal learning/training will still exist, but it’s going to be different.

Obviously, all the concern about regulations and ethics and privacy in any conversation on AI. While I don’t mention that much in my notes, it’s an ever-present thread (thus some of the fear).

Lots of fear, uncertainty – but also a lot of EXCITEMENT about the possibilities.

 

See you next year, #DevLearn!

Tuesday, August 08, 2023

ATD Core 4 - July 24-26, 2023 - Washington, DC

OK – so it’s been a few minutes since I got back from ATD Core 4 in DC, July 24-26.  Better late than never, so here are my notes and insights from a packed couple of days.

ATD Core 4 – What’s Up with That?

Association for Talent Development (ATD) hosts huge conferences with 10,000 people plus. This is not one of those. Core 4 (Foundations in Talent Development) is an intimate conference format with about 400 attendees. Core 4 is held in different place once or twice a year. With four core tracks – Evaluating Impact, Instructional Design, Learning Technologies, Training Delivery & Facilitation – there’s a little something for everyone, provided at a FOUNDATIONAL level.

During the opening keynote session, a raise of hands indicated that a huge percentage – maybe more than 80%? – of attendees were attending their first ATD conference ever, and for many it was also their first conference EVER.

If you’re new to the industry or a seasoned practitioner looking to widen your knowledge, this is a great conference to attend.

Session Summaries & Notes

MONDAY

Opening Keynote: Megan Torrance, Torrance Learning – “Full Speed Ahead: Thriving in a Rapidly Changing Learning Tech Environment”


“Even if tech isn’t our job, it’s kind of everywhere and we need to be processing it.”
> Megan reminds us that talent development and training does not exist without technology anymore. If you’ve been hesitant to better understand tech, it’s time to get past your hesitancy and get grounded in the basics.

Megan’s fun opening session took us through the 30 year adoption cycles for new technologies (see Paul Saffo, The 30 Year Rule from 1992) – there’s the early hype days of a tech and then 30 years later it might become the norm. Megan put this into the context of xAPI which is 10 years in market, still with low-spread adoption (in the room, 2 people raised their hands to say they were doing something with xAPI!)

I loved her race care metaphor – who do you want in your car as you navigate the technical terrain? Who’s on your team? You’re going to need a whole crew:

·        Driver: At the wheel

·        Scanner: “What are all the cool things out there that we should be aware of and checking out?”

·        Navigator: “How are we going to get there?”

·        Doer: “Let’s do this!”

·        Challenger: “Have we thought of….?”

·        Damage Control: fixes it when things go wrong, “I’m giving you all she’s got!”

·        Supporter: “You’ve got this!”

·        Sharer: “Wanna see how?” – this guy is the hype wo/man.

·        Business Partner: “Let’s see some results.”

Megan Torrance is the author of Data & Analytics for Instructional Designers. Check it out.

Cammy Bean, From Accidental to Intentional: Building and Instructional Design Career with Purpose and Passion


My session! If you’ve seen or heard me speak before, I talked about Learning Pie and getting grounded in the whole pie that is L&D: Learning, Creativity, Technology, Business. Finding your sweet spot and leaning into your weakness. Also, knowing that one person really can’t do it all – in spite of popular opinion that a one-stop-shop-training-department can be all things to the enterprise.

I was blown away by the engagement from this crowd. Again, turns out most attendees were at their very first conference ever. A lot of people new to instructional design and corporate L&D, along with seasoned veterans 27+ years into their careers looking for that elusive “what’s next for me?” moment. It was truly an honor and a privilege to share my experience with you all.

Be sure to pick up your copy of my second edition of The Accidental Instructional Designer: Learning Design for the Digital Age! (I thank you with all my heart. Truly.)

TUESDAY

Keynote: Sardek Love, “The Science of Participant Engagement”

I arrived about 10 minutes late to this one and it was like entering a huge party. Sardek was running people through a wild activity that had the room in a full on roar. Talk about audience engagement. This was complete audience engagement.

Sardek threaded his laws of attention and engagement through this session, with lots of application and demonstrations, including getting a room full of adults to suck on some really sour candy to demonstrate our perceptions to change….

ENGAGEMENT:

  • The Law of Attention – attention goes where curiosity grows; attention increases with novelty. Be different. And make people laugh. Laughter releases dopamine and erases boredom.
  • The Law of Connection: we grow together where we go together. Build trust. Release oxytocin through connection – we feel good when we’re part of a group.
  • InterACTION: interaction = people doing something with EACH OTHER.

If you’re into facilitation and general rules of engagement, be sure to look into Sardek Love’s Presentation Essentials.

Cara North, “Building an L&D Control Tower for Metrics”

Six months into a new job where Cara had been focusing on getting good training assets out for a semiconductor manufacturing company, her boss asked her, “So, how do you go from being reactive to being proactive?” Oof. Right? No more order taker – get out there and help us solve the problems that need solving!

Cara shared her journey of working with her org’s business units to better understand their goals and metrics so she could focus and target her L&D activities to support those people. She says to understand “how does my organization make money?” and “What are the critical tasks to get there?”

She took us through her process of building a data CONTROL TOWER – which was a dashboard of the business, so they could see the sum of all parts. Then be able to look at the biz data to see trends, gaps, and where we’re going. These are the metrics that your operational partners are already tracking.

Pre-control tower, her L&D team was tracking course completions and ratings. But “if we are true professionals, we need to keep challenging best practices.”

So she could see on her newly built control tower that turnover was really high and onboarding time was also really long. They worked on creating better hiring training (she showed some cool examples of interview scenarios made quickly with Synthesia an AI video tool). These microlearns were a targeted, proactive training solution that resulted in better interviews, better hires, and ultimately lower turnover. She could see the impact on her control tower.

Cara’s got a new book that I picked up at the conference bookstore and suggest you add to your reading list as well: Learning Experience Design Essentials.

Alaina Szlachta, By Design Development Solutions - “Make Measurement Easier – Find the Simplest Solution for Evaluating Your Learning Programs”

#1 challenge of Measurement & Evaluation (M&E) = OVERWHELM. Because we don’t have strategy or alignment on that strategy (what we’re doing and why).

Some data (with apologies for lack of proper citations; Alaina did have everything noted but I didn’t capture it all!)

  • 20% of L&D professions use data to improve their L&D strategies (2022)
  • 38% professionals in the field say lack of organization focus and strategy is a big barrier to L&D
  • 50% measuring outcomes of L&D not a priority for senior leaders (Brandon Hall, 202)
  • 55% are unhappy with M&E efforts (CLO, 2017)

Strategy DOES NOT EQUAL Tactics

Strategy = the WHAT – what are you doing and why?

Tactics = the HOW – using the most appropriate measurement model to get there.

To figure out what you want to measure, start with a HYPOTHESIS.

If X then Y then Zshort-term then ZLong-term

In other words, did we do what we said we’re going to do?)

Example:

  • Did people participate in program?
  • Did they change KSA (knowledge, skills, attitude)?
  • So what? (the short term outcome)
  • So what – because of the short term, what happened long term?

If people complete the leadership dev program, then they improve their delegation skills which will reduced stress and increase optimism (short term) which increases retention of new managers (long term).

Map out your hypothesis, then ask “what data do we need to prove that?”

Next challenge: how do we get data to show short term and long term outcomes?

This may be through a GROWTH REPORT – a self-reporting rubric (e.g., for the leadership program example) or ask on an evaluation form “what was the most impactful part of this program?”

She showed an example of using Typeform to ask questions during training in real time – creates visual reports in real time.

Over relying on common models (e.g, LTEM, Kirkpatrick) may be adding more complexity, cost, resources. You may not need to follow those models, however, those models give you a way to think about data.

Kristin Torrence, Tailspin – “Learning Engineering 101: Applying Evidence-Based Practices”

Like the title of this session says, Kristin took us on an overview of learning engineering practices and processes – a lot covered in a short time. Here are some of the highlights:


LEARNER PERSONAS - design decision support our learners?” Embody WHO your learners are. Create an “empathy map” to document what they think, feel, say, do (or what you want them to think, feel, say, do as a result of your learning experience).

EVIDENCE-CENTERED DESIGN – what learners need to DO.

Create a Task Shell Canvas (document what people need to DO)

Conjecture Map – takes your HYPOTHESIS to EXPECTED OUTCOME

Communicate your design to stakeholders through wireframes, prototypes, or an MVP (minimum viable product)

USER TESTING? Yes! Do it!

EVALUATE & MEASURE

Are learners transferring what they’ve learned on the job?

Did learners find the training beneficial?

ANALYZE THE DATA

  • Look for patterns/trends
  • Analyze data for statistical and practical signficance
  • Review the context/interpret results
  • Draw conclusions

Rance Greene, School of Story Design – “Story Design: A Foundationally Human Approach to Instruction”

How do we humanize instruction? Why, with stories of course! Rance runs a great session – his genius is getting the crowd to talk amongst themselves for much of the time, asking probing questions that help us collectively discover the storytelling framework. He shows videos of really boring training and then at the end, transforms that same boring training experience into something MUCH MORE HUMAN.

So much of learning comes back to ATTITUDE – we want people to feel/be enlightened, empowered, confident, changed as a results of our training.

To humanize the learning experience, talk to YOU the learner; make it conversational.

Belonging is about TELLING STORIES, having a CONVERSATION, creating of feeling of YOU’RE NOT ALONE.

The elements of story = RELATABLE CHARACTERS + STRONG CONFLICT (this crease a DESIRE FOR RESOLUTION)

Want to get more into Story Design? Be sure to pick up a copy of Rance’s book: Instructional Story Design: Develop Stories that Train.

Wrap Up

All of that in one and a half days. I missed the final half day, which included some enticing sessions like Kevin Yate’s L&D “Detective Measurement Mysteries” and Mark Sheppard’s “You Want it When? Lessons Learned From a Short-Notice Gap Training Project”.

If you’re looking for a smaller conference setting that doesn’t have a huge expo that you then feel compelled to walk through and collect swag, this one is for you.

I’ve been a huge Core 4 fan since its inception and I’m sure I’ll be back. Keep your eyes out for the next one!