DevLearn 2023 - Las Vegas (Show Notes!)

DevLearn has always been my favorite L&D, elearning-nerd out conference. It's where I first hit the stage back in 2009 with my Kineo colleague with Stephen Walsh and then most every year until about 2019. And then we had a pandemic and I sat out a few years. 2023 was my first DevLearn since 2019 and it was really great to be back!

• Kineo was a proud sponsor  of DevLearn this year and it was super cool seeing our logo at the registration stage and on the big screen.
• Amazing to be on the ground with three of my Kineo colleagues: Joe Replin (Technical Consultant coming to Vegas from Chicagoland), Rodrigo Bolanos (Head of Kineo LATAM from Argentina), and Vicky Bartolacci (Kineo Managing Director all the way from Sydney Australia). 
• So great to connect with so many amazing people - old friends and new connections made. 
• Data and Analytics are top of mind for people and extremely low on the L&D team's list of skills. Orgs and individuals are eager to build this capability.
• AI was the darling - and it's coming soon to a workflow near you if it hasn't already. AI is transforming how we work and what we'll be delivering to learners. Game changer. People are feeling a lot of uncertainty and fear and...yes, excitement. Hold on to your hats, the ride is just getting started.
• Ooh - and I signed a whole bunch of copies of The Accidental Instructional Designer: Learning Design for the Digital Age. Always fun to connect with readers and fellow accidents.
• And finally, Vegas has amazing food - no doubt. AND I got to experience the Sphere (we didn't splurge for U2 tickets, but we did see an amazing $90 movie! Hold on to your seats, it was awesome.)

Here's a run down of the sessions I attended (and some I led!).

Morning Buzz – AI and Your L&D Role

Josh Cavalier

Josh is a big AI player right now in the industry, definitely one to follow.

Don’t wait. Start now. Things are changing FAST. We don’t know where things are going to be in 3 months, let alone a year. Everything is changing at rapid pace, but we do know that AI is here and it’s disrupting everything.

Lots of fear in the crowd – “what’s this going to do to my job?”

Get a corporate policy in place.

Policy > Manifesto > Playbook

Manifesto – how your L&D team is going to use AI

Playbook – what tools you’re going to use and how – and then meet on that playbook monthly (this idea actually came from Marcus Bernhardt who was in the crowd – also an AI guy)

Opening Keynote: Phil Rosenthal

 

Phil Rosenthal was a producer/writer of the show "Everybody Loves Raymond" and now has a show on Netflix - 7 seasons - "Somebody Feed Raymond" where he travels the world and eats good food and makes people laugh.

 

Use your constraints - working within your box can unleash your creativity.

 

The show Raymond - they wrote it to be intentionally timeless. Keep out any time bound references (e.g., you don't mention Bill Clinton in the 90's) and you make the content RELATABLE.

 

The best stories are those that are relatable. As a storyteller, satisfy your own interests and be specific. When we are specific, people are more able to relate to your story.

 

"Do the show you want to do because we all get canclled."

 

Creating a Robust Company Learning Culture with Values-Driven Development

Jennifer Gilley, Anuvu; and Amy Wheldon, Go1

Building a values-driven learning culture at an org obviously needs to start with stated values. Then follow a methodology:

1. Build an authentic FOUNDATION that centers on philosophy, values, and the employee experience.
2. Execute meaningful form - uncover the opportunity, informed strategy, culture shift, feedback loops
3. Leverage Effective tools - training and delivery, input and output

FOUNDATION

Anchor in your beliefs and values.

Your org's learning philosophy informs your L&D strategy.

• Gather input from stakeholders
• Map out what learning looks like
• Write a philosophy statement (who, what, why)
• This is high level - no perfect formulas for this
• What does learning look like and what does it mean in your organization?

Core values:

• How do you integrate these values into your culture?
• Incorporate these values to inform training focus and to elevate the employee experience

The Employee Experience:

• Journey mapping
• What does it feel like to work here?
• How can we create an environment where people can truly grow?
• Where can we partner across the business as reinforcement?

L&D Program is really about change management - a form to drive change. Think about:

• Opportunities
• Strategy
• Culture Shift
• Feedback Loops

Uncovering Opportunities:

• Use results from annual and pulse engagement surveys
• Look at performance metrics - review SLAS - to identify gaps around skills/competencies
• Get stakeholder feedback - informal and formal to round out data points

Create an Informed Strategy:

• What are our biggest challenges?
• Prioritize based on those that will have the greatest impact
• ID groups that are most impacted
• Build from competencies
• Automate training
• "This is how the new L&D program is going to support…"

Culture Stuff:

• Model enthusiasm - leverage positive psychology, mirroring, cascade across and down all levels of leadership
• Overt communications
• Shared accountability - tracking with targets to meet

Feedback Loops:

• Formal tools to get employee feedback
• Leverage managers for informal feedback and formal
• Leaders to demonstrate the new behaviors (modeling) and do check-ins with teams

Tools:

Your LMS, facilitators, surveys, what they called "elevators" (ways to elevate the experience - whiteboards, polls, gamification of sessions).

For compliance training - if you're trying to actually change culture, think about how these behaviors ladder up to your values.

For leaders/managers - provide them with a check-in template (for meetings, conversations)

When change happens, values usually don't change. But execution does chage. Ask "How do we define what it means to exhibit our values?"

The Hitchiker's Guide to AI

Matt Donavan, GP Strategies

Start first with what the work is you need to do (not the tools, etc)

Put up Enterprise Guard Rails:

• Responsible Use of AI
• AI Policy
• Approved Tools

Guidelines for responsible use of AI:

• Maintain privacy and security (how identifying data being used)
• Ensure safety and well-being (provide guidance)
• Embrace DEI (how well does the AI work in different regions of the world 0 does it worry the same in Japan and Brazil….
• Drive accountability (is the content generated accurate, do you have the rights for copy and or images being managed. Does it meet quality standards?
• Convey Transparency: let people know when you're using it - don’t be ashamed, but open.
• Promote Fair and Equitable: does it bring people together. Does it drive equity?

GPT4 sitting in an azure environment. Put all of your past RFPs on your system - point it to meaningful data from historical RFPs. (but remember most org's data is going to be a hot mess - may require a lot of mopping up afterwards).

Your policy might be "don't put high-risk data onto that…"

Training Data Set Through 2020 (?)

Constitutional Training (how you set these ecosystems up).

Need both human guidelines in place and the how we're structuring machines

How can AI be used to support a manager in the timeline completion of quality of performance conversations?

Performance Management Conversation Workflow….where could AI support this workflow?

• These are the tasks in the workflow: Remember (to do it!), schedule, locate prior documentation, prep or refresh, coaching or mentoring, conducting the conversation, complete the documentation
• Could you have AI help with those sticky spots? AI could check if you've scheduled it. It could go out to your system and get feedback from people, then summarized the people - then it locates the docs for you, then brings in other data and info.

Morning Buzz – eLearning – what do people really want and need?

Cammy Bean, Kineo

This was an informal conversation about learner needs – and the reality of what’s going on in the real world.  Individuals and orgs are at all levels of maturity in terms of elearning. Lots of Storyline use. People leveraging YouTube videos into course content (why recreate what’s already out there and that people really want to see anyways – first thing most people do is Google It or go to You Tube to learn).

Thursday Keynote: The Five Mindsets of Innovation

John Linkner

Business leaders need to be like jazz musicians

Adapt, navigate uncertainty, situation awareness active listening, real-time collab and problem solving, recovering  quickly from setbacks.

Mundita - venture capital firm with a soul (this is his VC company)

Everyday Innovations

 

1. Start before you're ready

Awaken the artist within

How have I always done it in the past. What would it look like if you did the opposite  the Judo flip.

Savannah Bananas

Reduce risk through experimentation. Lots of little ideas that you test - 15 minutes and 20 bucks fast. Cheap and fast.

Create a To-Test list - like a to do list. Keep a running list of things to try and test - the mere existence of the list will boost your innovation. Do a shared list. "What are we doing to test this week." Inject an experimentation mindset.

 

2. Break it to Fix it.

Start with  a blank page - Jerry Seinfeld throws out all his material every year.

 

Let go to move ahead. A hallmark of innovators. Release an existing system to discover a new one.

 

At Ben & Jerry's when a flavor is no longer successful, they physically/literally BURY IT. They have a graveyard. They give it a champagne toast and read a eulogy. The let it go. It's ok to let go to move ahead. And they have a digital version on their website. It's ok to break it to fix it.

 

Try this….the business funeral. Look to a system, process, technique, approach that yo've been doing for decades. Give it a funeral and a cake and let it go. We spend too much time trying to save the past.

 

Speedo - break it to fix it - it used to be the fastest suit was smaller, smoother. Then they made a sharkskin suit…full body.

 

Try this…speedo did the judo flip. Instead of small and smooth it was large and not smooth.

 

Shatter conventional thinking

 

3. Use every drop of toothpaste: MacGyver It

 

Try this….print an image of a tv on your bike box to reduce damage by 70%

4. SEEK THE UNEXPECTED

 

Before choosing A, B or C - think if there's an Option D or an Option X.

 

Option X is what are the obvious answers and what are those far out answers, the unexpected approach.

 

Pittsburgh Children's Hospital wanted to create a better experience - they outfitted the window washers in super hero costumes. Little innovations, big breakthroughs. This cost virtually $0 and made a huge difference.

 

Bring those ideas to the surface. Find the oddball idea that makes all the difference in the world.

 

5. Fall seven times, stand eight.

 

Setbacks are part of the process, not shameful - a learning opportunity. Each time, try something new (don't keep doing the same thing). Try a different creative approach.

100% chance with jazz music that I'm going to screw it up. Confidence doesn't come from expecting to get it right, it's from knowing how to bounce back.

Downtown Boxing Gym (check it out)

 

Learning Architecture Design: Anchoring ID/LXD in Business Transformation

Frank Nguyen, Genentech (and formerly Amazon)

ID came about out of WWII.

Robert Gagne, 1956 The Conditions of Learning - nine events of instruction - established the beginning of instructional design.

Does training always results in learning? No - sometimes.

Training is a tool - sometimes a jackhammer - may be effective  but not always so efficient.

Walk out of a room, you'll forget 42% of what you heard; and then if in 3 months you haven't used it it'll go down way more. Ebbinghaus Curve was 1859 (?) - but results continue to be replicated.

Gagne's nine events have really served as the basis of ID.

You might sit in a class and follow all the nine events, it might still be really boring. Following conventional instructional design can be really dull.

• Early 90s - world wide web!
• 2005 Wikipedia and Web 2.0
• 2005 YouTube
• 2007 iPhone
• "Mobile Learning" people don't even know what that is now - it's just part of the world.

So how do we design around this new world?

Instructional Design + User Experience Design = Learning Experience Design - ID has to evolve to encompass ALL these ways of learning and working.

So does Learning Experience Design always result in learning? No - maybe sometimes.

Do Learning Experiences always achieve Business Goals or Transformation?

Instruction Design > Learning Experience Design > are we due for the next evolution?

What is the problem with LD and LXD?

Are we focused on learning outcomes or business outcomes? Remember, training/learning may not always be the solution.

These don't move at the speed of the business - the business needs much faster solutions

Immediacy, application, tying it back to the learner.

Tenet #1:

Sometimes learning happens because of training, but learning happens all the time with or without us.

Learning produces business outcomes. Run a learning organization and not a training organization.

Jane Hart's list of top learning tools:

YouTube, Google Search, Teams, ChatGPT….then things like Zoom went down….there are no "training tools" on the top ten list - a user focused list. Where do you go to learn? Learning is happening all the time, with or without us.

How are you pulling in youtube etc into your learning architecture?

Tenet #2:

Learning is just one leg of the tripod for change management and business transformation

Change management model: ADKAR

• Awareness (Communication - lighter weight, less frequency - training class is a jackhammer when you need a screwdriver),
• Desire (marketing & comms),
• Knowledge (training),
• Ability (workflow learning and experiential learning),
• Reinforcement (workflow learning, incentives, performance management, marketing and comms)

How do you bring value to people's daily jobs…to drive real business transformation, we have to think outside the training box.

Tenet #3:

Focus on learning outcomes not business outcomes

Working Backwards - Insights, Stories and Secrets from Inside Amazon (Colin Bryar and Bill Carr)

Solve real business problems through learning, not just learning objectives

Working Backwards:

• Who is the customer?
• What is the customer problem or opportunity? What problem does the customer need to solve?
• What does the customer experience look like?
• Is the opportunity compelling enough?
• Build a roadmap, working backward from the end-state.

You may need to have a long perspective - it can take a long time to change.

"That's where we want to be and these are the iterative steps tto get there."

Instructional Design > Learning Experience Design > Learning Architecture Design (THIS is the current transformation)

Learning Architecture Design = Much more high-level.

Learning Architect Working backwards

• Who is the customer (learner)?
• What is the business problem (not the learning problem)?
• What will be different in the business over next one, two, three years?
• Design a learning architecture by working backwards from the end state
• Anchor marketing, comms, and learning to business milestones or seasonality

Learning side will need to be agile because the business may change.

Amazon fulfillment center:

Jan - ramp down

Feb-May - invention/continuous improvement

June - ramp up

July Prime Day (created to be a test day for inventions)

Aug-Sept iteration

October ramp up

Nov-Dec peak

• Align training/learning architecture with business seasonality

Think about:

• Business Milestone
• Marketing
• Communicate
• Workflow learning
• Training (last resort)

Managers - consider creating a 15 minute meeting in a box for an experiential learning exercise in a team meting

If training/learning is tied to your job level, pay, etc. you may pay attention/learn

https://frankn.net/DL23-WBT.pptx

 

 

Getting Started with Learning Content Anaytics 

Cammy Bean, Joe Replin, Rodrigo Bolanos, Kineo 

 

This was our session! Quick story - analytics is top of mind for L&D but it's a low skill set on most teams. Many orgs are just trying to figure out how to get started and there's a lot of overwhelm. We wanted to set the stage for why data & analytics are so crucial to L&D, share a framework for getting started with building these skills by looking more closely at learning content analytics, and share some examples and case studies from some of our work with different types of organizations.

Our deck is up on slideshare for your viewing pleasure.

 

Harnessing the Power of AI to Develop Scenarios

 

Jennifer Foster, Udutu

 

Using AI tools to jump start your workflow and generate branching scenarios:

 

• Use ChatGPT4 ($20 a month) - to generate list of questions to ask your SME about X topic.

 

• Then record the interview  with your SME- put into Descript and then put that into Chat PDF ChatPDF - Chat with any PDF!

 

• Use Descript for transcription…..Free Audio and Video Transcription Nearly Instant | Descript

 

• Scenario Builder Epic ChatGPT (based on Cristy Tucker's branching scenario work)

 

• Eleven Labs for speech synthesis ElevenLabs - Generative AI Text to Speech & Voice Cloning

 

• Designing branches: Twine (not AI) Twine / An open-source tool for telling interactive, nonlinear stories (twinery.org)

 

• D-ID - Creative Reality Studio - D-ID AI Video Generator: Creating AI Videos from Photos & Avatars

 

• Synthesia's workflows are also really great. Synthesia - #1 AI Video Generator

 

• Zoom just came out with an AI tool last week - it transcribes your whole meeting for you and then throw that into ChatGPT - "hey give me my action items from that call"

 

• Make an image/avatar in midjourney or DALL-E DALL·E 2 (openai.com)

 

• Get around the uncanny valley feeling about avatars by just telling your learners upfront - "hey, this was created with AI".

 

• Research show: AI created avatar videos match human presented ones in effectiveness

 

• InWorld - AI Generated Scenario Adventures Inworld – The most advanced Character Engine for AI NPCs

 

• The Searchable PDF that people want to chat with…ChatPDF - Chat with any PDF!

 

Making Your L&D Case by Storytelling with Learning Data

Eli Bendet-Taicher, Wix

Think about the story you want to tell - think about different ways to tell that data story - two to three ways (each version will vary depending on where you put your hook, your facts, etc.)

What do you want your audience to…?

• Know - facts data
• Think - perception
• Feel - how do you want them to feel about the problem
• Do - what do you actually want them to do after the session?

"Data in L&D is a lot like sex….everyone is talking about it, but not many are doing it, or even doing it well."

First party data:

• What can I collect myself? (e.g., directly from your LMS if you're in L&D, etc.)

Second party data:

• What kind of data do other departments collect within the organization that I can leverage?
• How can I get my hands on second party data?
• Second party data - sales, revenue, engagement surveys, performance reviews, business roles performance, quality control

Ideally, you want to correlate first party with second data to show impact

• Reaction = first party data
• Learning = first party data
• Behavior Change =  first and second party data
• Results = second party data

ROI (financial metric) vs. ROE (indicator of value

ROE - measurement of qualitative and quantitative benefits - did you meet expectations?

Correlating L&D Data with KPIS - the flow.

• What are your organizational business KPIS
• How does L&D support the business KPIS
• Derive your own KPI from business KPIs
• Collect data
• Correlate all data points
• What kind of story does it tell you?
• Communicate that story to stakeholders

We remember numbers when the numbers serve the story (he showed a pitch video and the guy shared a lot of numbers) - when you tell a story with data, think about the story you want to tell (replace your adjectives with numbers --- a lot, again and again, any, not a lot-huge, game-changer, surprising - can probably all be replaced with numbers.)

Use infographics

Tell the data story and PR your work.

Closing Keynote: Michelle Poler

Michelle Poler, author of Hello Fears and survivor of her own “100 Days Without Fear Experiment”

This was a feel good session that literally had the crowd dancing by the end. Michelle shared stories of facing ones fears, being clear on what you want, and making a shift in your life to live more authentically and courageously.

The Expo Hall

150+ vendors – biggest year ever

BIG booths like DevLearn has never seen before!

·       Articulate – showing off AI power that’s coming soon to a Rise course near you (upload your content docs and press a button and voila – a course! Click a button to translate your content. Includes Video Avatar creator).

·       Regis – showed off an AI driven branching scenario builder.

·       Note – both Articulate and Regis were a bit smoke and mirrors as I suspect all developers were working until the day before the conference to get things together. This is all happening in REAL TIME.

·       Big LMS booths from Docebo, Schoox.

·       SmartCat – for AI driven translations

OTHER

And then there’s all the conversations in the halls and over dinners and captured in passing.

Lots of interesting things happening with AI driven chatbots and agents – this will change how people search for and access content – looking for answers when they need it in the workflow.

AI will change our workflow – how we do our everyday tasks. This will put huge pressure on content companies to reduce cost and speed to market. We need to pivot fast.

It will also change how learners search for and consume content – Learner Experience will be less about a big 45 minute course and more about that chatbot….of course, formal learning/training will still exist, but it’s going to be different.

Obviously, all the concern about regulations and ethics and privacy in any conversation on AI. While I don’t mention that much in my notes, it’s an ever-present thread (thus some of the fear).

Lots of fear, uncertainty – but also a lot of EXCITEMENT about the possibilities.

 

See you next year, #DevLearn!