Wednesday, May 07, 2014

Become a Strategic Partner Through Effective Training Evaluation #ASTD2014 @Jim_Kirkpatrick

These are my live blogged notes from a concurrent session (W104) at the ASTD International Conference & Expo, this week in Washington, D.C. Forgive any typos or incoherencies.

James Kirkpatrick, Senior Consultant Kirkpatrick Partners

Sometimes we have to do the politically incorrect thing. We have to choose to take the disruptive, high road of business partnership instead of the easy road.

Three action points to help you take the high road:
  1. Gain agreement on the required level of effort for program success for key programs
  2. Seek the truth through training evaluation
  3. Speak the truth about the program

We need to know the truth so we can make adjustments.

In the new world, we start with Level 4 - we start with the results. To what degree targeted outcomes happen. The only way you get to level 4 is through level 3 (behavior) -- application on the job. And we need to know we have prepared them (level 2), and we need to make sure we've had a good experience (level 1).

We start at level 4 because it's where we want to go. It's our GPS. Traditionally people freak out about level 4 -- how can we possibly get to that flag way up on the mountain. We need to start with the end in mind so we can create a path to get there.

Why evaluate?

Execs need to see training effectiveness -- maximizing organizational results (that's level 3). 

Training effectiveness = maximizing organizational results and demonstrating program value.

Three stages of a training program: planning --> execution --> demonstration of value

The E (evaluation) should not only come at the end. It needs to be woven throughout. Making sure things are working as well as possible.

Learning is a journey and not an event. 
  1. Gain agreement on the required level of effort for program success for key programs
You need to talk about the coordinated approach the organization needs to take.  Instead of learning objectives, we need to talk about critical behaviors.  "Are you going to watch things happen or make things happen?"

What will be the ultimate business criteria for program success? What will they need to see from a business point of view that will allow them to say "job well done". (We need to increase profitability, etc.)

What will ned to occur before, during, and after training in order to maximize application and results? This is where the level of investment comes in - it's not just about writing the check.

Your customer says "we need better leadership training because we need stronger leaders" you, the training consultant, ask questions to dig deeper.

So you ask: What's behind the request for training? What are the business reasons? Are there numbers behind this request?


See the Brinkerhoff Study on the effectiveness of training. (I'm not sure which study that is, but here's Brinkerhoff on Amazon:

Seek the truth
Gather honest training feedback -- focus on learner-centered items rather than training
Measure and monitor critical behaviors and leading indicators

Get the whole truth in your evaluation data and what actions you may wish to take.

If we can increase the amount of loans by 20% will your stakeholders be satisfied? Get as high up as you can and get that validated.

Ask - does the training meet expectations? If not why? take action to correct it.

Speaking the truth means:
  • initiating conversations with stakeholders to share the good and the bad
  • speak the truth about the future

Case study with the Emirates Group
Leadership felt that the cost of running the 157 person training department was more than the value they were providing.

The training team took a hard look at where and how they were adding value. They realized they needed to go beyond the classroom and get in to the workplace.

They realized they needed to do something completely different. 

Partnership Principals:

The end is the beginning. 
Everything they did had to yield increased performance in the business

Return on expectation (ROE) is the ultimate indicator of value.
Needed to explain what we could do as a training support.

Business partnership is necessary to bring about positive ROE.
Then we had to tai about what we were going to do to bring about results.

Value must be created before it can be demonstrated.
Can we impact the bottom line? can we impact performance?

Chain of Evidence.
Finally, connecting it all together. What evidence can we now show that the value we're contributing makes impact to business.

The end is the beginning. a non-negotiable. Where are we going. How will we impact the business. Only then would we apply a resources.

We need to have the difficult conversations. We needed to iron out the issues and decide who was going to do what.

MOU -- Memorandum of Understanding
Reaching agreement about what we're going to do collaboratively for the good of the organization. And it would help us gather evidence.
It forms the foundation; defines roles, responsibilities, timelines; helps reach agreement...

Moving beyond comfort zones out of the classroom and getting closer to the business.

Pilot Case
They had inconsistency in customer service experiences across their different call centers.  They decided on their level 4 first -- to have 90% satisfaction scores. What were the core behaviors they wanted -- demonstrated empathy, being calm and assured, knowing your job, being counter-cultural.

SOAR -- "Service Over and Above the Rest" - coach to perform -- This was  a change management program.

So they watched their customer complaints and they were going down.

They linked to performance management behaviors (empathy, cosmopolitan, etc.) - they could show they were adding value.

They could look at their Quality scores across locations and they were all going up.

Now they are converting those leading indicators into that flag on the top of the mountain -- revenue. 

Become a strategic business partner.

More at

No comments: