Wednesday, November 11, 2009

Ruth Clark: Evidence Based E-Learning #dl09 #dl09-104

These are live blogged notes from DevLearn '09 -- session with Ruth Clark on Evidenced Based E-Learning. I arrived a few minutes late to the session and just had to dive in...


Learning Styles/the Learning Styles Myth --

Did an experiment: Self report, Barsch learning style inventory, memory test. What were the correlations? If someone said they were kinesthetic, were the other measures showing the same? Instead, they found NO Relationships!

(Her new book takes on the biggest myths of the field including Learning Styles. This is controversial!)

Replace it with Mayer’s Multimedia Principles

Richard Mayer (25 years of research)

Experimental evidence…

Do graphics improve learning?

(Having just read Mayer’s Multimedia Learning on the plane over here, I’m not sure if this will be a useful session for me – but I really wanted to see Ruth Clark speak!)

Multimedia Principle – having visual results in better learning.

For whom do graphics improve learning?

experiment: visuals in the courtroom --

Judge gives instructions on self-defense to 2 juries: 90 legally untrained adults, 90 law students.

  • One got all audio instruction
  • The other was audio + visuals (flow chart)

The worst was novices with audio only. Visuals helped learners with no prior knowledge the most. The law students didn’t show great improvement with video.

Invest more in visuals in beginning level courses. If people have prior knowledge in domain, they can create own visual by activating their prior knowledge.

Spatial Aptitude

Are all visuals equal?

Jazz it up and make it more engaging. Are these types of visuals helpful?

Which gets better learning:

  • Base text and graphic
  • Interesting anecdotes added to base lesson

The basic version wins.

The Coherence Principle

The interest factor did not serve learning. Distracting.

Learning is better when extraneous materials are eliminated.

What is the relationship between student ratings and learning?

Liking vs. learning.

Recent study looked at thousands of surveys and looked at correlation. This was a meta analysis of all the ratings on the courses – looked at classroom based learning and not elearning. Correlation between liking (ratings) and learning (tests) is:

  • Delcarative learning (concepts and facts) – really small (.12)
  • procedural learning – really small (.15)
  • delayed procedural learning) – really small

The relationship between ratings (level 1 student rating sheets) is too small to assess lesson effectiveness.

Use explanatory graphics

3 types of graphics: Decorative (generally overdone!), explanatory and representational (here’s what the screen looks like – these are important in our work)

Explanatory Visuals -- Show relationships among your content topics.

  • organizational (shows qualitative relationships among topics – tree, concept map)
  • relational (summary of quantitative, pie charts and bar charts)
  • transformational (shows change in time and space)
  • interpretive (take invisible, abstract ideas and make visible – used in science a lot to show molecules, etc.)

[Cammy sidebar: My burning question is about using visuals in storytelling/scenarios. I have a course on sexual harrassment and I use a picture to help the story. If it’s a picture of the woman looking upset after an incident what is that? I don’t think that’s a distracting visual as it puts a human face to the story.]


Which is best?

  • Visuals (animation) with narration
  • Visuals with Text
  • Visuals with Text and narration

Visuals (animation) with narration.

Modality Principle:

When modality applies -- exceptions

  • The content and/or visual are complex
  • learners are relatively novice
  • instructional pacing
  • words NOT needed for reference (important new words may need to be on screen)
  • native language

Redundancy Principle: learning is better when visuals are explained by audio narration than by text

We are ALL visual learners. We ALL benefit from audio!

Contiguity Principle: Put text in with the graphics (not off to the side or under the screen) – integrate text as close to relevant visual as you can.

As you read a book – you have to turn page to see visual that goes with text on the page. Annoying.

Learning is better from integrated text.

Avoid scrolling screens when text is on bottom and visual at top…

When Less is More (new research)

1. complex vs. simple graphics

Comparing line drawing to realistic 3D drawing. – lean vs. rich multimedia.

Carol Butcher, University of Colorado study – where was learning best?

  • text
  • text & simple graphic (this was more effective!)
  • text & complex graphic

2. Stills vs. animations?

Are animations better? How a toilet works…

stills with text vs. animated with audio?

4 diff experiments done.

STILLS fared better in all experiments. The animation can give too much visual information and it’s often out of the control of the learner.

Two theories about why:

1) animations can – impose extraneous mental load have to hold animation frames in memory to link one to the next.

2) animations can - promote a passive mental state (vs. mentally animating or self-explaining the key steps) – we go into couch potato state with animations…

(The discussion is now transgressing to whether or not people know how toilets work…)

Are animations better? (part II)

stills vs. animation to learn a procedure – animations were MUCH better. Mirror neurons. Adapted part of our brain to learn movement – doesn’t impact working memory.

Learning of motor skills is better when illustrated with animation vs. stills.

3. Learning from examples in text, video and animation

Which led to better learning?

Which was rated higher?

Animation examples were the highest on both, followed closely by video. Visual examples were more effective – but animation/video not statistically different from each other.

(Gotta run!)


Rani H. Gill said...

Hi Cammy - I thought your post was going to be on on the Ruth Clark session - sounded like she talked mostly about the Mayer's principles.

FYI I just did a slidecast on MM Princples - trying to sort them out myself. Don't advise slidecasting lrg PPTs - too much work. It's here if you're interested:

keep twittering. cheers.

Unknown said...

I find your posting very confusing. I'm familiar with the collaborative publications by Clark and Mayer. I read the first book that you quoted from. Together, they have taken research findings and made then understandable and applicable to practical needs.

Their latest book is e-Learning and the Science of Instruction -- Proven Guidelines for Consumers and Designers of Multimedia Learning, 2nd Edition, Clark & Mayer, ISBN 978-0-7879-8683-4, published by Pfeiffer, an imprint of Wiley.

Rick Lillie
Cal State San Bernardino

Anonymous said...

One interesting point is that these studies have all been done in controlled environments, but that the principles don't always carry over into real learning situations where things are more complex (learning is messy!). For example, the coherence principle was directly contradicted in the authentic learning environment of "Coherence or interest: Which is most important
in online multimedia learning?"

Cammy Bean said...

Hi all,

Thanks for the comments. This post was live-blogged during the session -- I'm sure confusing! The first part of her session did seem to be a recap of the research Mayer covers in his book Multimedia Learning (which I read on the plane to San Jose).

The last ten-fifteen minutes covered some new research on animations vs. stills, which I found really interesting.

Rani H. Gill said...

@Christy - thx for the paper - i take these principles with a grain of salt - now i know why.

@Cammy - thanks for the conversation starter and keeping us informed of what's going on at #dl09!

Anonymous said...

173 影音 live 秀免費視訊
13060 免費視訊聊天
uthome 視訊聊天室
0509 免費視訊聊天秀

Anonymous said...

Thinking about your example of the images in the sexual harassment course, I'm inclined to agree with you that it isn't a distracting image. In sexual harassment training, the goal isn't just to get people to memorize the policy; it's getting people to change attitudes. Clark & Mayer's research nearly always measures recall and maybe basic comprehension. I don't think we can assume it automatically transfers to situations where you're trying to change attitudes since that isn't what you're measuring.

Is it possible that the photos distract and would reduce scores on a test of memorizing phrases from the sexual harassment policy? Yeah, it is; I expect that if you measure just recall that photos of upset people probably get in the way of that. But is that really what you're measuring?

It's like Kirschner, Sweller, & Clark's "Failure of Constructivism" article. (That's Richard Clark, BTW, not Ruth.) K, S, & C said problem-based learning for medical residents was a failure because the residents did worse on basic science tests than those in traditional med school programs. However, students in the problem-based program were rated higher in clinical performance. Is that actually a sign of failure? According to K, S, & C, the clinical performance isn't important; the written test scores are. I like Ruth Clark's stuff, but you have to keep in mind that her research tends to be more like K, S, & C.

Unknown said...

Good discussion. I posted a link to it on twitter, with a hint of my skepticism about aspects of RC's work. Thanks to those who posted links to the papers. It is vitally important that people who read Mayer, Sweller, and both Clarks works understand that the multimedia principles ARE based more on rote learning goals and with experiments often done out of context. Their positions have become more nuanced over time, but they are still (particularly Sweller) largely in the direct instruction camp vs the active learning constructivist approaches that most of us advocate these days.

Liz D.
twitter: ldinstl_chimera

Unknown said...

On the positive side, I totally agree with RC on her debunking of the myth of Learning Styles.

A friend reported to me years ago that a prominent learning theorist firmly stated "Learning Styles is FAILED theory!" I never forgot that and have seen a lot of papers since then that agree--it's a myth as commonly formulated.

jay said...

Cammy, I liked and learned from your summary. As a fellow live blogger, I expect things to sound a little screwy. Geez, just try to read a verbatim transcript some time.

What I find missing in Ruth's work is context. We don't learn in the lab. There's always so much going on that warps the austere results from controlled experiments.

Shelley said...

Cammy, belated thanks for taking the time to live-blog this interesting and thoughtful session.

I can now imagine folks who can't afford to attend conference live & in person offering to buy a competent live-blogger's lunch (missed my chance on this one, I guess)!

Jeffrey Holton said...

I found this really interesting.

I just spent six weeks producing 30 minutes of video footage to demonstrate how to use a software tool. Through the whole project, I remained unconvinced about the best course of action.

If I'm reading this correctly, it seems that it was at worst the second-best thing to do.

Thank you for posting this.