Formative Evaluations: 4 Types of Feedback

As an instructional designer, you depend completely on your subject matter experts and others to review what you’ve put together and give you feedback. But, how often do your reviews go awry? For example, you are able to secure the top expert in the area of widget design. You are designing a course on the history of widget design and have put together a first draft of your course. You send it to your great SME (subject matter expert) and ask him for his feedback on your content. He returns it three days later with grammatical changes. This leaves you scratching your head and wondering, “Does this mean the course content is right on track?”  “Did he even look at the content?”

The problem may be that the subject matter expert didn’t read your cover email, but then again, maybe he did and didn’t know what he was supposed to review.

From my experience, (and please feel free to comment if I miss any) there are four basic types of reviews you can ask someone to provide: Instructional, Functional, Grammatical, and Content.

Almost everyone is familiar with a grammatical review. We’ve been required to perform those since grade-school. Unless we explicitly tell our SME otherwise, shouldn’t we expect him or her to default to grammatical?

Below is a short review of the goals of each of these review types:

The Instructional Review

The Instructional Review is also sometimes referred to as Peer Review. We are asking someone with skill in our field to review our work for its instructional elements.

  • Is it readable? Did we write to the correct education level or using the correct terms for the country or region?
  • Is the style correct? Are we supposed to be writing in active voice? Conversationally? Does the style match other work in the same curriculum?
  • Are the transitions from one topic to the next smooth? Do they follow the accepted guidelines for transitions?
  • Does the course have a good instructional flow? Is the content clearly mapped to the objectives? Does the course have a good instructional framework?
  • Are the reviews and test items in the correct format? Do the questions map to the objectives? Are the distractors obvious or relevant?

The Functional Review

The Functional Review is something that you can perform on electronic courses such as eLearning, recorded Webinars, Vignettes, or mLearning. In this review, you are asking someone to check if the course actually works the way it is supposed to work. Some things to ask your reviewer to check for are:

  • Does the course follow the last approved storyboard exactly?
  • Do the buttons work as designed?
  • Can the learner move between screens as intended?
  • Does the branching work properly no matter which path is taken?
  • Are the quiz answers marked correctly?
  • Does the synchronization match the narration?
  • Is the artwork pixelated or otherwise unattractive?
  • Does the course meet the required standards for color, font, etc?

The Grammatical Review

As stated earlier, this is the easy one. Ask your reviewer to do the following:

Check the course against the Style Guide if it exists to determine:

  • Is the course in the correct tense? Voice?
  • Is all grammar, spelling, and punctuation correct?
  • Is there noun-verb agreement?

And finally…..The Content Review

This is the most difficult review to obtain in my opinion. The subject matter expert can be very difficult to nail down for this. In this review, your SME is reviewing the content for correctness, applicability to the job role, and correct depth of the subject.

There can even be different flavors of this type of review:

  • Pure functional content
  • Legal compliance of the content (boy does this one take a while!)
  • A review of staying in “brand” for a company
  • Others? Please feel free to add in the comments of the blog

You are asking the SME to review the material for the following:

  • Is the content correct?
  • Is terminology used correctly?
  • Is the content current? Will it be changing soon?
  • Is the content presented in the correct order?
  • Is all the content that needs to be a part of the course – in the course?
  • Are the test questions testing the correct level of understanding?
  • Is the content covering more / less / just enough to help achieve the learning objective?
  • Etc..

This list could get really long depending on the goals of the course. Maybe you have goals around sensitivity or creating a sense of urgency. If so, the SME should look for that as well. You get the idea.

In Summary

Some ways to make sure your SME is on the same page with you on the type of review you want is with a cover email, a review check-sheet on page one of the document being reviewed, and expectation setting during the kick-off meeting. The real point here is – make sure your SME is clear on what you want him or her to look for.

I welcome any suggestions for improvement or additional formative evaluation types.

4 Responses to “Formative Evaluations: 4 Types of Feedback”
  1. Richard says:

    Nice Article! Thanks.

  2. Jessica says:

    Great information, any update on how CPLP prep is going?

  3. Tom says:

    Great info for guiding SMEs. Thanks.

Check out what others are saying...

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: