Okay. I've made the
trip to the business library… which isn't actually where I thought it was. The
new library is beautiful but lacks the quaintness of the old library. That
said, let's get into this project.
Specifically, I have been tasked to develop some training material for
something we do internally and that I have been doing for a long time. I'm
developing some training that should be delivered electronically in either a
seminar format or self-serve from the LMS.
I've ripped a bunch
of books from HF5549.5.T7. Let's see what we find.
- Hardingham. 1996. Designing training.
The intro looks
good. It lays out a thought model and reviews choices for different types of
training. Venue choices. Costs. You have to consult to understand training
objectives. These things should be SMART. Manage politics, particularly
blockers and contributors.
There are five key
concepts in training design:
- Establish credibility
- Elicit Commitment
- Attract attention
- Allow maneuverability
Signpost. Lots. Vary
pace and rhythm. Chunk content. Map the participants world. Give participants
choices. Surface objections. Balance theory and practice. Design for closure
(entertainment, presentation of certificates, closing speeches, closing feedback,
presentation of gifts).
interventions are a form of training that could involve small group work,
facilitation, direct process manipulation.
There might be
differences in groups. Give participants choices and design in layers.
training the unwilling; money; training for the easily bored; training for
people who won't work together.
Overall, it's a
little book but I like it. Perhaps we could use it as a basis for building
workshops i.e., training on training.
- Stolovitch and Keeps. 2011. Telling ain't training.
This book was my
reference anchor since it was published by the ASTD. Let's see what it has to
It starts off with a
decent overview of how humans learn. There are different types of learning:
declarative and procedural (e.g., riding a bike). Learning requires ability,
prior knowledge, motivation (value, confidence, and mood).
We've got some more
principles of adult learning:
- Readiness. Adult learners
must participate in and contribute to the learning process.
- Experience. Ault learners
must be able to immediately apply what they are learning.
- Autonomy. They must see the
benefits of what they are doing.
- Action. Content and
activities should be integrated.
learning: scenarios, role plays, cases, brainstorming, practice activities.
The learner must be
the center of attention. Trainer success comes from the success of the
attendees. Andragogy = adult learning. Pedagogy = kid learning. Answer: What's
in it for me? Never assume they don't have experience. Let them contribute.
Focus on immediate application. Training must be applied on the job. Train
others as you would be trained.
There's a five step
model for structuring training. See:
There's also some
good planning sheets.
Learners need to be
engaged. They have different
meta-cognitive skills: planning, selecting, connecting, tuning, monitoring.
There are also the deficient factors: ability, prior knowledge, and motivation.
We have a few
different cognitive strategies: clustering, spatial (hierarchies, flow-charts,
matrix), advance organizers, image-rich comparisons (analogies, metaphors,
comparisons), repetition, memory aids (mnemonics: acronyms, acrostics, rhymes,
Four types of
training: receptive (telling), directive (do it this way), guided discovery
(cases), exploratory learning.
Gg - Perhaps guided
discovery is the best way to start F2F training.
- Better me. How can you do better than
me at a procedural task?
- Concentration. Match cards, etc.
- Confrontation. Groups of three. Two in
confrontation, one observer. Timed rounds. Participants cycle through five
strategy choices and observe outcomes. Six scenarios, each lasting 5
instruction, members have to produce the most important points. Iterate to
find most important points.
Decipher a puzzle.
cards with a term on one side and a definition on the other. Participants
must make a chain.
- Exam cram. Divide into study groups.
Give a test to each team. They have 20 minutes to cram. The leader can
actually test or not.
- Facts-in-Five. Give a card with a 5x5
matrix. Columns are categories; rows have letters of the alphabet. Fill
the matrix with aligning keywords.
some debatable issues. Divide into teams. They have 10 minutes to prep pro
or con for a 2 minute debate.
- Hit or myth. Participants get a list of
10 statements that are either true or false. They have to decide.
- Jeopardy. Given answers; have to
determine questions. Could be timed, scored, or involve increased
- Jigsaw. Class is divided into teams
that are given documentation about one part of a topic. The teams prepare
a presentation about their specific part including format and how to make
it interesting. Chain the presentations into a lecture. End with a quiz
and a debrief.
team quiz. Break
into teams. Lecture. Teams must prepare a quiz question and answers.
- Letter game. Break into teams. Each team
gets an envelope with a problem written on it. Each team gets one fewer
cards than envelopes. They write their solution to the problem on a card,
their identifier on the back, insert into the envelope, and pass to the
next group who does the same thing. Teams than rank order the solutions
and points are assigned.
teams. Each team
has to listed for key parts of information and repeat.
- Mismatch. Describe 4-8 scenarios
involving a question and an answer. The question and answer are
- Ours versus
theirs. Create a
lists of 10 to 15 statements. Beside each statement create three
checkboxes associated with competitive providers or products. Ask
participants for feedback.
There is a topic and quiz. There is 30 minutes. The participants must
force the content from the participants.
Teams of reporters. They must draw out information from an expert. The
conference has four or five themes. Teams create questions related to the
themes on colour-coded cards. Teams then take on a theme. They have 15
minutes to review questions and then 7 minutes to grill the expert.
- Quiz game. After a day of instruction,
participants are given a slip of paper to identify a single point worth
remembering. This is the answer. The participant also comes up with a
question to elicit the answer.
- Slap jack. Instructor creates cards
with a term on each card. Participants are grouped into teams of 3-5. The
instructor calls out knowledge domain, each participant throws a card,
participant has to correctly identify the domain and touch/slap the card.
Alternate between roles of customer and sales consultant. Rounds are 99
seconds. Clients ask questions; consultants respond to as many as
possible. Good review.
gets a BINGO card with terms. Definitions are read out related to the
- They say,
we say. Teams of
3 or 4. Half as competitor sales consultants; half as home team. A round =
15 seconds for one side to introduce a feature + 30 seconds for the other
side to rebut.
- True grid. Blank grid. Participants
can apply axis titles and fill it in or the axis titles can be
verification are different things.
There are advantages
to each approach:
Four steps to
training evaluation: 1. participant reaction; 2. immediate learning at the end
of the session; 3. on-the-job application post-training; 4. results -- did
something change organizationally?
RLO = reusable
Online learning best
- Right content
- Strong alignment with
- True interactivity
- Valued experience i.e., fun
- Few distractions
- Useful on the job
- Powerful feedback
- Valid assessments
- Good human factors
- Integrated follow-up
- Who is the course for and
- What are the content and
- How deeply should it go?
- What type of learning is
- How much time do you have?
- How will the training be
- What are the metrics?
Learning" involves information (repositories, job aids, procedures),
instruction (training, online, external), and collaboration (social media,
1993. acquiring knowledge and using it. ERIC 382 238.
That was a good one.
My brain is getting full and we're almost two hours into this effort. What's
- Kosalka, et
al. 2013. Instructional designer competencies.
It is a volume of
the IBSTPI (International Board of Standards for Training, Performance and
Instruction). So that's interesting.
So this is about
competencies for instructional design… cool. If I was coming up with job
descriptions. This thing is very much an auditing standard rather than a how-to
guide. Regardless, perhaps I should draw a distinction between
"training" and "instructional design".
- Phillips. 2010. Measuring and
Another ASTD title.
It is a collection of essays that basically follow Evaluation Planning, Data
Collection, Data Analysis, Measurement and Evaluation. Some interesting looking
stuff in here but it's probably more academic/mature than my current information
- Russ-Eft, et al. 2008. Evaluator
As long as we're on
the evaluator side of things, let's take a look at this book. Koszalka is an
author and the forward is by the past president of the American Evaluation
Association. Yet another association to deal with!
And… it's not what
I'm looking for. Next.
- Lawson. 2009.
The trainer's handbook.
The title looks
promising. Let's go.
So we need to an
needs assessment: identify problem or need; determine needs-assessment design
(interviews, surveys, etc.); collect data; analyze data; provide feedback. The
feedback report should have these sections: executive overview, description of
process, summary of findings, preliminary conclusions, recommendations,
We get some more
background on androgogy and adult learning styles.
overload: minimize lecture and use key learning points, checklists, graphs,
models, etc.; have participants do the work; create chunks of content and
distribute it incrementally; design workbooks for participants to follow-along;
create job aids.
We've got some
in-depth instructional style surveys which are pretty cool. Basically, you have
sellers, professors, entertainers, and coaches.
We have to write
some learning objectives. There needs to be an instructional plan, including:
- Part 1
- Course description
- Learning outcomes
- Participant preparation
- Instructional materials --
document list; equipment list; media list
- Reference list
- Facility check-off list
(i.e., how do we put the room together with materials, etc.)
- Part 2
- Time frames
- Content outline -- ideas,
concepts, principles, skills
- Training aids and materials
- Trainer's notes
include instructional methods and developing materials. A trainer's guide
should include: TOC, introductory background material, presentation guide with
facilitation tips, instructional plan, master copies of materials, list of
materials, and resources.
training is good. Videos can be good. Lectures are okay but some things are
better: question and answer; interesting visual aids; stories and anecdotes;
case problems; examples and analogies. Other techniques include group inquiry,
guided discussion, active knowledge sharing, peer lessons, jigsaw, learning
Role plays --
scripted, coaching, spontaneous, or rotating trio.
Case studies --
story form, with characters, and realistic dialog, and appropriate specific
details, make the story easy to follow, provide discussion questions
The book then gives
us some details about room design. Nice.
We have to be aware
of openings or "set induction". We could use icebreakers or openers.
- Human scavenger hunt.
"Find someone in the room who…"
- The party… I don't think I
would use this.
- Instant assessment. Prepare
cards with different colors and letters. You basically get them to hold up
cards when you ask questions such as: "My main motivation for
attending this session is…"
- What do you want to know?
- What's in it for them?
involves things like videos, role plays, simulations, storytelling (parables,
Clothing: dress a
bit above your audience. Don't wear anything distracting.
Make good slides.
Use flip charts appropriately.
might be valuable. For example, a cow bell to get people's attention. Giveaways
are helpful. As are games.
Finally, we have
some more guidelines on evaluation of training.
- Talbot. 2011. Training in
organizations : a cost-benefit analysis.
The TOC looks good
but my brain is fairly shot so I'm skipping it. I can always come back to the
I still have the
question about best practices in instructional design. This is the type of
question that has likely been answered by some sort of government body at some
point or by the aforementioned IBSTPI.
- The 2012 IBSTPI
Instructional Design Competencies (http://ibstpi.org/download/?did=2705&file=0).
Basically, they need
to have professional foundations, planning and analysis skills, design and
development skills, evaluation and implementation skills, and management
skills. There is a breakdown of performance statements. Moving on...
- There's an "instruction
design criteria checklist" from Michael Fors of Microsoft (http://www.unitar.org/hiroshima/sites/unitar.org.hiroshima/files/17_AF07WSII_Instructional_Design_Criteria_Checklist.pdf). The list is actually a
great synopsis of everything I just researched. It will go to Evernote.
- There's something called the Canadian
Association of Instruction Designers. It, too, has a competency model (http://accp-caid.org/docs/ACCP_Competencies_En.pdf).
- The IEEE offers some
resources, including a Reference Guide for Instruction Design (http://www.ieee.org/education_careers/education/reference_guide/index.html). It offers some great links
to a variety of different resources. The "Strategies for teaching at
a distance" from Idaho State University looks interesting. And,
there's yet another association: International Association for Continuing
Education and Training. This site is worth a return visit. The whole model
is based on the Dick and Carey model, see: http://www.ieee.org/documents/The_Systematic_Design_of_Instruction.pdf.
- Let's look at IACET. It
offers accreditation for training centers, research, resources and
references. It also has some sort of standard. The standard measures a
provider's program development across a 10 different categories (http://iacet.org/iacet-standard/ansiiacet-standard). And it complies with ANSI
recommendations for standards development. The standard is compatible with
the "ADDIE" model, which is a framework of generic processes for
instructional designers and training developers. ADDIE stands for
Analysis, Design, Development, Implementation, Evaluation. It is an
"Instruction Systems Design" (ISD) model such as the Dick and
Carey model or the Kemp ISD model. Apparently the ADDIE model was
initially create for military inter-service training... so there might be
some .mil information on it. Part of IACET efforts, however, are to create
a standard based on a canonical list of resources (http://iacet.org/standard-references-list). The most recent reference
is Morrison et al. 2006. Designing effective instruction (5th ed.). Wiley & Sons.
- For what it's worth,
Wikipedia has some interesting content on the history of instructional
design (http://en.wikipedia.org/wiki/Instructional_design). It provides some other
acronyms of interest for learning design: PALO, IMS LEARNING DESIGN,
LDL,SLD 2.0, TELOS, RELOAD, etc.
- Let's go back to our list of
federal information. There's another checklist available from the USGS which I should be able to
Department of Labor has published a collection of best practices for
(http://www.dol.gov/oasam/learninglink/2011BestPractices.pdf). It uses ADDIE and the
Kirkpatrick model of evaluation. It provides a pretty cool taxonomy of
course breakdowns, etc.
It also talks about
We get some good
tactical guidelines for interactivity:
- engage the
and lively language, fascinating facts; speak directly to the learner;
active voice; second person; imperative voice
relevance; explain long and short term benefits; consequences of not
learning the information
interaction every 3 to 5 screens. But don't introduce unnecessary breaks
content into segments with questions, reviews, and summaries
- ask as many
questions as possible
questions at the on-the-job level
rhetorical questions to get learners to think ahead
where learners can learn by exploring
There are different
types of web learning.
- Category 1. sequential telling with
little user control.
- Category 2. Includes simple simulations
(e.g., turn dials, rotate switches, could include basic images, videos,
- Category 3. High simulation. Learners
might have to alternate between multiple screens. Uses more peripherals.
Good for operation and maintenance procedures.
- Category 4. Emersion. Generally
supports certification or qualification requirements. Lots of
interactivity and branching but no AI.
You need to be s.508
Section 1.2 provides
course structure which is typically: course introduction, module, lesson,
A module is a group
of lessons. The module introduction should have two screens: overview of the
module and description of what the learner will learn; second screen has module
objectives. There should also be a module menu, lessons, and a module summary.
to Terminal Learning Objectives and include various topics, each associated
with a Enabling Learning Objective. The maximum time for lesson completion
(seat time) is 45 minutes.
Lessons should have
the introduction screens, content and practice screens, lesson summary screens,
and a lesson knowledge check.
At the end of the
course, there should be a "comprehensive knowledge check" (a test).
Provide remediation for questions that learners answer incorrectly.
Scenarios are good.
Use case studies, stories, etc. Ask: "What would you do in this
You generally need a
Detailed Content Outline or DCO.
the overall Course
Design Plan (CDP) should include:
- Introduction. project overview, target
description, course outline, course flow chart, estimated number of
screens, contact time
overall approach to course, scenarios, and plan for supplemental material
practice and assessment strategies
strategy for each module and lesson. Terminal Learning Objectives, Enabling Learning
Objectives, descriptions, instructional strategies, assessment strategies,
contains guidelines for actually building out the storyboard.
It provides some
best practices for screen design:
- Graphics left; text right
- Top down, left-to-right
- Fewest steps necessary to
deliver the right information
- No timed effects
- Once concept, procedure, or
item of instruction per screen
- Consistent use of color
- Stay compliant with s508 of
- Don't violate copyright
- Active voice, 2nd person,
- Simple, concise, consistent
- Avoid hyphenation (except for
- Avoid jargon and slang
- Maintain parallel
construction and noun-pronoun agreement
- use narration when the visual
channel is overloaded, there is a need for immediate learner response, or
if the message is simple
- Narration should complement
text. It should be somewhat different from onscreen text.
- an audio script for every
- Avoid long pauses in visuals
to support narration
- Make clear concept
- The storyboard should conain
- Keep language simple, active,
- Format text appropriately if
the acronym is to be spelled out (e.g., D-O-L)
For visual elements:
- They should relate to the
- Provide recurring information
in consistent locations
- Maintain a constant
- Get consent for logos
- Avoid clutter
- Re-use graphics for basic
- Use these rations: 10%
complex (VR, 3D); 30% simple animations; 60% photo /illustrations
- Don't use too many colours at
- Have sufficient contrast
between background and text
- Use short clips to reinforce
- Use appropriate techniques
(talking head, show and tell, interview, panel, simulation, dramatization)
- User video if the content
- Only use special effects
rarely for emphasis
- Should initiate only on
We have the four
level Kirkpatrick evaluation model. DOL aims for level 2 or 3.
Level 2 can be
assessed via practice questions, knowledge checks, and pre-post testing.
Level 3 testing is
conducted 6 weeks to 6 months after training.
- Now we have something from
the United States Distance Learning Association (another one!?!) called _An
instructional media selection guide for distance learning_ (http://www.ien.idaho.gov/media/Best_Practices/USDLAReport.pdf). The USDLA has conferences
and certifications. It seems to focus more on providers, online
The document notes
that distance learning must include -- at a minimum -- physical distance
between student and teacher; content providing organization; curriculum;
measurement of learning. Apparently there have been over 70 different learning
styles identified but low validity and reliability casts some questions. It
also cites UCLA's Cooperative Institutional Research Program work on
generational differences. Apparently, they are slow and gradual. There are no
significant differences from one generation to the next (the study did,
however, note a pattern of increased entitlement, decreasing literacy, and
decreasing factual knowledge).
to be considered include:
- Identification of knowledge
and skills gaps
- Assessment and measurement
- Level of interaction
- Instructional strategies
- Complexity of content
- Rate of content change
- Level and domain (cognitive,
affective, psychomotor) of learning objectives
- Delivery issues to consider
include audience size and distribution, cost (in-house vs outsource;
availability of infrastructure; delivery hardware endpoints --
teleconferencing, satellite, WAN/LAN, TV/monitor, portability)
Lots of good
weeds-level content about various topics, including case studies.