Facetation: Developing training material -- a round up

Okay. I've made the trip to the business library… which isn't actually where I thought it was. The new library is beautiful but lacks the quaintness of the old library. That said, let's get into this project.

Training. Specifically, I have been tasked to develop some training material for something we do internally and that I have been doing for a long time. I'm developing some training that should be delivered electronically in either a seminar format or self-serve from the LMS.

I've ripped a bunch of books from HF5549.5.T7. Let's see what we find.

Hardingham. 1996. Designing training.

The intro looks good. It lays out a thought model and reviews choices for different types of training. Venue choices. Costs. You have to consult to understand training objectives. These things should be SMART. Manage politics, particularly blockers and contributors.

There are five key concepts in training design:

Credibility
Commitment
Risk
Attention
Maneuverability

Fundamental design principles:

Establish credibility
Elicit Commitment
Attract attention
Allow maneuverability

Signpost. Lots. Vary pace and rhythm. Chunk content. Map the participants world. Give participants choices. Surface objections. Balance theory and practice. Design for closure (entertainment, presentation of certificates, closing speeches, closing feedback, presentation of gifts).

Process interventions are a form of training that could involve small group work, facilitation, direct process manipulation.

There might be differences in groups. Give participants choices and design in layers.

Classic challenges: training the unwilling; money; training for the easily bored; training for people who won't work together.

Overall, it's a little book but I like it. Perhaps we could use it as a basis for building workshops i.e., training on training.

Stolovitch and Keeps. 2011. Telling ain't training.

This book was my reference anchor since it was published by the ASTD. Let's see what it has to say.

It starts off with a decent overview of how humans learn. There are different types of learning: declarative and procedural (e.g., riding a bike). Learning requires ability, prior knowledge, motivation (value, confidence, and mood).

We've got some more principles of adult learning:

Readiness. Adult learners must participate in and contribute to the learning process.
Experience. Ault learners must be able to immediately apply what they are learning.
Autonomy. They must see the benefits of what they are doing.
Action. Content and activities should be integrated.

Participatory learning: scenarios, role plays, cases, brainstorming, practice activities.

The learner must be the center of attention. Trainer success comes from the success of the attendees. Andragogy = adult learning. Pedagogy = kid learning. Answer: What's in it for me? Never assume they don't have experience. Let them contribute. Focus on immediate application. Training must be applied on the job. Train others as you would be trained.

There's a five step model for structuring training. See:

<01>

There's also some good planning sheets.

<02>

<03>

Learners need to be engaged. They have different meta-cognitive skills: planning, selecting, connecting, tuning, monitoring. There are also the deficient factors: ability, prior knowledge, and motivation.

<04>

We have a few different cognitive strategies: clustering, spatial (hierarchies, flow-charts, matrix), advance organizers, image-rich comparisons (analogies, metaphors, comparisons), repetition, memory aids (mnemonics: acronyms, acrostics, rhymes, key words).

Four types of training: receptive (telling), directive (do it this way), guided discovery (cases), exploratory learning.

Gg - Perhaps guided discovery is the best way to start F2F training.

<05>

<06>

Potential activities:

Better me. How can you do better than me at a procedural task?
Concentration. Match cards, etc.
Confrontation. Groups of three. Two in confrontation, one observer. Timed rounds. Participants cycle through five strategy choices and observe outcomes. Six scenarios, each lasting 5 minutes. Debrief.
Critical list. After instruction, members have to produce the most important points. Iterate to find most important points.
Crypto cluster. Decipher a puzzle.
Domino Effect. 20-30 cards with a term on one side and a definition on the other. Participants must make a chain.
Exam cram. Divide into study groups. Give a test to each team. They have 20 minutes to cram. The leader can actually test or not.
Facts-in-Five. Give a card with a 5x5 matrix. Columns are categories; rows have letters of the alphabet. Fill the matrix with aligning keywords.
Great debate. Take some debatable issues. Divide into teams. They have 10 minutes to prep pro or con for a 2 minute debate.
Hit or myth. Participants get a list of 10 statements that are either true or false. They have to decide.
Jeopardy. Given answers; have to determine questions. Could be timed, scored, or involve increased difficulty.
Jigsaw. Class is divided into teams that are given documentation about one part of a topic. The teams prepare a presentation about their specific part including format and how to make it interesting. Chain the presentations into a lecture. End with a quiz and a debrief.
Lecture team quiz. Break into teams. Lecture. Teams must prepare a quiz question and answers. Exchange.
Letter game. Break into teams. Each team gets an envelope with a problem written on it. Each team gets one fewer cards than envelopes. They write their solution to the problem on a card, their identifier on the back, insert into the envelope, and pass to the next group who does the same thing. Teams than rank order the solutions and points are assigned.
Listening teams. Each team has to listed for key parts of information and repeat.
Mismatch. Describe 4-8 scenarios involving a question and an answer. The question and answer are mismatched.
Ours versus theirs. Create a lists of 10 to 15 statements. Beside each statement create three checkboxes associated with competitive providers or products. Ask participants for feedback.
Police interrogation. There is a topic and quiz. There is 30 minutes. The participants must force the content from the participants.
Press conference. Teams of reporters. They must draw out information from an expert. The conference has four or five themes. Teams create questions related to the themes on colour-coded cards. Teams then take on a theme. They have 15 minutes to review questions and then 7 minutes to grill the expert.
Quiz game. After a day of instruction, participants are given a slip of paper to identify a single point worth remembering. This is the answer. The participant also comes up with a question to elicit the answer.
Slap jack. Instructor creates cards with a term on each card. Participants are grouped into teams of 3-5. The instructor calls out knowledge domain, each participant throws a card, participant has to correctly identify the domain and touch/slap the card.
Techno challenge. Alternate between roles of customer and sales consultant. Rounds are 99 seconds. Clients ask questions; consultants respond to as many as possible. Good review.
Terminology tussle. Everyone gets a BINGO card with terms. Definitions are read out related to the terms.
They say, we say. Teams of 3 or 4. Half as competitor sales consultants; half as home team. A round = 15 seconds for one side to introduce a feature + 30 seconds for the other side to rebut.
True grid. Blank grid. Participants can apply axis titles and fill it in or the axis titles can be preassigned.

Tests and verification are different things.

There are advantages to each approach:

<07>

Four steps to training evaluation: 1. participant reaction; 2. immediate learning at the end of the session; 3. on-the-job application post-training; 4. results -- did something change organizationally?

RLO = reusable learning object.

Online learning best practices:

Right content
Strong alignment with business goals
True interactivity
Valued experience i.e., fun
Few distractions
Useful on the job
Powerful feedback
Valid assessments
Good human factors
Integrated follow-up

Important decisions:

Who is the course for and why?
What are the content and context?
How deeply should it go?
What type of learning is needed?
How much time do you have?
How will the training be delivered?
What are the metrics?

"Blended Learning" involves information (repositories, job aids, procedures), instruction (training, online, external), and collaboration (social media, CoP).

See: smilkstein 1993. acquiring knowledge and using it. ERIC 382 238.

That was a good one. My brain is getting full and we're almost two hours into this effort. What's next?

<08>

<09>

<10>

Kosalka, et al. 2013. Instructional designer competencies.

It is a volume of the IBSTPI (International Board of Standards for Training, Performance and Instruction). So that's interesting.

So this is about competencies for instructional design… cool. If I was coming up with job descriptions. This thing is very much an auditing standard rather than a how-to guide. Regardless, perhaps I should draw a distinction between "training" and "instructional design".

Phillips. 2010. Measuring and evaluating training.

Another ASTD title. It is a collection of essays that basically follow Evaluation Planning, Data Collection, Data Analysis, Measurement and Evaluation. Some interesting looking stuff in here but it's probably more academic/mature than my current information needs mandate.

Russ-Eft, et al. 2008. Evaluator competencies.

As long as we're on the evaluator side of things, let's take a look at this book. Koszalka is an author and the forward is by the past president of the American Evaluation Association. Yet another association to deal with!

And… it's not what I'm looking for. Next.

Lawson. 2009. The trainer's handbook.

The title looks promising. Let's go.

So we need to an needs assessment: identify problem or need; determine needs-assessment design (interviews, surveys, etc.); collect data; analyze data; provide feedback. The feedback report should have these sections: executive overview, description of process, summary of findings, preliminary conclusions, recommendations, potential barriers.

We get some more background on androgogy and adult learning styles.

Preventing cognitive overload: minimize lecture and use key learning points, checklists, graphs, models, etc.; have participants do the work; create chunks of content and distribute it incrementally; design workbooks for participants to follow-along; create job aids.

We've got some in-depth instructional style surveys which are pretty cool. Basically, you have sellers, professors, entertainers, and coaches.

We have to write some learning objectives. There needs to be an instructional plan, including:

Part 1

Title
Course description
Learning outcomes
Length
Format/methodology
Audience
Participant preparation
Instructional materials -- document list; equipment list; media list
Reference list
Facility check-off list (i.e., how do we put the room together with materials, etc.)

Part 2

Time frames
Content outline -- ideas, concepts, principles, skills
Training aids and materials
Trainer's notes

Other considerations include instructional methods and developing materials. A trainer's guide should include: TOC, introductory background material, presentation guide with facilitation tips, instructional plan, master copies of materials, list of materials, and resources.

Co-operative training is good. Videos can be good. Lectures are okay but some things are better: question and answer; interesting visual aids; stories and anecdotes; case problems; examples and analogies. Other techniques include group inquiry, guided discussion, active knowledge sharing, peer lessons, jigsaw, learning tournament, etc.

Role plays -- scripted, coaching, spontaneous, or rotating trio.

Case studies -- story form, with characters, and realistic dialog, and appropriate specific details, make the story easy to follow, provide discussion questions

Inbasket activities

Simulations

Instructional games

The book then gives us some details about room design. Nice.

We have to be aware of openings or "set induction". We could use icebreakers or openers.

Human scavenger hunt. "Find someone in the room who…"
The party… I don't think I would use this.
Instant assessment. Prepare cards with different colors and letters. You basically get them to hold up cards when you ask questions such as: "My main motivation for attending this session is…"
What do you want to know?
What's in it for them?

Interactive delivery involves things like videos, role plays, simulations, storytelling (parables, fables).

Clothing: dress a bit above your audience. Don't wear anything distracting.

Make good slides. Use flip charts appropriately.

Theatrical tools might be valuable. For example, a cow bell to get people's attention. Giveaways are helpful. As are games.

Finally, we have some more guidelines on evaluation of training.

Talbot. 2011. Training in organizations : a cost-benefit analysis.

The TOC looks good but my brain is fairly shot so I'm skipping it. I can always come back to the library!

I still have the question about best practices in instructional design. This is the type of question that has likely been answered by some sort of government body at some point or by the aforementioned IBSTPI.

First off:

The 2012 IBSTPI Instructional Design Competencies (http://ibstpi.org/download/?did=2705&file=0).

Basically, they need to have professional foundations, planning and analysis skills, design and development skills, evaluation and implementation skills, and management skills. There is a breakdown of performance statements. Moving on...

There's an "instruction design criteria checklist" from Michael Fors of Microsoft (http://www.unitar.org/hiroshima/sites/unitar.org.hiroshima/files/17_AF07WSII_Instructional_Design_Criteria_Checklist.pdf). The list is actually a great synopsis of everything I just researched. It will go to Evernote.

There's something called the Canadian Association of Instruction Designers. It, too, has a competency model (http://accp-caid.org/docs/ACCP_Competencies_En.pdf).

The IEEE offers some resources, including a Reference Guide for Instruction Design (http://www.ieee.org/education_careers/education/reference_guide/index.html). It offers some great links to a variety of different resources. The "Strategies for teaching at a distance" from Idaho State University looks interesting. And, there's yet another association: International Association for Continuing Education and Training. This site is worth a return visit. The whole model is based on the Dick and Carey model, see: http://www.ieee.org/documents/The_Systematic_Design_of_Instruction.pdf.

Let's look at IACET. It offers accreditation for training centers, research, resources and references. It also has some sort of standard. The standard measures a provider's program development across a 10 different categories (http://iacet.org/iacet-standard/ansiiacet-standard). And it complies with ANSI recommendations for standards development. The standard is compatible with the "ADDIE" model, which is a framework of generic processes for instructional designers and training developers. ADDIE stands for Analysis, Design, Development, Implementation, Evaluation. It is an "Instruction Systems Design" (ISD) model such as the Dick and Carey model or the Kemp ISD model. Apparently the ADDIE model was initially create for military inter-service training... so there might be some .mil information on it. Part of IACET efforts, however, are to create a standard based on a canonical list of resources (http://iacet.org/standard-references-list). The most recent reference is Morrison et al. 2006. Designing effective instruction (5th ed.). Wiley & Sons.

For what it's worth, Wikipedia has some interesting content on the history of instructional design (http://en.wikipedia.org/wiki/Instructional_design). It provides some other acronyms of interest for learning design: PALO, IMS LEARNING DESIGN, LDL,SLD 2.0, TELOS, RELOAD, etc.

Let's go back to our list of federal information. There's another checklist available from the USGS which I should be able to parse (http://www.usgs.gov/humancapital/documents/tel_instructionalmodeule.pdf).

The Department of Labor has published a collection of best practices for Web-based training (http://www.dol.gov/oasam/learninglink/2011BestPractices.pdf). It uses ADDIE and the Kirkpatrick model of evaluation. It provides a pretty cool taxonomy of course breakdowns, etc.

<11>

It also talks about training flow:

<12>

We get some good tactical guidelines for interactivity:

engage the learner. vivid and lively language, fascinating facts; speak directly to the learner; active voice; second person; imperative voice
address WIIFM. explain relevance; explain long and short term benefits; consequences of not learning the information
provide interaction every 3 to 5 screens. But don't introduce unnecessary breaks
groups content into segments with questions, reviews, and summaries
ask as many questions as possible
ask questions at the on-the-job level
use rhetorical questions to get learners to think ahead
consider presentations where learners can learn by exploring

There are different types of web learning.

Category 1. sequential telling with little user control.
Category 2. Includes simple simulations (e.g., turn dials, rotate switches, could include basic images, videos, and audio).
Category 3. High simulation. Learners might have to alternate between multiple screens. Uses more peripherals. Good for operation and maintenance procedures.
Category 4. Emersion. Generally supports certification or qualification requirements. Lots of interactivity and branching but no AI.

You need to be s.508 compliant.

Section 1.2 provides course structure which is typically: course introduction, module, lesson, topic.

A module is a group of lessons. The module introduction should have two screens: overview of the module and description of what the learner will learn; second screen has module objectives. There should also be a module menu, lessons, and a module summary.

Lessons correspond to Terminal Learning Objectives and include various topics, each associated with a Enabling Learning Objective. The maximum time for lesson completion (seat time) is 45 minutes.

Lessons should have the introduction screens, content and practice screens, lesson summary screens, and a lesson knowledge check.

At the end of the course, there should be a "comprehensive knowledge check" (a test). Provide remediation for questions that learners answer incorrectly.

Scenarios are good. Use case studies, stories, etc. Ask: "What would you do in this situation?"

You generally need a Detailed Content Outline or DCO.

the overall Course Design Plan (CDP) should include:

Introduction. project overview, target audience, purpose
Course structure. description, course outline, course flow chart, estimated number of screens, contact time
Course design strategy. overall approach to course, scenarios, and plan for supplemental material
Course practice and assessment strategies
Design strategy for each module and lesson. Terminal Learning Objectives, Enabling Learning Objectives, descriptions, instructional strategies, assessment strategies, content outline

The document contains guidelines for actually building out the storyboard.

It provides some best practices for screen design:

Graphics left; text right
Top down, left-to-right presentation
Fewest steps necessary to deliver the right information
No timed effects
Once concept, procedure, or item of instruction per screen
Consistent use of color
Stay compliant with s508 of ADA
Don't violate copyright

recommendations for language:

Active voice, 2nd person, conversational tone
Simple, concise, consistent language
Avoid hyphenation (except for compounds)
Avoid jargon and slang
Maintain parallel construction and noun-pronoun agreement

For audio:

use narration when the visual channel is overloaded, there is a need for immediate learner response, or if the message is simple
Narration should complement text. It should be somewhat different from onscreen text.

For audio development/script development:

an audio script for every synched element
Avoid long pauses in visuals to support narration
Make clear concept transitions
The storyboard should conain the script
Keep language simple, active, and direct
Format text appropriately if the acronym is to be spelled out (e.g., D-O-L)

For visual elements:

They should relate to the content
Provide recurring information in consistent locations
Maintain a constant perspective
Get consent for logos
Avoid clutter
Re-use graphics for basic concepts
Use these rations: 10% complex (VR, 3D); 30% simple animations; 60% photo /illustrations

For graphics:

Don't use too many colours at once
Have sufficient contrast between background and text

For videos:

Use short clips to reinforce learning objective
Use appropriate techniques (talking head, show and tell, interview, panel, simulation, dramatization)
User video if the content requires motion

For animation:

Only use special effects rarely for emphasis
Should initiate only on learner action

We have the four level Kirkpatrick evaluation model. DOL aims for level 2 or 3.

Level 2 can be assessed via practice questions, knowledge checks, and pre-post testing.

Level 3 testing is conducted 6 weeks to 6 months after training.

Overall, good document.

Now we have something from the United States Distance Learning Association (another one!?!) called _An instructional media selection guide for distance learning_ (http://www.ien.idaho.gov/media/Best_Practices/USDLAReport.pdf). The USDLA has conferences and certifications. It seems to focus more on providers, online universities, etc.

The document notes that distance learning must include -- at a minimum -- physical distance between student and teacher; content providing organization; curriculum; measurement of learning. Apparently there have been over 70 different learning styles identified but low validity and reliability casts some questions. It also cites UCLA's Cooperative Institutional Research Program work on generational differences. Apparently, they are slow and gradual. There are no significant differences from one generation to the next (the study did, however, note a pattern of increased entitlement, decreasing literacy, and decreasing factual knowledge).

<13>

Instructional issues to be considered include:

Identification of knowledge and skills gaps
Assessment and measurement tools
Level of interaction
Instructional strategies
Complexity of content
Rate of content change
Level and domain (cognitive, affective, psychomotor) of learning objectives
Delivery issues to consider include audience size and distribution, cost (in-house vs outsource; availability of infrastructure; delivery hardware endpoints -- teleconferencing, satellite, WAN/LAN, TV/monitor, portability)