More lessons on knowledge capture from NASA

It turns out that I'm not quite done with the NASA documents. I stumbled across another interesting report JSC-66136r1 Space shuttle guidance, navigation, and rendezvous knowledge capture reports (October 2011).

Knowledge capture is a crucial issue for NASA given the nature of its mission: big, dangerous, expensive, constantly shifting, etc. The report talks about the challenges of actually capturing knowledge largely due to granularity. Histories, for example, are interesting but typically too general for specialist use while other types of codification such as computer code and detailed reports are simply too narrow. It's a Goldilocks problem.

The good stuff starts on page 32. The reports notes that "knowledge capture and management" includes four steps:

1. People
2. Content
3. Process
4. Information Technology

The report then laments that knowledge capture efforts often start with IT and work backwards, to poor results. Specifically, it is difficult to find "technical subject matter experts that possess the written, verbal, and graphic communications skills need[ed] to effectively perform knowledge capture".

The authors are quite adamant that the lower steps in the model are most important and note that: "While a significant amount of knowledge capture... can be performed at the lower levels of corporations and government agencies, visible leadership and support from senior management is needed to ensure the success and continuity of such efforts, particularly for the creation and implementation of knowledge management processes... and identification, procurement, and integration of information technology."

Some knowledge capture techniques include:

• just-in-time learning via social networking, but it is susceptible to limitations of human memory
• formal reports
• informal memos written in complete sentences with tables, illustrations, and references
• well-written status reports

"Charts with bullet points and spreadsheets omit much background information that may be understood by the original audience, but will not be known to future researchers. This makes charts and spreadsheets difficult to learn from."

Documentation should include experiences, not just lessons learned. People tend to remember stories and they are effective for teaching and transferring knowledge. Exemplars of good reports include "Hypergolic propellants: the handling hazards and lessons learned from use" the collection of "System Failure Case Studies".

A multi-step process would include:

1. Identification -- conduct research to identify lessons. This involves primary source materials and interviews. Research and interviewing skills are required to identify key knowledge, experience and lessons.
2. Creation -- create a story or narrative using hte results of the research in step 1. This requires skills in reasoning an din verbal, written, and visual communication.
3. Capture -- document the story in some form of media (formal report, informal memo, presentation, training material, procedure, case study, video, audio, etc.)
4. Sharing/Retrieval -- make the media available.

Presentations are troublesome artifacts.  Additional valuable information includes: why was the presentation created? What discussion was conducted during the presentation? What action was taken, if any, as a result?

A basic set of questions includes:

1. What did we do?
2. Why did we do it?
3. When did we do it?
4. How did we do it?
5. Why did we do it that way?
6. What happened?
7. What challenges did we encounter?
8. What did we learn?
9. Is there something we wish we would have done differently?

Interesting. The references lead me to two other docs:

Goodman's Best practices for researching and documenting lessons learned

• The scope of the report must be determined before starting out.
• People working on lessons learned projects must have some aptitude for writing, researching, and creating engaging documents.
• Define the report audience, scope, and outline
• Lessons learned reports are not requirements documents
• Address both successful and challenged projects
• Keep lessons learned separate from technical history or process descriptions
• A section should have only one author
• Standardize production tools, review processes, and editorial process.
• Recognize that documentation can come from anywhere
• Interview SMEs (but recognize that "the optimal source of lessons learned is usually documentation created at the time the lessons were learned")
• Explain the goals, process, and value of lessons learned to the SMEs
• Document lessons learned during the project and at project conclusion

Writing the report

• Stick to the facts
• Be objective
• Document lessons so that they will be easy to understand and apply
• Include high-level background material for context
• Avoid duplicating low-level detail that is available elsewhere

Review and revision

• Review SMEs who did not contribute
• Build consensus
• Assign priorities to suggestions for expanding the report scope

The second document is Goodman's Knowledge capture and management for space flight systems

The introduction gives us an interesting statement: "Follow the transition from the development to the flight phase, loss of underlying theory and rationale governing design and requirements occur through a number of mechanisms. This degrades the quality of engineering work resulting in increased life cycle costs and risk to mission success and safety of flight."

Knowledge capture is important for the development of professionals. In most cases, engineers and others don't have the opportunity to work on many small projects and develop appropriate technical skills.

Sources of knowledge include:

1. Program histories
2. Textbooks, university classes, short courses/continuing education
3. Journal articles and conference papers
4. Internal sources
1. presentations and technical reports
2. software requirements documentation
3. derivations of equations
4. databases and archives

Brain book? See 32.

NASA maintains an informal knowledge base that allows users to submit/capture informal artifacts in a very loosely structured manner. Donors get their material back (and, if applicable, a digital copy).

The Columbia accident reports clearly pointed to the limitations of presentations as a means of knowledge transfer. Detailed reports are better.