Exploring the Analytical Processes of Intelligence Analysts

Related comments: no other posts on this paper, so I commented on William's blog.


Summary:
This paper presents the results of an observational case study of intelligence analysts (IAs) at work, conducted by a group of three researchers from Pacific Northwest National Laboratory (George Chin, Jr., Olga A. Kuchar, and Katherine E. Wolf).  Research cases like this one have gained more interest over the past decade; these efforts attempt to find ways of developing new information technologies and visualization tools to improve or assist intelligence analysis.  The bulk of the paper describes two case scenarios presented to a group of IAs.  The first scenario charted the intelligence-gathering and analysis methodology of the IAs, and the second scenario examined how IAs collaborated in real-time to carry out a group analysis.  In each scenario, IAs were given material similar to what one would find in an actual case, such as group background info, intercepted communications, electronic files, witness statements, etc.  New information was also given periodically through the course of the analysis to see how it would be integrated with the existing information.  The researchers found that the strategies used in analysis varied from IA to IA, but that there were a few that were most prominent -- the IAs all followed a similar sequence of steps to conduct their analysis.  The first step, obviously, is information gathering.  All of the IAs printed out all the case files despite being given electronic copies, and all the IAs proceeded to arrange or order the information in some way so that they could extract relevant facts.  Once data was ordered and facts were extracted, IAs attempted to identify patterns and trends in the evidence.  Most of the IAs displayed relationships in the data visually, either via a hand-drawn diagram or an electronic aid like a graph or spreadsheet.  The weight given to certain pieces of evidence shifted for each IA depending on the credibility of the source or method of obtainment.  The IA's judgement in this area was based largely on his or her previous experience.  In the second scenario, IAs collaborated to corroborate data and share resources rather than compare and pass on conclusions; they worked to form a single more accurate analysis.  


From all this observation, the researchers were able to identify a few areas in which technology might aid IAs in their work.  Computers could auto-generate standard analysis views based on given facts and relationships, improving the speed and efficiency of an IA's analysis; optical character recognition tools for converting oft-used sketches into text or graphs to enable easy electronic storage and sharing; taking advantage of multiple displays to accomodate simultaneous viewing of many documents; and integrating link analysis and case management tools in a way that would provide more sophisticated pattern-matching so IAs could locate and draw conclusions from past cases more quickly.


Discussion:
Intelligence analysis is a critically important field that gets a pretty wonky dramatization on television (see 24 for evidence), but it nonetheless often requires the snap decisions and life-or-death implications depicted in such mediums.  One of the things I found most encouraging about this study was the willingness of IAs to collaborate with one another, and the results they achieved by sharing facts and helping each other eliminate information.  The single largest problem facing our intelligence community today isn't al-Qaeda or missing Soviet nukes -- it's the lack of collaboration between various agencies and a competitive attitude that weakens our ability to assess and identify threats.  It's in this area, not in multiple displays or workflow management, that technology researchers need to develop new tools.  Something like a secure digital "whiteboard" where information could be shared, edited and annotated across multiple global agencies would be extremely useful in enabling that kind of collaboration.