Friday, May 30, 2014

Spring Directed Field Work: Final Artifacts

The final artifacts for my Spring 2014 Directed Field Work are listed below
(Click to open in a new browser):

Spring Directed Field Work Week 9

This week I continued to finish up the notes and slides of the 14 cognitive functions of advanced analysis in Hall and Citrenbaum’s textbook. I wrote their definitions and made notes in regards to relationships between each other.  Here are a couple screen shots of some of the cognitive functions:

 

Week 9 Slide 1

week 9 Slide 2

Week 9 Slide 3 

Week 9 Slide 4

Week 9 Slide 5

Week 9 Slide 6

Because of how text heavy the book is, in between I took breaks and downloaded journal articles regarding “deception detection”, “key assumptions”, with “social media” from some of the online databases PNNL has subscriptions to and from other online sources.  I downloaded the articles and uploaded to my mentor’s Sharepoint site.  As I was researching I located a funding opportunity and sent that information to him so he could send to his colleagues to possibly apply to. 

I also met with my mentor to discuss any loose ends that needed to be tied up and if any other information was needed before ending my directed field work stint with him.  I wanted to mention I did not mention my mentor’s name throughout the weeks of this directed field work stint. 

My final artifacts included:

Friday, May 23, 2014

Spring Directed Field Work Week 8

While my mentor was on travel I began reading about the intelligence analysis techniques listed in an additional textbook he had given me and started a slide deck of notes from the content.

Intelligence Analysis: How to Think in Complex Environments by Wayne Michael Hall and Gary Citrenbaum, 2010

Intelligence Analysis

The methods listed in this book had some very similar characteristics as the previous text, but were not all structured analytical techniques.  These were a mixture of structured and unstructured techniques and gave examples in terms of complex environments supporting military operations, modern intelligence analysis, operational environments, and adversaries.  It didn’t directly discuss how the techniques can fit in with social media, but can certainly be inferred.  The text of this book was written in a  very technical, higher level of thinking than the previous textbook by Heuer, but was fascinating to see how the U.S. government uses these techniques in connections with terrorism, adversaries, and national security.

The purpose of the intelligence officer and some key definitions listed in the introduction of the text I noted in the slide deck.

Week 8 PPT 1

There are 14 Cognitive Functions of Advanced Analysis and were separated by individual chapters.  I started to take notes for each of the sections and will hopefully complete sometime next week.  The book is extremely text heavy and is time consuming to get through.  But I am highlighting content that best describes the techniques listed in each chapter.

Week 8 PPT 2

Based on last week’s meeting I downloaded the ICDs (Intelligence Community Directive standards and policies) a co-worker mentioned would be useful for my mentor’s research and uploaded to his share drive to be used at a later time.  Additionally I uploaded to the share drive references mentioned in Heuer’s text book that were readily available online. 

Friday, May 16, 2014

Spring Directed Field Work Week 7

This week my mentor and I met with the research scientists who are familiar with social media analysis for intelligence purposes.  We covered the slide deck that I created the week before with the higher level techniques spliced out of the grand slide deck of all techniques.  The team determined there were 15 out of the 52 we wanted to focus on.  

Table 1. The Chapters, Section and Title of Highly Relevant Techniques for Social Media Analysis

Week 7 Table 1

*Chapter and sections highlighted in green the team has decided should be investigated further in the future

Short descriptions as to why these techniques are applicable to social media were taken and are listed below:

  • Process Maps and Gantt Charts: Help intelligence analysts track, understand, and monitor activities of interest
  • Issue Redefinition: People think that social media is the end all be all to explain all problems out there in the world, but in fact each problem is a specific and unique case when social media can help answer questions
  • Chronologies and Timelines: Have a special case for social media, information hopping between different channels, news versus social media
  • Network Analysis: My mentor has subject domain knowledge on this technique so we didn’t focus our discussion on this
  • Starbursting: Different ways people represent themselves, locations are different, easier to be masked, another research scientists calls this "conjoinment" analysis, organization versus individual
  • Cross-Impact Matrix: You have to consider all variables affecting, rack and stack, how do you compare the relationships and other data sources you are using, source or content matrix, trending up or down, comparison of variable to variable, positive, negative, neutral, this could potentially show an interrelationship of variables, weighing by voracity versus trust
  • Indicators: Have a special unique case for different social media methods
  • Hypothesis Generation: A new way of thinking for each dataset, every source will have a different question, something that is measurable, testable and falsifiable, written as a definitive statement, things you have to consider with SM, quantifiable, dependent and independent variable, certain threshold
  • Diagnostic Reasoning: Helps balance people’s natural tendency to interpret new information as consistent with their existing understanding of what is happening – analysts mental model, how can I find evidence that refutes this hypothesis?
  • Deception Detection: Deception Detection Checklists taken from slide deck

Week 7 Deception Detection 1

Week 7 Deception Detection 2

  • Key Assumptions Check: What are people's assumptions about SM, behavior and adoption in society, people's perceptions versus what is reality, talking with people, "card sorting", get people together and vet out ideas, might not be realistic for a specific domain
  • Outside-In Thinking: Take a step back and I've got all this information, understand who is generating the information, what are the biases, coming from certain parts of the population? Who uses this particular method?
  • What If? Analysis: It's a way that helps you generate and measure indicators for each potential scenario
  • Devil’s Advocate: Cooperative peer critique, if a novice prepares a report, bring in expert for review
  • Social Networks and Analytic Teams: This has more to do how to work with each other’s strengths and weaknesses, didn’t focus our discussion on this

At the end of the meeting, we went through Intelligence Community Directive standards and policies the Office of the Director of National Intelligence expects analysts to abide by.  They can be found on the website: www.dni.gov

DNI2

DNI1

We also looked at some other textbooks one of the scientists had brought to see if there are other techniques we should consider.

I also uploaded the large slide deck of 52 techniques and the spliced out 15 techniques he brought with him to the meeting he had in Washington DC. I also uploaded a Brownbag abstract for his colleagues to look at as well.

Week 7 BB1

Week 7 BB2

Friday, May 9, 2014

Spring Directed Field Work Week 6

After the meeting with the research scientists, I compiled an excel spreadsheet of the broad chapter headings and title (Table 1) and the individual techniques that included the chapter/section, title, and level (ranking of the technique) in Table 2.   

Table 1. Chapter and Titles of Structured Analytic Techniques for Intelligence Analysis

Week 6 Table 1

Table 2. All of the Relevancy Rankings of Structured Analytical Techniques by Research Scientists for Social Media Analysisimage

I then filtered the higher level techniques that would be most beneficial with social media data analysis and spliced those out for my mentor to look at. He wanted to make sure there was at least one technique that was of high value in each of the Chapters.

Table 3. Structured Analytical Techniques Ranked of High Value, Chapters, Sections and Title

Week 6 Table 3

Following this discussion, I spliced out of the original full slide deck all of the high ranked techniques based on Table 3, and created a new slide deck that my mentor was going to use at a client visit he was attending in Washington D.C. in two weeks.  We are going to meet with the research scientists next week to associate examples and write notes of when these techniques would be applicable with social media. 

Friday, May 2, 2014

Spring Directed Field Work Week 5

Following last week’s meeting with my mentor, he requested a meeting with three research scientists/specialists who are currently analyzing various social media datasets used in intelligence analysis.  We presented the large slide deck of the 52 different structured analytical techniques and had the scientists flag.  As we went through chapter by chapter, the scientists use sticky notes to flag which ones were of high to low relevancy when associated with the analysis of social media type of data.  In making their decisions some of the issues that were brought up included:

  • Source/type of social media data
  • Would this data be used in conjunction with other information
  • The story or questions needing to be answered
  • Who all would be present in the analysis of the information

After each of the scientists wrote on their sticky notes their rankings, we gathered at a giant table and placed their responses on high, high-med, medium, medium-low, and low on a giant piece of butcher paper.  After all of the sticky notes were placed, we looked at the placement of everyone’s responses.  Because there were three people, everyone needed to be on the same page.  For the most part everyone was in agreement about the placement of responses.  Any responses that were not in agreement each person discussed why they felt it belong in its ranking.  They all weighed in until they were all in agreement.

Here are some images on that day working to categorize the techniques in relevancy.

 

Table1 Highs

Table2 
Medium

Table3

Lows

Friday, April 25, 2014

Spring Directed Field Work Week 4

This week I slammed and got through the other five chapters of structured analytical techniques.  I again highlighted techniques I felt were pertinent to my mentor’s needs of assessing social media data.

Chapter 8. Cause and Effect

  • 8.1 Key Assumptions Check
  • 8.2 Structured Analogies
  • 8.3 Role Playing
  • 8.4 Red Hat Analysis
  • 8.5 Outside-In Thinking
  • 8.6 Policy Outcomes Forecasting Model
  • 8.7 Prediction Markets


Chapter 9. Challenge Analysis

  • 9.1 Premortem Analysis
  • 9.2 Structured Self-Critique
  • 9.3 What If? Analysis
  • 9.4 High Impact/Low Probability Analysis
  • 9.5 Devil’s Advocate
  • 9.6 Red Team Analysis
  • 9.7 Delphi Method


Chapter 10. Conflict Management

  • 10.1 Adversarial Collaboration
  • 10.2 Structured Debate

Chapter 11. Decision Support

  • 11.1 Complexity Manager
  • 11.2 Decision Matrix
  • 11.3 Force Field Analysis
  • 11.4 Pros-Cons-Faults-Fixes
  • 11.5 SWOT Analysis


Chapter 12. Guide to Collaboration

  • 12.1 Social Networks and Analytic Teams
  • 12.2 Dividing the Work
  • 12.3 Common Pitfalls with Small Groups
  • 12.4 Benefiting from Diversity
  • 12.5 Advocacy vs Objective Inquiry
  • 12.6 Leadership and Training

At the end of the week, I sat down with my mentor and went through the extremely large slide deck as quickly as possible and told him techniques I highlighted were meant for the problem they were looking at originally.  He was really happy with my slides but would like me to focus on social media techniques in general that can be used as a streamlined process.  So I will be going through all of the slides again and inputting information on all of the techniques and we will go through each one, one by one next week with a few scientists in the visual analytics group and get their input as well.  So he asked me to set up this meeting and add the additional information for next week.