Security and Intelligence Studies in the Private Sector

By Scott Small

When considering a career in the field of international security, one’s mind is often first drawn to the public sector. Students graduating from academic programs in international security or intelligence studies may first think of pursuing a position with the U.S. or their home government, or working for a relevant branch of an international governmental organization such as the United Nations. However, numerous opportunities exist in the private sector for those possessing the skills developed in a security-focused degree program. Furthermore, contrary to common perceptions about the differing nature of the public vs. private sector, the vast majority of the day-to-day tasks and ultimate goals of an individual working for an international security company are the same as those of a public employee in a comparable position.

I work for a private company that helps other organizations identify and mitigate risk in the international supply chain. The ultimate goal of our company is to enhance our clients’ supply chain security – a term that describes efforts to ensure the integrity of goods from the point of production to the ultimate point of resale and consumption. As a member of my company’s intelligence team, I am tasked with collecting information about the myriad risks to international supply chains, analyzing this information, and then producing and disseminating finished supply chain security intelligence to our clients.

Much like analysts employed at a government agency, our team is responsible for producing a variety of intelligence products in any given week. Intelligence briefs give the consumer a concise overview of a particular incident and the significance of that event in the context of the broader risk environment. Special reports allow us to dig deeper into a particular country or a specific threat. Like many analysts employed in government intelligence organizations, I am responsible for a particular geographic region. While this provides the opportunity to delve into the specific risks present in a particular part of the world, the transnational nature of many supply chain risks also requires close coordination with colleagues focused on other regions of the world.

The overall objectives of an intelligence professional will remain the same whether the individual is working for a government agency or a private company. Most of the day-to-day tasks of an intelligence analyst are comparable between private and public employment. All analysts will follow some form of the intelligence cycle: first collecting raw information; processing and analyzing this information; disseminating a finished intelligence product; and constantly evaluating the entire process for areas of improvement or optimization.

Furthermore, the key tenets and ultimate goals of an intelligence professional should remain consistent regardless of the employer. Most importantly, objectivity is fundamental in order to provide the most accurate assessment of any issue to the intelligence consumer, whether they are a policymaker in Congress or the White House or a corporate executive. Critical thinking skills are essential in order to synthesize seemingly disparate pieces of information into a more complete picture. Finally, but certainly not least important, conciseness and solid writing skills are necessary to clearly convey your message to individuals who often have only minutes to read and process a finished intelligence product.

Although it may not often be as apparent, working for a private company in the field of international security offers a directly comparable experience to many similar positions in the public sector. The day-to-day tasks of intelligence officers are overwhelmingly similar regardless of the employer, and the end- goals and skill sets required to accomplish these goals are identical. My own experience with a private-sector company in the field of supply chain security has afforded the opportunity to further develop the skills I was first introduced to in the classroom, in addition to providing the chance to have a direct impact with my work.


What ‘Moneyball’ can teach us about Intelligence Analysis

In calling for the improvement of intelligence analysis, other professions have been invoked, such as medicine and law.  But, what about that most quintessentially American of all sports: baseball? In the story of Moneyball we find a modern day David versus Goliath story which has been used challenge thinking in diverse fields ranging from business to medicine, and even corrections. As with these other fields, the lessons of Moneyball can transform intelligence analysis.

Moneyball is the story of how in 2002 the cash-strapped Oakland A’s management revolutionized baseball. In the film version of the story, A’s general manager Billy Beane, played by Brad Pitt, sums up the difficult position of his club: “There are rich teams, and there are poor teams. Then there’s 50 feet of crap. And then there’s us.” And Beane wasn’t exaggerating; the A’s 30 million dollar budget was a mere fraction the powerhouse clubs, such as the New York Yankees (c. 130 million).

2003 Major League Budgets by Team

Graph Source:

steve blog 3 pic 1

Combining an unorthodox perspective on recruitment and traditional baseball statistics, the A’s front office imported sophisticated statistical analysis to build a highly competitive team with a shoe-string budget. With their revolutionary approach the A’s won 20 consecutive games, an American League record, and to this day are among the most efficient teams, purchasing a win in the 2013 season at $631,000 versus big payroll clubs such as the Yankees purchasing each win for $2.6 million.

But, if you dig deeper the real message of Moneyball isn’t about statistics, or even money, for that matter: it is really about improving a profession by challenging orthodoxy with novel ideas.

Experience is useful, but it’s no crystal ball

Consider the role of experience in baseball recruiting. Traditionally, baseball recruiting is a subjective art practiced by experienced scouts. According to established baseball thinking:

                  you found a big league ballplayer by driving sixty thousand miles, staying in a                       hundred crappy motels, and eating god knows how many meals at Denny’s all                     so you could watch 200 high school and college baseball games inside of four                     months….Most of your worth derived from your membership in the fraternity of                   old scouts who did this for a living… (Lewis p. 37)

After staying in a ‘hundred crappy hotels’ and eating all those Denny’s meals, the scouts relied on their experienced to select their top picks on the basis of the appearance and anecdotal information they know about the player, an error that led some players to be vastly overvalued and others undervalued.  Players that didn’t “look” the part of a major league player or who didn’t have the right back story were consistently passed up, something that the A’s capitalized on to buy the best team for their buck. Take for example, Chad Bradford, and his unusual submarine pitch.  While Bradford ended up a staple relief pitcher for the A’s, he was so overlooked by scouts that he began his pitching career at a community college.

Chad Bradford delivering his unique submarine pitch

steve blog 3 pic 2

But, Bradford’s case could be a ‘black swan’ and you could even reason that it is OK to miss the occasional submarine pitcher. However, the problem isn’t just about ‘unique’ ballplayers. Consider the common scouting task of selecting the superior hitter. Look at the picture below of Pittsburgh Pirates slugger, Neil Walker. Can you guess what 2013 his batting average is?

Guess Neil Walker’s Batting Average

steve blog 3 pic 3

If you know baseball you might come close because you know roughly how good Walker is, or other standout hitters. However, if you never watch baseball, I am willing to bet you’d be wildly off (but for those of you keeping score at home, Neil Walker batted .280). Now, if you were a scout, you might come a little closer to being right than someone who watches baseball, but you probably wouldn’t guess much differently.

A meta-analysisof clinical judgment in medicine suggests that experts (e.g baseball scouts) slightly outperform well-informed observers (e.g. baseball fans).  However, as complexity increases in the judgment task and the opportunities for chances to learn goes down—as is the case with most intelligence tasks— experts will not do much better than well-informed observers.  In foreign affairs forecasting, Tetlock found a similar dynamic: there was a rapid and diminishing return on expertise.

The Diminishing Returns of Expertise

steve blog 3 pic 4

In one study I asked participants to make judgments about the extent to which the Assad regime will comply with a UN resolution to remove and destroy all declared weapons before June 30th, 2014. The participants of the study included 24 graduate students in a security studies program, representing well-informed observers, and 16 International Association for Intelligence Education members (IAFIE), and 5 analysts from the IC, representing experts, for a total of 45 participants.

This modest experiment seems to reflect the diminishing returns of experience argument: the grad students guessed 62% versus the 51% for IAFIE and IC analysts. Whether one group has guessed a figure closer to actual number of weapons destroyed is a more complex matter for another discussion, but it would seem that there is not at least a large difference between the two groups, despite the presumably large experience gap.


Estimates of Percentage of Weapons Destroyed & Removed


Since such estimative judgments are but one part of the analyst’s job, I also asked each participant for their rationale. Again, like the estimations, the rationales were similar but not the same. Below are frequency distributions for each group’s cited reason for their judgments. Both groups identified reluctance of the Assad regime, interference of the civil war, and difficult timeline as their main reasons for their judgments, although the two groups prioritized the first two justifications differently. Where I found any substantial difference was that IAFIE and analyst groups noted the importance of the weapons falling into a 3rd parties hands (e.g. a rebel group).

Graduate Student Justifications

steve blog 3 pic 6

IAFIE and Analyst Justifications

steve blog 3 pic 5

The point of this lesson and the findings of the study is not that experience doesn’t matter, but that it can only take us so far as the difference between well-informed observers and experts is not large. In lieu of this issue we need to keep developing new methodologies and techniques that can supplement—not supplant—the experience of analysts. However, if analytic methodologies and techniques are to be created we would learn well to heed the next Moneyball lesson.

Don’t assume the established way of thinking or doing something is right

Realizing the limitations of the scouts, the A’s turned to the analysis of baseball statistics, sabermetrics, but they didn’t just assume the numbers would save them. In fact, the A’s scrutinized many of the standard baseball statistics and found some were just as biased as traditional scouting.

Take for example, the fielding error, which occurs when a fielder misplays a ball such that a runner from the opposing team can advance.  The fielding error statistic was thought up in the early years of baseball as a way to account for how barehanded players fielded (baseball gloves weren’t common till the 1890s).  A century later baseball statisticians began noticing the fielding error statistic was misleading because an error could only occur if a player made an attempt to pursue the ball in the first place, therefore punishing those who made the attempt to get the ball, and rewarding those who either avoided or couldn’t get to the ball in time.

In short, fielding error statistics “weren’t just inadequate; they lied ” (Lewis, p. 67)

The result of the misleading error statistic is that many players were passed up, and teams relying on error as a measure, were mismanaging how they appraised their defense.

The Fielding Error made more sense in the time of rough fields & bare hands

blog 3 pic 7

In intelligence analysis similar mistakes can be made. For example, consider the use social network analysis (SNA), the analysis of links between people, often used in intelligence to study terrorist and criminal groups. With the increase of social network analysis tools and ‘big data’ analysts increasingly rely on SNA and associated statistics such as, degree centrality, a simple measure of how well-connected a person is in a network. However, reliance on this statistic as a measure of influence can be as troublesome for determining influence in the network, as an error statistic is for determining fielding skill.

Consider the case presented by Bienenstock and Salwen (forthcoming) of Abu Zubaida. While Zubaida was identified as a number 3 in al Qaeda by U.S. leadership, he was later found to be a low-level operative. Yet, he was heavily connected in the network because his role as a courier, and therefore would have had a high degree centrality score.  Below is a sociogram from Marc Sageman’s well-known global violent Salafist data linking al Qaeda to other violent Salafist plots with Zubaida in the center of the graph near bin Laden.

Al Qaeda-Madrid-Singapore Network (source: Sageman et al)

pic 8

I witnessed firsthand how network statistics can be misleading while working with Mike Kenney on a project using SNA to map al-Muhajiroun, an Islamist extremist group in the UK.  To create our network we used a massive dataset of news reports and automated data extraction tools similar to what IC uses, but what we did that was novel was we cross-validated our network statistics with in depth field research. Over the course of two years, Mike visited the UK several times interviewing 86 people within al-Muhajiroun, from the top leaders down to the rank-and-file members.

When we attempted to cross-validate our networks against hours of interview recordings we found something pretty surprising: the standard network statistics by themselves were incredibly misleading.  For example, in our SNA we found individuals that had no operational ties to the network but were ranked artificially high in the network, such as Osama Bin Laden (yellow). Others had were ranked high because they were well-known in the British media, such as Salahuddin Amin (green), but certainly weren’t ranking leaders of the network as the SNA would imply.

Betweenness Centrality in al-Muhajiroun

pic 9

The moral of the fielding error, Zubaida, and  al-Muhajiroun stories is not that these statistics have no value, but that it is necessary to make a conscious effort to determine if a particular way of thinking or method actually works.  While it might be difficult to evaluate some of these practices it is entirely possible with additional effort.

Looking to the outside for innovation

If the Moneyball revolution had an ideological father it would certainly be Bill James. It was his mistrust of experienced judgment and concern with the traditional baseball statistics, such as the fielding error, that led the A’s management to storm the Bastille of professional baseball. Yet, James was an ‘outsider’ in the purest sense of the word—he penned his first tract on baseball analysis while working as a night watchman at a bean factory.

Beginning with his self-published books in the 1970s, his popularity grew a cult-following among computer and stats nerds, but even as his circle grew larger and larger, his message landed on deaf ears of the men who ran professional baseball.  This all changed once the A’s front office brought James’ ideas into practice, drawing the ire of the traditional ‘baseball men.’ Still, even with strong resistance, the A’s were able to inaugurate the Moneyball revolution with a combination of outsider innovation and insider know-how.

Bill James on the cover of a 1981 issue of Sports Illustrated 

pic 10

In intelligence analysis there is a similar dynamic underway as many post 9/11 programs have opened up numerous channels for new ideas, such as the Intelligence Advanced Research Projects Activity’s (IARPA) grants and the Centers of Academic Excellence program. In short, it would seem inroads are being made to bring in more outsiders.

Still, there are still few places and opportunities for those interested in implementing the core message of Moneyball in intelligence analysis. For my own part I am trying to validate some of the structured analytic techniques promoted after 9/11, but I’ve faced an endless set of institutional and cultural barriers. This stems from the fact that as a young intelligence studies researcher, I am caught between a rock and hard place; my research subject is unfamiliar to an academic audience and some practitioners are distrustful of applied social science research.

It would seem we need not just an institutional shift but also a cultural shift to bring in new ideas. In the beginning, there will certainly always be resistance, but if the objective is to improve the profession, considering the limitations of experience and questioning ‘what works’—the lessons of Moneyball—can take us a long way.

[Bracketing] the Black Swan (Part II)

In my last blog post I discussed the possibility of being able to bracket, or identify, a ‘black swan,’ an extremely rare event which has significant consequences. Trying to identify a black swan event is a pretty tall order since these events by definition, are highly unlikely. As I discussed in my blog entry last month, the challenge is to ‘reach out’ on to the statistical distribution towards the unlikely hypotheses.

Research on knowledge systems  suggests that the most commonly identified hypotheses among a group of experts are on the extreme left of the distribution. In most analytic tasks, the most instrumental hypothesis is probably here. For example, there are a few commonly discussed hypothesesfor the outcome of the Syrian Civil War (e.g. Assad regime wins, stalemate, etc.). In the graph below these hypotheses would fall in the green shaded region as H1, H2, and H3. But, in the case of black swan events, the hypothesis (or hypotheses) are less frequently suggested and are further out on the right. In the Syrian example, this might include Iran invading and achieving victory out in the yellow shaded region.

Steve Blog 2 Pic 1


Imaginative structured analytic techniques assist analysts in reaching out further on this distribution, but , some of the techniques have notable  limitations. For example, one such technique, brainstorming, assumes equal participation among diverse group members, which defies conventional experience. Further, most of these techniques cannot tell the analyst where they are on the distribution, and more importantly, when they have reached saturation and generated the bulk of plausible hypotheses. In a traditional brainstorming session, this is usually identified by a lull in the conversation and participants are satisfied they have captured the likely hypotheses.

Boundary analysis, developed by William N. Dunn, is another way to generate hypotheses. The technique requires analysts to sample documents containing hypotheses (e.g. news reports) and write down each hypothesis. As an analyst records more hypotheses he should observe the effect of Bradford’s Law: after a point the number of new hypotheses gathered from each document drops precipitously. Since the hypotheses come from the documents rather than the group itself, the technique may ameliorate some of the negative effects of group dynamics on hypothesis generation. Furthermore, one can simply expand the scope of the search for more documents to gain access to rarely cited hypotheses.

Stopping Point of Bradford’s Law

Steve Blog 2 Pic 2

For most analytic tasks, stopping at the “knee of the curve” (where the marginal frequency of each new hypothesis levels off) will likely include the correct hypothesis. But for “black swan” events, we have no such defined rule. By definition it would seem that a black swan should fall after the stopping rule, but it is also entirely possible that the black swan really was foreseeable.

We simply don’t know.

To address this question I teamed up with my colleague Jay Rickabaugh to apply boundary analysis retrospectively to a ‘real world’ intelligence analysis task: the 2012 University of Pittsburgh bomb threats.

The Pitt Bomb Threats

Over the course of ten weeks in the spring of 2012, the University of Pittsburgh received approximately 140 bomb threats. While the threats took a variety of forms, beginning with scrawled threats in campus restrooms, the most persistent and numerous threats came from emails sent through a remailer, which masked the location of the perpetrator. Further, confounding the investigation were copycat actions, false accusations and others seeking publicity by capitalizing on the chaos.  The swarming of these threats made this case different from a traditional bomb scare and thus the possibility of black swan explanations seems more possible.

During the multi-agency investigation, several leads were pursued but each led to a dead-end. Finally on April 19th, after weeks of threats causing the University of Pittsburgh to spend more than $300,000 in direct costs alone, the University met the demand of one of the threateners to rescind a $50,000 reward, and immediately thereafter, the emailed threats stopped.

In mid-August, after a months-long investigation, authorities held a press conference to announce that they were charging Adam Busby, a 64-year-old Scottish nationalist involved with the Scottish National Liberation Army (SNLA) in connection with the emailed threats. The result was stunning and best summed up by Andrew Fournaridis, administrator of a blog developed during the bomb threats who wrote:

“This is the mind-bending stuff intelligence analysts must deal with on a daily basis, especially in the 21st century cyber-crime era.”

To this day authorities have never divulged Busby’s motivation.

The question is: will boundary analysis find the black swan before the stopping rule?

Using Boundary Analysis & Findings

For our analysis we used open source documents from two local newspapers (the Pittsburgh Post-Gazette and Pittsburgh Tribune-Review) and blog postings from, a major platform for crowd-sourcing during the threats. After compiling all the sources we had more 130 news articles and numerous blog posts ranging from January 1, 2012 to August 30, 2012.

Articles that did not contain useful information (e.g. articles about how students coped with threats) were omitted, leaving us with 73 articles that we coded by date in an Excel spreadsheet.  Next, each article was scrutinized for hypotheses, a process that took a single coder approximately 8-10 hours.

Our boundary analysis of the bomb threats yields two findings:

  • Boundary analysis identified the ‘usual suspects’ quickly

In conducting our retrospective boundary analysis we quickly found our stopping rule. In fact, within in a time span of roughly one month, from March to April, almost all of our hypotheses were identified in our documents (see graph). These original hypotheses included typical explanations such as students avoiding exams, students who have conflicts with university administration, pranksters, etc.

Steve Blog 2 Pic 3

The ability of boundary analysis to locate the main hypotheses quickly may also be helpful when combined with hypothesis testing techniques. For example, once the analyst extracts the most common hypotheses he can begin testing each one using a diagnostic technique (for example, alternative competing hypotheses) and move further out on the distribution as needed.

  • The normal stopping rule did not bracket the black swan hypothesis

After an examination of our three data sources, the correct hypothesis—a foreign national from the UK pranking the University—was not identified in the documents. However, we stopped our analysis at the stopping rule, or “knee of the curve.” We do not have enough information to suggest what a good limit to set would be, but applying these same principles to more black swan intelligence cases (the DC Sniper, Eric Rudolph, etc.) would give us a better indication. With more research, we can begin to identify how far past the knee one would need to research to be reasonably confident the black swans are identified. Thus, when unanticipated or abnormal events begin to occur, we do not use ordinary methods for unique circumstances.


While we were unable to bracket the black swan using traditional limits, the two findings have important implications for intelligence analysis. Probably the greatest benefit of boundary analysis could be to give analysts a list of ‘usual suspects’ hypotheses. Analysts can then use diagnostic techniques to whittle down the number of plausible hypotheses. If these usual hypotheses are not useful, the analyst can keep moving to the right of the distribution by extending the boundary analysis or employ an imaginative technique. As we note, an area of future research is conducting more research retrospectively to determine if there is a stopping rule that will catch most black swans.

Crossing the Rubicon: The Dangers of Martial Law

“…the ubiquitous M16 assault rifle, a humble enough weapon until you see it in the hands of a man outside your local bowling alley or 7-11. It will be noisy, it will be scary and it will not be ignored for a VFW parade” – General Deveraux (the Siege) in reference to declaring martial law in New York City.

The recent imposition of martial law in Thailand, Egypt and parts of Ukraine highlights the dangers and benefits of imposing martial law. The term “martial law” has an ominous ring to it, especially in those countries founded upon notions of individual rights and privileges. Martial law is defined not as law in the ordinary sense, but rather, the exercise of military authority, coupled with the temporary suspension of some or all of the functions of civil government. Normally, martial law is only instated in times of war; but during times of peace, when faced with domestic upheaval or natural disaster martial law is occasionally required to impose order. Often times, military rule is essential to re-impose order and defeat factions challenging state authority. However, martial law also has been utilized to gain political power or to repress legitimate opposition.  Admittedly, much like Julius Caesar crossing the Rubicon in 49 BC, even with the best intentions, imposing martial law has severe ramifications.

Consequently, although martial law, is necessary tool for any regime, it should be utilized sparingly, as it can lead to many unforeseen consequences. These unforeseen consequences might include: decay in public opinion concerning the ruling regime and the military, increased possibility of military coups, infringement on individual rights, and continued utilization of the military in politics by various actors. Members of armed forces should also hesitate to become involved in instating martial law, since these consequences may be significant and long reaching, largely because martial law legitimizes the use of force. Therefore, if policy pitfalls are to be avoided, a careful examination of various instances of martial law should be done.

selle blog 2 fig 1 selle blog 2 fig 2
Figure 1 & 2: Canadian Troops in Quebec, on street searches and guarding infrastructure

The “Good” – October Crisis 1970.

In the fall of 1970, a radical Quebec separatist group, the Front de Liberation du Quebec (FLQ) kidnapped the British trade commissioner James Richard Cross from his Montreal residence. The FLQ had been involved in over 200 bombings in Quebec since 1963, with an agenda for achieving Quebec independence by any means. At first the government response was limited, with very public negotiations between the regime and the FLQ. However, the kidnapping and subsequent strangling of Quebec Minister of Labor, Pierre Laporte by a cell of the FLQ dramatically changed the nature of the crisis. Panic rippled through the public, and gave the impression that the regime was losing power. The young Premiere of Quebec, Pierre Trudeau, a lifelong champion of individual rights, turned to the national government for help. The national government responded by invoking the War Measures Act, and instituting martial law in the greater Montreal area. Troops were utilized to guard vulnerable points and important individuals as well as to support the role of civil authorities such as the police in their investigative efforts. The Canadian army and police conducted 3000 searches and detained 497 individuals within the first 24 hours. While Habeas Corpus (the individual’s right that a judge confirms they have been legally detained) was suspended, and some accused were held for 21 days without legal counsel, the cell members responsible for Laporte’s death were arrested and charged with first-degree murder.

While the regime faced criticism for the suspension of civil liberties, there was overwhelming contemporary support for the invocation of martial law, and the use of military forces. Premiere Trudeau, in an interview expressed his choice to impose martial law:

“I think the society must take every means at its disposal to defend itself against the emergence of a parallel power which defies the elected power in this country and I think that goes to any distance. So long as there is a power in here which is challenging the elected representative of the people I think that power must be stopped.”

Assuredly, any suspension of civil liberties and the imposition of military rule on a civil population should be a last resort, but as the Canadian example demonstrates, having strict controls and overall civil authority with the military playing a subordinate role in martial law may lead to less misuse of the military and martial law.

selle blog 2 fig 3 selle blog 2 fig 4
Figures 3 & 4: Thai troops and Egyptian tanks imposing martial law.

The “Bad” – Thailand & Egypt

In sharp contrast, as recent events in Thailand and Egypt demonstrate, having the military play the role of final arbitrator on political issues, in conjunction with successive use of military force in suppressing political dissent and opposition will lead to more overt military intervention in political affairs. The fact that the Thai military plays a dominant role in the politics of Thailand is well known to the world. The coup of 1932 marked the point when the Thai military crossed into the political sector from which has never quite retreated. Additionally, in Egypt, the military and martial law have become tools for the consolidation and maintenance of political power. Consequently, it has become very difficult for citizens to either distinguish between the military just reasserting order or obtaining control over the reins of political power by establishing martial law.

The “Ugly” – New Orleans, 2005 & parts of Ukraine, 2014.

In 2005, Hurricane Katrina led to a breakdown of effective governance in New Orleans, Louisiana. While the term martial law does not exist in Louisiana’s constitution, a form of emergency management akin to martial law was used to address the widespread violence and disorder. The primary forces tasked to restore order were the New Orleans police department and the Louisiana National Guard (LANG), with federal troops primarily deploying medical, search-and-rescue as well as other forms of relief. Those forces instating martial law faced significant opposition from citizens who were attempting to protect their private property through personal firearms. However, considering the rising rates of violence in the face of relief efforts, military and law enforcement forces had little choice but to infringe on the rights of individuals. The crisis highlighted the challenge of ensuring public safety and effective governance at the expense of the rights of individual bringing the issue into contemporary American political discourse.

In a similar way, the government of Ukraine faced a similar dilemma when instating martial law. In February 2014, as the pro-western movement gathered momentum, President Yanukovych granted increased powers akin to martial law to the army and police. However, the increased repression only emboldened the pro-western groups eventually leading to the ousting of the President. A few months later the temporary pro-western regime now faces a similar dilemma in combating the rising pro-Russian separatists in the eastern parts of the Ukraine. By declaring martial law, the pro-western regime in Kiev may alienate supporters or even embolden opponents as Yanukovych did. Nevertheless, with rising violence in certain regions, such as Luhansk Oblast, the regime might to impose instated martial law, or face a public backlash against ineffective governance.

The Lessons

In times of peace under martial law, a soldier, for the time being, becomes a glorified policeman, with powers greater than those of an ordinary peace officer, but ideally is still under civilian control. The soldier is tasked to restore order in the aid of the civil authority, which for the moment has been submerged either by natural forces or actors attempting overthrow the existing regime. A government establishing martial law should assure the public that such a step is necessary and temporary and that civil control over the military remains in effect. The military forces carrying out martial law when it is necessary, should behave with restraint and seek to restore full civilian control as quickly as possible.


If you have any questions or comments please contact Werner at




New Ridgway Fellows Announced for Fall 2014 and Spring 2015!

GSPIA’s Matthew B. Ridgway Center for International Security Studies is pleased to announce that Jeffrey Carasiti and Christine Unger have been selected as recipients of the Ridgway Fellowship for the fall 2014 semester and Darren Hedland and Diana Antonian Israelian for the spring 2015 semester.  These scholarships are awarded by the Ridgway Scholarship Selection Committee and are provided by the University Center for International Studies (UCIS).  Congratulations to these students!

Next spring the Ridgway Center will be announcing the fellowship competition for the fall 2015 and spring 2016 semesters. Please watch for the email announcement.  You will need to submit a cover letter outlining your involvement with the Ridgway Center and a resume. We encourage all full-time SIS graduate students to apply.

[Bracketing] the Black Swan in Intelligence Analysis (Part I)

This article is part of a recurring series by Steve Coulthart, a PhD candidate at the Graduate School for Public and International Affairs at the University of Pittsburgh.  If you have any questions or comments, feel free to contact him at

Intelligence analysts have a ‘black swan problem,’ or, if you want to be more academic you might call it the ‘problem of induction.’ The problem of induction touches on the conundrum of how we can know the future given the experience we have today.  Or, in other words, at what point can we say we know what we know?

An example from Taleb’s well-known book on black swans helps to clarify. Take the turkey’s dilemma: for the first 1,000 days of its life the turkey is fed and treated well, thus each day increasing its expectation that the following day will be the same. Yet, at day 1,001 that assumption is proven false and the turkey ends up for Thanksgiving dinner.

steve blog 1 fig 1

Substitute the turkey and insert an analyst trying to forecast the next revolution, coup, or terrorist attack and the task comes into focus. To avoid surprise, analysts attempt to foresee different possible outcomes. We can think of these different possible outcomes as hypotheses about things that could happen in the future. For example, in the case of outcomes for the Syrian Civil War, there are several possible hypotheses floating around: the Assad government wins, the stalemate lingers, rebels win, etc.

The first step for analysts working on a forecasting task is to conjure up hypotheses from their own experience, intelligence reports, experts, etc.  Most likely, the analyst will identify a few well-known hypotheses (such as the ones mentioned above). We know this because how well known a hypothesis is, measured in the form of how often it is cited in discussion (e.g. in news articles, amongst experts, etc.), fits a power law distribution. The practical result of this is that there is a set of core hypotheses almost everyone knows and a long tail of lesser known ones (this is due to Zipf’s Law, check out the link for more information).

In his pioneering research in the policy analysis field, William Dunn found that the most cited or discussed hypotheses are on the extreme left of the distribution while the black swans are on the extreme right. In intelligence analysis, these are the hypotheses that are often ignored until it is too late (e.g. the 9/11 attack). In our Syria example this could include something seemingly unlikely like an Iranian invasion of Syria.

steve blog 1 fig 2

What can analysts do to ‘reach out’ on to the tail? The common answer is to encourage analysts to think creatively and/or consider the complexity of the situation. To do this, analysts are trained in ‘imaginative’ structured analytic techniques that supposedly open their minds.   The U.S. Intelligence Community’s tradecraft primer lists a few of these techniques and Heuer and Pherson’s standard texthas several hypothesis generation techniques. Unfortunately, these techniques have a crucial weakness: there is no stopping rule.

What is a stopping rule? Well, like the turkey in the example above, the analyst doesn’t know when he or she can stop considering new hypotheses, including a potential black swan waiting in the wings (no pun intended).

Consider a hypothetical group of analysts brainstorming the outcomes of the Syrian Civil War. At what point should the analysts stop generating hypotheses?  Perhaps they have identified our black swan of Iran invading, but what now? Are they done? The common answer is to say when it “feels right,” but as we know, cognitive biases can creep in, and further, what if the black swan is still lurking out on the tail?

One possible answer, yet to be discussed in the intelligence analysis literature, is the use of boundary analysis developed by Dunn. As the name implies, boundary analysis is a method to determine the analytic ‘boundaries’ of a problem, in this case the number of plausible hypotheses. The technique also addresses the stopping rule problem plaguing imaginative structured analytic techniques.

Here’s how it works:

The first step in boundary analysis is the specification of the analytic problem. For example, “what are the likely outcomes of the Syrian Civil War?” Next, analysts sample data sources that hold hypotheses related to the analytic question. A common source of hypotheses can be found in open source documents, such as news reports. Once the data is compiled, it can be mined by coding each unique hypothesis.

At first the list of hypotheses will grow exponentially with each document, however, the analyst will soon see something very puzzling: after the initial rapid increase of new hypotheses, each new successive document will yield less, and less new hypotheses.  This rapid leveling-off is due to Bradford’s Law

steve blog 1 fig 3

An Example of Bradford’s Law: Citations

In 1934, British mathematician Samuel Bradford was searching physics journals and found that after locating approximately two dozen core journals he had found the bulk of all physics academic citations. After these core journals each subsequent journal provided a diminishing amount of new citations. The leveling-off effect of the Bradford Law also applies to hypotheses and provides a stopping point at which analysts know they have reviewed almost all known hypotheses.

Returning to our power-law distribution of hypotheses, we could imagine that a boundary analysis might get us closer to finding the black swan, but boundary analysis is still no panacea because at this point we really do not know how well the technique does in identifying possible black swans in intelligence analysis tasks.

steve blog 1 fig 4

Fortunately, the question of how boundary analysis performs on intelligence analysis tasks is an answerable empirical question. In my next blog post I will present results from a research study using boundary analysis on a ‘real world’ intelligence analysis problem.

Russia and the Ukraine: Re-Emergence of the Fifth Column

This article is part of a recurring series by Werner Selle, a Masters of Public and International Affairs candidate at the Graduate School for Public and International Affairs at the University of Pittsburgh.  If you have any questions or comments, feel free to contact him at


The Russian annexation of Crimea and Russian efforts to destabilize Ukraine, have some political pundits and scholars claiming that Russian President Vladimir Putin has discovered a novel new form of warfare and diplomacy, aptly named the “Putin Tactic.” In February 2014, utilizing unmarked armed forces with support from local pro-Russian groups and other paramilitaries, Russia annexed Crimea. However, this tactic is neither novel nor unique to the Putin administration. Instead the term Fifth Column, credited to Emilio Mola y Vidal, a Nationalist General during the Spanish Civil War (1936-1939) is more appropriate. General Mola told journalists in 1936 that a fifth column of supporters inside Madrid would support and undermine the Republican government from within as his other four columns approached the city. A fifth column is therefore traditionally defined as the infiltration of government-backed subversives or sympathizers into the entire fabric of a nation, spreading disinformation and rumors as well as employing force undermining the effective rule of law and the strength of the state under attack. Yet neither this nor Putin’s approach is new. It is important to consider, therefore, what lessons can be derived from both the previous and current use of fifth columns.

The Fifth Column strategy was utilized even in antiquity. In 431 BC, on a rainy spring night, some three hundred Thebans entered the Boetian Town of Platea, an ally of Athens. Through support of local pro-Theban Plateans, the gates of the city were opened and the Thebans were admitted. The Athenian historian Thucydides comments that this was commonplace for the next twenty seven years of the Peloponnesian War, with both oligarchic and democratic fifth columns inviting through subterfuge, either Athenian or Peloponnesian forces into their cities.       In American history, fifth columns were active in both the Revolutionary War and the American Civil War. An informative book by George Fort Milton, describes in detail not only the composition and tactics of the Confederate fifth columnists, but also the difficulties facing President Lincoln in combating fifth column elements.

During World War Two Fifth Columns were not only feared but became de facto scapegoats for military defeats. One headline even proclaimed that “German tourists with machine guns seized Luxembourg before German regular forces arrived.” Admittedly, these fifth columns did play significant roles in certain German successes.

selle blog 1 fig 1 selle blog 1 fig 2              Figure 1: “Russian” Troops in Crimea        Figure 2: “German” Troops in Sudetenland

The German seizure of the Sudetenland in Czechoslovakia in 1938 is similar to the Russian seizure of parts of Ukraine in 2014. Both Czechoslovakia and Ukraine had/have minority populations bordering an avaricious major power neighbor purporting to be a minority’s protector or national homeland. Both Hitler and Putin claimed they were merely protecting their ethnic countrymen. Furthermore, both the Third Reich, and Putin’s Russia not only expressed public support for local movements, but gave overt and covert support for these movements. Nevertheless, the tactical benefits of a fifth column are myriad. They undermine the efforts of an opponent prior to any hostilities or before actions commence. In the case of the Sudetenland, German Freikorps, armed self-defense forces would have undermined any efforts by the Czech government to resist. Similarly, in Crimea, well-armed pro-Russian groups significantly reduced the possibility of pro-Ukrainian forces from resisting the Russian takeover.

The infiltration of sponsored infiltrators or sympathizers into the entire fabric of a nation, spreading disinformation, rumors as well as employing force undermines the effective rule of law and the strength of the state under attack. From a military perspective fifth columns are able to subvert the enemy’s defensive strategies, harass or cripple supply and communication lines. Yet, most significantly utilizing a fifth column, forces the enemy to confront or combat its own populace which can be not only detrimental to morale, but also might spur others to join the fifth column or alienate the military force. Furthermore, the utilization of a fifth column reduces the international exposure of utilizing one’s own forces, while leaving the opponent with possibly embarrassing incidents of use of force against civilians.

Unfortunately, a Fifth Column strategy may have limited success depending on the strength of the fifth column and the extent of outside support. In the case of World War Two, the fascists groups in Britain, such as the British Union of Fascists led by Oswald Mosley, may have been significant contributors to German success during the Battle of Britain, but without German support and limited in size, it was quickly disbanded and its leaders interned. In sharp contrast, in Norway, Vidkun Quisling was able to seize power in a coup d’état with German support, and remain a German collaborator until the end of the war.

In conclusion, the “Putin Tactic” is neither novel nor unprecedented. This is not surprising; let us not forget that history often repeats itself, and that successful strategies no matter what guise they are in, are often reintroduced.

If you have any questions or comments please contact Werner at

Islamic Terrorism and Online Video Games: Virtual Worlds, Real Threats

This article is part of a recurring series by Alex Halman, a PhD student at the Graduate School for Public and International Affairs at the University of Pittsburgh.  If you have any questions or comments, feel free to contact him at

When Americans think terrorism, their minds shift to caves, bearded men, and high-tech ignorance.  Our conceptions of Jihadi organizations are that they abhor technological innovation; however, there is evidence that these groups are embracing new options. The idea that terrorists use Call of Duty to train and plot attacks is equally both laughable and frightening.  Online gaming is at an all-time high.  With such a massive and untapped resource at their disposal, terrorists may use these virtual environments for communication, funding, and other illicit activities.   Security analysts and academics have barely scratched the surface of this capability.  Recently, Edward Snowden released a bevy of classified documents; one was an NSA report from 2007 that addressed concerns about violent non-state actors utilizing Games and Virtual Environments (GVE).  Two years later, the Intelligence Advanced Research Projects Activity (IARPA) revealed Project Reynard, a venture aimed at supporting research on and in Virtual Worlds.  When the project was funded, there was little academic literature on the subject and limited evidence demonstrating terrorists were using this technology.  If the extent of future video game use by violent non-state actors is uncertain, it is also potentially, quite significant.  It is important, therefore, to consider the options available to these groups.  Bounding the problem in the ways described here could be useful for both intelligence agencies and the DOD.


There is evidence that points to a continued real world focus for violent non-state actors.  First, terrorist organizations, and more broadly criminal syndicates, often rely on real world funding, communication, and recruitment mechanisms.  In other words, they don’t need to fix what is not broken.  Despite technological advances, drug trafficking, jewelry trades, and traditional money laundering schemes have worked for thousands of years.  Using GVEs, even though they might be more efficient, leaves footprints for our agencies to follow.  Thus, it is possible that GVE utilization would be detrimental for Jihadist terrorist organizations – and will be avoided rather than embraced by them.

In this case, budgeting and policy direction would remain unchanged in regards to U.S. DOD data-mining capabilities.  There are several advantages to this option.  First, this is the most risk averse of the policy options.  If you are unconvinced that online game usage by enemies of the state is a serious threat, no further action is required.  Project Reynard ended in 2009 with little evidence of such activities.  The political risks are also quite low with this option.  When Snowden released the documents in 2013, there was significant public uproar.  The program was trivialized and an embarrassment for the NSA and DOD.  Furthermore, some researchers from the National Research Council argue that data-mining is futile.  Ultimately, this is the option with few potential benefits as the risks are rather low.


The second option represents an incremental change from the first.  In this situation, CYBERCOM and/or the NSA would launch a development similar to Project Reynard.  Project Reynard was a crowdsourcing project.  This, on the other hand, will be an in-house job.

There is some support for such an initiative.  Jihadi organizations are designing video games, and this demonstrates their willingness to adapt and incorporate new technologies into strategy.  Games like “Under Ash” and the sequel “Under Siege” can be potent recruitment and communication tools for these groups.  In these games, a Palestinian protagonist seeks revenge on Zionists and their supporters for occupying his homeland.  In a sense, this is the jihadi version of “America’s Army”, a game developed by the U.S. Army.  There are several more games that have been developed to draw in and radicalize Arab youth.  Because these games are so overt, it is unlikely developing Jihadi games will be a primary strategy.  Nevertheless, these games are evidence that terrorist organizations are shifting to virtual worlds.

Another example of Islamic fundamentalists expressing a willingness to adopt advanced technology, and possibly GVEs, is the Syrian Electronic Army (SEA).  The SEA, although supporting the Syrian government, is one of the first public and virtual armies in the Arab world to strike cyber targets.  This online activism by the youth in the Arab world possibly demonstrates a paradigm shift in recruiting and communication for Jihadi organizations.  Globalization and power diffusion have empowered small and previously insignificant actors, enabling them to pose a serious threat to states.  Game technologies could do the same.

Further evidence supporting Jihadi terrorists’ potential use of GVEs is the funding opportunities through online game markets.  Games like World of Warcraft have autonomous markets that are targeted and penetrated by illicit actors. Although illegal, the market is massive and extremely lucrative; estimated at several billion dollars. Jihadist groups may use virtual markets as they are resilient and similar to other illicit markets.  Keegan et al (2010), using social network analysis, finds that gold markets (GVE currency markets) are structurally similar to drug trafficking networks.  Moreover, the state markets have similar characteristics like secrecy, resilience, and efficiency that are displayed in real world black markets.  This suggests that terrorism funding through GVEs might be a serious issue in the future.  A covert policy and budgetary shift toward data-mining has “low” political and “medium” financial risks and moderate benefits.  Pilot program development would prevent public resentment and allow the DOD to assess the extent of this problem.


This is the high-risk/ high-reward option.  It would represent a significant and public shift for DOD agencies to enhance data-mining techniques.  The political and financial risks would be extensive, but the benefits would be substantial.

There are several indications this is the right course of action.  First, data output will be 50 times greater in 2020 than it was in 2010; a nearly exponential increase. The DOD needs to keep up with trends in data use.  Second, with the proliferation of technology, third party game developers are becoming commonplace.  Games are easier and cheaper to produce and there will be more and larger haystacks to search for the needles.  Finally, the effectiveness of current programs are unknown.  For example, in an experiment by British researchers, their data-mining software caught 60% of the malicious/terrorist-like activity. With significant public investment, we could dramatically increase our detection capabilities in the virtual realm.

In sum, there is sufficient evidence to conclude that GVEs are a potential asset for violent non-state actors, especially Islamic terrorist groups.  In fact, Islamic extremists have demonstrated a willingness to adopt new technologies and gaming into their arsenal.  However, the most likely outcome is that usage of virtual worlds for funding, communication, and recruitment will be rather limited.  The perks of GVEs are a significant draw for these organizations, but one blunder or oversight can be devastating for the network; once discovered, the advantages quickly dissolve.  As the IC community ramps up SOCMINT and other digitally-related INT’s with innovative data-mining methodologies, illicit networks may retreat into the shadows of real world activities.

Consequently, the pilot program development and evaluation is the most politically and financially feasible option when the costs and benefits are considered.  It is imperative we assess the extent to which terrorists are using GVEs.  Maintaining the status quo would be beneficial in the short-term, but recent trends illustrate that data-mining will be significant for law enforcement and national security in the near future.  A massive investment in these capabilities, on the other hand, is premature and will result in public blowback in the current economic climate. Nevertheless it is time we consider seriously new Islamic terrorism and the uses of virtual worlds.  After all, the worlds might be virtual, but the threats are certainly real.

Iran Backgrounder

Author: Michael Flickinger

Iran endures as the last functional theocracy in the world, steeped in a history rich with conquest, subjugation, and cultural innovations. Popular conceptions of Iran stir images of the 1979 Revolution, Ayatollah Ruhollah Khomeini, seizure of the American embassy, and murals in Tehran depicting America as “the Great Satan.” More recent portrayals, however, offer what some speculate might be a crack in the impenetrable regime that has ruled Iran since the Revolution. Pictures of men and women adorned in green to support Mir Hossein Mosavi, leader of the Reformists and the Green Movement, chanting, “Death to the dictator” mingle with footage of Neda Agha-Soltan bleeding to death, while Basij militiamen on motorcycles beat protestors. In the span of three decades, violence once again erupted within Iran and protestors railed against the current regime with a fervor similar to Khomeini’s followers against the shah. For those who favor a less opaque and more cooperative Iran, these events raised the hope that change looms on the horizon. But does it?
Continue reading