Faculty Blog

One of the fundamental aspects of gaming that influences your research and game design are the research questions. What questions are you attempting to answer in this game? Accordingly, a purpose statement is first developed which establishes the central direction of the game. Mainly in qualitative games, the purpose statement will describe the central phenomenon, indicate the strategy of inquiry and may even mention the players or individual, group or organization of which a game may support.  From the broad, general purpose statement, the research team narrows the purpose down even further into the game’s objectives. Specific research questions are then developed. In a heavy qualitative, inductive game such as the Global Maritime Partnerships (GMP) Game 2010, the research questions developed assumed two forms: a central question and subsidiary questions. The central question is a broad question that asks for an exploration of the central phenomenon or concept being explored in this game. Ask yourself, “What is the broadest question I can ask in this game in order to effectively investigate the sponsors underlying problem?” The intent of many games is to explore the complex set of factors surrounding the central phenomenon and present the varied perspectives or meanings that the game participants hold.

Applying Creswell’s model to the development of research question in gaming, the following are some general principles that may assist you in writing broad, qualitative research questions:

a) Ask one or two central questions followed by five to seven sub questions. The several sub questions follow each general central question; the sub questions help narrow the focus of the game and bound the problem you’re exploring.

b) The central question(s) and subsidiary questions should be grounded from the game’s purpose, objectives and related literature (military documents, scholarly articles, etc.).

c) Begin the research questions with the words what or how to convey an open and emerging research and game design.

d) Expect the research questions to evolve over the course of the game in a manner consistent with the assumptions of an emerging game design.

e) Use open ended questions, without reference to the literature or theory. However, in a game where you’re examining specific polices, plans or strategies, or using a document to generate specific hypothesis, it may be appropriate to cite or use this document as part of your research question. For example, one may consider a central research question to be, "Based on the existing Fleet CONOPS, what are the gaps that limit the Navy’s ability to sustain maritime operations in the Caribbean? The specific mission sets outlined in the CONOPS, which are then applied in the game, is your independent variable, and the gaps that preclude you from accomplishing those missions is your dependent variable.

Your independent variables are those that almost certainly cause, influence, or affect outcomes. Your dependent variables are those that depend on the independent variable; they are the outcomes or results of the influence of the independent variables. We may have intervening or mediating variables that stand between the independent and dependent variable (Creswell, 2009). These could be viewed as other factors that may also influence an outcome.  

Once we are equipped with the research questions and hypotheses, we will then be prepared to apply the appropriate model of gaming to a project (e.g. one-sided, one and a half sided, and two sided), develop the data collection instruments, and prepare the scenario and game products.

Quite often in gaming, the research being conducted to address the problems or questions raised by sponsors is highly inductive and qualitative in nature. An inductive approach in gaming, of which is described in the Data Collection and Analysis Plan (DCAP), calls for the collection and analysis of data to identify any patterns emerge that suggest a relationship between variables. The team of gaming professionals will undertake a comprehensive literature review at the very beginning of the research project. However, the literature review is not something that is just conducted early on in the gaming process. Many of the new insights and implications garnered from the data during post game analysis encourages the analysis team to investigate even further in the literature.    

With an approach grounded in learning from the game participants, the literature review is used to frame the problem in the introduction of the game report. This provides a useful backdrop for the reader or sponsor of the problems or issues leading up to the need for the game. Periodically, the literature is often broken out towards the end of the game report, where it is used to compare and contrast with the results that emerged from game play.  However, when a grounded theory or phenomenological study is employed, such as in the case of the Navy Irregular Challenges Game (October 2010), the literature is less often used to set the stage for the game.

The literature review enables the research team to identify the gaps in knowledge that are worthy of future study, challenge current ideas in the field and acquire an accepted theory and apply to a specific area being explored in a game. The review of related literature assist in bounding the scope of the problem, developing the research questions, and understanding the various perspectives garnered from the game’s sponsor. Quite often the most practical and useful application of a games findings occurs when those new insights and ideas can be applied and examined to future games, such as in serial gaming. Between the inner war years, the War Gaming Department was successful in building upon many of the previous games executed, which in turn, assisted in identifying the growing naval threat in the pacific.

When a deductive approach is applied to gaming, such as in the case of the MDA Operational Game (July 2010), the literature review helps provide a source for the focus of the research, which included the development of the games objectives, research questions and hypothesis. Conversely, if a game favors more toward an inductive approach, as it did in the Global Maritime Partnerships Game (October 2010), the analysis team will begin with the collection and analysis of data, ‘a process that leads to questions that are then addressed through engagement with the literature’ (Gray, 2009).

The other component of a literature review that has yet to be mentioned focuses on research methods. The DCAP will discuss not only the research design, approaches and tools used for that particular game, but also for other studies and games and the academic sources used to justify the use of them.

The War Gaming Department strives to provide interested stakeholders with intellectually honest analysis of complex problems. Research in this context is a ‘systematic and organized effort to investigate a specific research problem that requires a solution’ (Sekaran, 2007:4). This type of applied research in War Gaming uses an accumulation of theories, knowledge, and techniques for the purposes of examining specific issues driven by sponsors.

Because many of the issues that interested parties ask us to explore involve qualitative issues and data (e.g., decision-making processes) war games rarely allow for inferential analysis. In other words, war games are not predictive; we cannot make generalizations beyond the data about how future players might respond to the problem at hand. The participants or players are often hand selected making it nearly impossible to use a random sample. That is why a significant focus in gaming is placed in the methodology (e.g. methods, tools and techniques) used.

In gaming, the initial process of inquiry usually starts at the Concept Development Conference or CDC.  From an analytical perspective, this is the most important planning meeting that occurs through the gaming process. Normally this is the first time the game sponsor and war gaming faculty meet to discuss the perceived stressors and challenges encountered.  Following Pete Pellegrino’s Purpose-Method-End State structure discussed in his latest blog entry will enable the game sponsor and war gaming faculty zero in on the specific objective and motivating factors for using gaming as a tool to explore these issues. This event sets the foundation for the reminder of the gaming process and helps define the problem and narrow the scope of the research down so that an analytical game can be developed. In order for a game to answer a sponsor’s problem(s), analysis must drive the design from the very beginning.

The development of the objectives is an art in itself. One problem solving technique used during the CDC is a technique called the ‘5 Why Drill Down’. This technique breaks down fairly complex problems into progressively smaller parts. Using this technique helps the sponsor identify the root causes of problems or incidents faced. When walking away from a successful CDC, both the game sponsor and War Gaming faculty should have a clear and common understanding of the specific problems leading to this game, any recommended literature or resources available about the research problem and deficiencies in the literature about the problem. From there the game’s purpose (e.g. why are we conducting this study - analytical, experiential, both) and objectives (e.g. clearly defined result) are finally formulated along with a good understanding of the sponsors hypothesis or assumptions brought into the game. At this point, potential participants or players can be identified. Saving this towards the end enables both the sponsor and gaming team to identify WHO really fits within the context of this game and WHY. 

Once the purpose and objectives are developed and agreed upon by the sponsor and the War Gaming Department, an overarching research question (1-2) and subsidiary questions (5-7) are developed.  The research design can be qualitative, quantitative or mixed depending on the purpose and questions developed. Going through this rigorous yet sometimes cumbersome process assures that that the subsequent game design fits according to the analytical framing and research design. 

Too often war game objectives become laundry lists detailing 20+ things the game will accomplish. After all, if one objective is good, then more must be better! Multiple objectives are often cited as creating more “bang for the buck.” Adding just one more objective shouldn’t be a problem, right? However, if you see the objective as:

(1) An outgrowth of the sponsor’s problem statement (i.e. why he wants a game),

(2) The driver for game design, and

(3) The successful outcome of the game (i.e. end state),  

Then how do multiple objectives fit the design logic? The tendency is to mistake methods for objectives, which explains why so many games end up with multiple “objectives,” most of which are verb based. “Explore the implication…” or “Identify shortfalls…” or “Review operational plans…” etc. 

Using a Purpose-Method-End State structure, you can see how you could have multiple METHODS, but there can only be one END STATE (objective). How do you want the participants or the state of the project to be different at the end of the event? One question to ask a sponsor is “This event will be a success if…” to get at an end state type objective. You’ll need a way to know if you actually achieved the objective, so it should be specific and measurable, otherwise how will you know if your game did what is was supposed to do?  

War gaming is much more than “hosting a war” and sending out invites - “You’re Invited to a War Game (bring your own order of battle).” Without a clear objective, you might as well throw some plastic army men, model ships and toy planes into a shoe box, shake it up, and ‘hope’ something interesting happens to justify the travel and per diem expense.  While you’re at it, toss in a Buzz Lightyear figure and your favorite Godzilla toy, because without a solid objective, Godzilla is just as likely to produce “something interesting” as anything else. 

After dumping the contents of your war game box back out onto the table, everyone will “see” what they want to see amongst the pile of toys. Like Ouija boards and horoscopes, everyone will be able to claim that the war game has validated their preferred outcome, even if they’re analytic opposites.  

This “throw a war together” approach is too often the default. It becomes more and more about creating the virtual environment, and less and less about why you’re trying to create the environment in the first place. You can see the potential problem when you have toys lying around, and for lack of a better approach, feel compelled to toss them into play.  

The objective is the game’s foundation. Bad foundations seldom lead to good outcomes!