Evaluation of Breakthrough's ICED! Video Game

This report is an evaluation of a video game called ICED! (I Can End Deportation), which the human rights organisation Breakthrough developed in an effort to educate a wide audience about United States (US) immigration laws and practices and deportation policies. In winter 2007, Breakthrough engaged the Education Development Center/Center for Children and Technology (EDC/CCT) to assess changes in ICED! game players' knowledge and attitudes about US immigration and deportation policies due to game play. EDC/CCT developed a 14-item, online, pre/post questionnaire to collect data. The questionnaire items addressed 3 broad content areas to which players are introduced in the game: US immigration policies; immigrant detention and deportation policies; and immigrants' civic behaviour options as they are influenced by policies and laws.

Researchers analysed questionnaire responses from 6,007 cases. One hundred twenty-nine players completed all pre- and post-questionnaire items, 99 of whom had complete data sets (in which all survey items were completed) for assessment.

The analysis yielded the following findings, discussed in greater detail in the full report:

• Playing ICED! contributed to an increase in player knowledge about US immigration and deportation policies. An analysis of the paired pre/post knowledge scores for a group of 99 respondents revealed a mean score increase from pre- to post-game play: average 6 out of 12 correct at pre, 9 out of 12 correct at post. Similarly, the trends in the much larger groups of players who only completed the pre (n=5,148) or post-tests (n=730) suggest that playing ICED! did lead to gains in learning outcomes.
• Over half (56.5%) of the respondents in the matched pre/post group indicated that playing ICED! had changed their attitude about the ways in which immigrants are treated in the US. While the majority of respondents did not complete the open-response associated with this question, those who did indicated the game had changed their attitudes positively toward immigrants' rights.
• A question about player experiences during game play revealed these themes: (i) Perspective taking and role playing enabled some players to consider the facts presented in the game from immigrants' points of view; (ii) ethical and situational dilemmas encountered by immigrant characters in the game engendered strong written responses to the open-ended question, primarily in 2 directions: some players argued that the scenarios revealed a strong bias among the game developers, while others wrote that they did not realise how precarious day-to-day living can be for immigrants; (iii) several players noted that playing ICED! might contribute to a heightened distrust of local law enforcement authorities by stating that immigrants face a greater risk of detention and deportation by reporting crimes.

The authors of this report contend that these findings suggest that ICED! is an effective tool for educating a broad audience about facts associated with these issues and for influencing their attitudes about US immigration policies. The final section of this report includes the following recommendations:

• Rather than presenting content in terms of true/false dichotomies, consider embedding it in problems that require game players to apply what they learn through game play to the solutions. While the majority (83.5% for the matched group, 59.7% for post-only group) of all respondents indicated that immigrants are entitled to the same human rights as US citizens, many seemed to feel that these "situational dilemmas" are complex and perhaps less easily answered through true-false options. Game problems that require players to consider multiple perspectives in order to solve problems may help them to develop a richer understanding of the real-world problems.
• Provide clear links to the sources for the content presented in the game. While game play allows players to contextualise issues, links to sources might allow them to explore the issues on their own and to develop a greater sense of confidence in the information that is presented.
• For future data collection, Breakthrough should consider "tightening" the connection between pre- and post-test. An increase in matched pairs will facilitate more analysis in terms of player learning outcomes and attitude changes.
• If ICED! is used as a tool to complement classroom curricula, consider asking an open-ended question about player attitudes toward US immigration policies in the pre- and post-tests in order to explore whether and how game play influences those attitudes.