Efficacy of MedMyst: an Internet Teaching Tool for Middle School Microbiology

LESLIE M. MILLER,*1 JANETTE MORENO,1 VICKY ESTRERA,1 AND DAVID LANE2
Center for Technology in Teaching and Learning1 and Department of Psychology2, Rice University, Houston, Texas, 77005

Can web-based technology be used to effectively introduce or reinforce aspects of microbiology to middle school students? This central hypothesis examines whether brief exposure to a web adventure format containing virtual lab experiments and computer games within an engaging story line can impact student learning. An episodic adventure series, MedMyst (http://medmyst.rice.edu), focuses on infectious diseases and the microbes that cause them. The website is not intended to replace classroom instruction, but rather to engage students in problem-solving activities not likely to be encountered elsewhere. It also provides scientists with a resource to introduce microbiology to adolescent audiences through outreach activities. In the online adventure, the player (student) enters a futuristic world in which he or she becomes a "Reconstructor,” a member of an elite team charged with preventing the spread of infectious disease. The series consists of three "missions,” each lasting approximately 30 to 40 minutes and designed to address a limited set of learning objectives. Middle school students participated in the creation of the characters and the stylized design through focus groups. Classroom teachers oversaw the alignment of the web adventure objectives with the National Science Content Standards. Scientists and clinicians reviewed the web adventure for content and accuracy. A field test involving over 700 students from nine different schools assessed the knowledge gains attributable to playing MedMyst. Gain scores from pretest to posttest indicated that middle school students retained important information by interacting with the online material for as little as 30 minutes per adventure; however, gains for high school students were less persuasive, perhaps indicating a different learning tool or content is required for this age audience.

The roadmap for incorporating best practice in education with the creativity of multimedia is based upon the type of learning environment described in Benchmarks for Scientific Literacy (1) and the National Science Education Standards (6, 20). The alignment of the MedMyst content with these standards was central to the materials development process. A summary of the specific standards incorporated is provided in Table 1.

 

TABLE 1. Correlation of MedMyst content with the National Science Education Standards

 

Much of what is available on the web is a compendium of information with little structure or few scaffolds on which students might build their own knowledge. Some observers have charged that this has led to —lazy" learning models, —where the student is simply confronted with a vast resource and left unguided" (23). This type of exploratory learning can sometimes have an effect opposite from that which was intended. It can cause disorientation, difficulty in navigating from point to point, and cognitive overload (24). To overcome these challenges, several researchers have used a storyline or narrative as a means of providing structure to multimedia materials (12, 13). Researchers have proposed —that narrative shapes our knowledge and experience and is central to the processes of teaching and learning because it aids reconstruction, retrospection, prediction and memory as well as motivation" (21). The use of narrative in combination with multimedia has several advantages:

  • It sets a context or situates a problem that is to be solved (4, 7, 8).

  • It grounds the use of text, graphics, animation, voice, music, and interactivity with a certain mood or theme (16).
  • It allows the layering of learning objectives so that an array of objectives as described by Bloom‘s Taxonomy (3) can be woven into the story or problem.
  • It exploits theories of constructivism that suggest narrative is a powerful instructional tool (5, 25).

The survey, — Internet Access in U.S. Public Schools, Fall 2001," revealed that 99% of public schools in the United States had Internet access (19). Recent national studies indicated that 70% of American households with children ages 2 to 17 years have computers (10) and 52% are connected to the Internet (2, 22). Our own 1999 survey of middle school students in the Houston metropolitan area indicated that 80% report having Internet access at home (14). Therefore, crafting materials for this new mode of delivery is forward looking, more economical than print media, and has tremendous dissemination advantages. Creative electronic resources can reach students not only in the formal classroom environment, but can also become a part of the education environment in homes or museums and among the home-schooled population.

Students of both genders reported their dominant form of computer use was —playing games" (11, 17). These preferences speak to the viability of using innovative —game playing" as a way to teach adolescents science. If gaming strategies and animations can be designed for microbiology content, might this lead to student learning?

MATERIALS

Development of web adventures. Medical Mysteries (MedMyst) was produced through an interactive design process that engaged researchers, clinicians, educators, and students. This process of moving from Science Standards to

final production of the interactive web adventures is described elsewhere (15). Three of the four authors of this study were also involved in the website design. Prior to creating the web adventure, an informal survey of teachers and scientists was conducted to determine what middle school students should know about microbiology and infectious diseases. The distillation of survey responses, in combination with the Benchmarks (1) and Science Education Content Standards (20), led to the specification of learning objectives. These identified objectives were translated into learning segments within a narrative, then outlined in a screen-by-screen storyboard. Macromedia FLASH was selected as the programming tool because of its web capabilities. Animation and sound were added to the storyline only where it supported the content without unreasonably enlarging the file size. Aware that some users may not always have a high-speed connection to the Internet or a machine with a fast processor, the creators weighed each programming decision

The storyline allows the introduction of a variety of mechanisms to present and test learning outcomes as described by Bloom‘s Taxonomy (3). Objectives range from the simplest Knowledge level in which the student is asked for factual answers, testing recall and recognition, to the more complex levels of Application, Analysis, or Synthesis, which require students to predict what would happen, draw conclusions based upon new information, or propose an alternative. For example, in order to enter the lab, the player must know the answer to a content question; in another scene, the player must draw conclusions from an experiment‘s results before being allowed to proceed.

By way of illustration, the objectives from Mission One are:

  1. Simulate an experiment demonstrating Koch’s Postulates.

  2. Apply the steps of Koch’s Postulates to the identification of a disease in humans.

  3. Identify the six types of infectious agents and their characteristics.

  4. Describe the role of the immune system in fighting diseases.

  5. Associate modes of transmission with each type of pathogen.

  6. Distinguish the different treatments and preventatives that are most effective for each pathogen.

  7. Match types of infectious agents to the specific disease they cause.

  8. Recognize persons from history who made contributions to Germ Theory.

The storyline is the thread that pulls the students through the content and levels of questioning in a meaningful way. In MedMyst the underlying premise is that the player (student) enters a futuristic world in which he or she assumes the role of a "Reconstructor" charged with preventing the spread of infectious diseases. Figure 1 presents the initial context that the player encounters.

 


FIG. 1. The underlying scenario for the MedMyst series.

 

At the beginning of each adventure, the student is presented with a —problem" that must be solved. During a mission, students conduct field and laboratory investigations with the aid of the MedMyst characters. Each mission can be played within one class period (approximately 40 to 50 minutes). It is recommended that Mission One be played first because it covers the basics, and as the title indicates, it serves as an —orientation" to the concepts and the characters; however, either Mission Two (cholera) or Mission Three (smallpox) may be chosen to follow Mission One. Because each is a self-contained problem, either one may be played without reliance on the other Mission. Cholera and smallpox were selected because they represent two different categories of infectious agents (bacteria and viruses). The series unfolds along these lines:

Mission One: Orientation in Orb. The mission begins in the year 2254 in the Neuropolis Center for Disease Control. Prior to assignment to the field as new members of the Reconstructors, students must successfully complete five challenges based on their knowledge of microbes and infectious disease. The games require students to demonstrate their knowledge of Germ Theory, infectious agents, infectious disease vectors, the immune system, and treatment and preventative measures for infectious diseases.

Mission Two: Peril in Prokaryon. In the Neuropolis outpost of Prokaryon, the student must stop an outbreak of an infectious disease that threatens a refugee camp. The player must conduct an epidemiological study; identify the infectious agent through a microscope; discover the source of the illness through interviews, maps, and a case control study; and determine how to treat and prevent the illness from spreading.

Mission Three: Nemesis in Neuropolis. Working against time, each player must solve the mystery of an infectious disease never seen before in Neuropolis. Is it the result of bioterrorism? How can it be stopped? To succeed, players must piece together clues from the life of the first victim and his family, use a virtual electron microscope to identify the infectious agent, and decide whether to implement a vaccination program for the community.

Throughout the adventure, interactive puzzles and quizzes are used to ensure that the students have understood the information. Figures 2 and 3 provide specific illustrations of the interactivity. If a student has difficulty with the knowledge assessments, hints are provided, usually by one of the characters in his/her role as tutor (Fig. 4).

 


FIG. 2. An activity requiring students to sequence Koch's postulates.

 


FIG. 3. An activity requiring students to sort bacteria according to shape.


FIG. 4. Example of the feedback provided to students.

 

METHODS

Subjects. In order to secure a student population for testing, teachers from around the United States were recruited at MedMyst workshops presented at the regional and national meetings of the National Science Teachers Association. As an incentive, a stipend of $400 was offered. From among the teacher applications, the final selections were made to ensure that the sample of students reflected diversity in socio-economics, rural and urban locations, and ethnicities. The final list of nine schools, from eight different states, is displayed in Table 2.

 

TABLE 2. Demographics of field test population

 

From these nine teachers‘ classes, a total of 710 students participated in the evaluation. Since human subjects were involved, research approval from the University's Institutional Review Board was sought under Exemption 1. The website‘s content is part of regular science curricula that addresses aspects of the immune system and infectious diseases; therefore, the content of MedMyst is part of regular classroom instruction. All data collected are kept confidential and reported by school number without identifying individual students.

The percentage of students that receive free or reduced lunch (under guidelines set by the National School Lunch Program administered by the United States Department of Agriculture) was used to determine the school‘s socio-economic status (Table 2). Students were from grades six (n = 73), seven (n = 539), nine and ten (n = 98). There were nearly equal numbers of females (n = 350, 49.3%) and males (n = 349, 49.2%) with only 1.5% (n = 11) of students not responding to the gender identification question. The students ranged in age from 11 to 17 years (mean = 13, SD = 1.65). The majority of students were Anglo (n = 423, 59.5%), followed by Hispanic (n = 113, 15.9%), African-American (n = 56, 7.9%), Asian (n = 19, 2.6%) and Native American (n = 5, 0.7%). A few students identified themselves as —mixed" (n = 35, 4.9%), while 8% (n = 59) did not identify their ethnicity. The students were representative of varied achievement levels, ranging from —below average" to —above average" as reported by the teachers.

Procedures. Evaluation of the web adventure was independent of other instructional intervention. Teachers were asked to avoid preteaching the content of the website, but after the testing they were encouraged to extend the website content according to their own curricula. All schools were assigned Mission One since it presents the basic concepts that are elaborated in later Missions. Five of the schools were randomly assigned Mission Two and four were assigned Mission Three as indicated in Table 2.

Assessment instruments. The evaluation consisted of four instruments: (i) a pretest for all three missions administered three days prior to playing the assigned adventures, (ii) an opinion questionnaire administered immediately after completing the missions, (iii) a posttest administered 3 days after completing the adventures, and (iv) a teacher feedback questionnaire. Only the student test data are reported in this paper since the hypothesis under investigation focuses on learning gains, rather than preferences or perceptions. While a formal analysis of the attitudinal data is not reported here, it is relevant to the integrity of the scores to provide some insight into the teacher and student responses. Of central importance is whether the students found the materials engaging or whether they dismissed them as boring. The following comments in response to the question, —What did you like most about MedMyst?" are from both teachers and students.

Sample teacher comments:

  • "It was interactive and gave students choices and decision points.”

  • "I liked the fact that it has quizzes after each activity.”

  • "I like using it with my students because they are engaged. When students say to me, "I wish we could learn like this more,” then I am hooked. When they want the website to work on it at home, I know that it is effective!”

Sample student comments:

  • "I liked how they turned it into a mystery, that really got me into it!”

  • "I felt like a professional agent chosen to be on a mission.”

  • "You could interact with objects like viewing slides under a microscope.”

  • "It was better than reading and writing things.”

The pretest-posttest instruments normally required 15 to 30 minutes for students to complete. The actual interaction time at the computer for each mission was typically 30 to 40 minutes. The pretest and posttest included 40 multiple-choice science and history content questions (13 from Mission One, 15 from Mission Two, 12 from Mission Three).

Since both the tests contained the same items, the questions were randomized in a different order for the pretest and the posttest. Content questions focused on knowledge of the six infectious agents and their prevention and treatment methods, cause and treatment of cholera, cause and treatment of smallpox, structure and replication of bacteria, structure and reproduction of viruses, epidemiology, and people of historical significance and historical events related to Germ Theory. See Table 3 for a sample of questions contained in both the pretest and posttest.

 

TABLE 3. Sample questions from MedMyst pretest and posttest

 

Teachers were encouraged to play the adventures themselves prior to using them with their students. All teachers were instructed to administer the pretest three days prior to having students play the two assigned missions, allow students to play the two assigned Missions, wait three days and administer the posttest. The three-day time delay between the testing and the use of the web adventure were purposely planned to test the website‘s impact on student knowledge without the immediacy of content and exposure.

RESULTS

For each Mission a paired t test was performed on the total sample and for each school to determine whether there was a statistically significant increase in scores. Typically, the paired t test is used when the same subjects are measured before and after a process change. Additionally, the effect size was calculated for each pair-wise comparison. Effect size provides a measure of the magnitude of a treatment effect and unlike significance tests, these indices are independent of sample size (9).

Mission One. Table 4 shows the pretest and posttest scores, the gain scores, the t test, and effect size (d) for all those subjects who played Mission One. It is clear from Table 4 that students‘ performance improved after playing MedMyst Mission One. Across all schools, the gain score was significantly greater than 0, P < 0.01. Effect sizes of 0.50 are commonly considered medium and those greater than 0.80 are considered large (9). By these standards, the size of the gains appears to be reasonably large for the overall sample and for eight of the nine schools. Only the gains for School 5, seventh grade demonstrated an effect size well below 0.50.

 

TABLE 4. Mission One scores on 13 item test

 

Separate analyses of variance (ANOVAs) were performed for the group of schools that played Missions One and Two and the group that played Missions One and Three. For the former schools, the sizes of the gains varied somewhat as a function of grade. The mean gain scores for sixth, seventh, and ninth grades were 1.93, 2.47, and 1.02. There were no apparent effects of gender. A grade by gender ANOVA found a significant effect of grade, F(2,377) = 8.73, P < 0.01, but no significant effect of gender or a grade by gender interaction. For the group of schools that played Missions One and Three, no significant effects were found.

Mission Two. As can be seen in Table 5, comparable results were obtained for Mission Two. The only difference is that on Mission Two, School 9 ninth grade students showed virtually no gain. The effect size for School 9 was extremely small 0.15. The gains for the other schools were all significant, P < 0.001, and the effect sizes were large. As in Mission One, the sizes of the gain varied somewhat as a function of grade. The mean gains for sixth, seventh, and ninth grades were 1.63, 2.03, and 0.31. There were no apparent effects of gender. A grade by gender ANOVA found a significant effect of grade, F(2,377) = 18.82, P < 0.01, but no significant effect of gender or a grade by gender interaction.

 

TABLE 5. Mission Two scores on 15 item test

 

Mission Three. Table 6 shows the gains for Mission Three, all of which were significant, P < 0.01; however, the effect sizes range from 0.56 to 0.33, which are less impressive than Mission One or Mission Two. A grade by gender ANOVA revealed no evidence of a grade effect, a gender effect, or a grade by gender interaction on Mission Three.

 

TABLE 6. Mission Three scores on 12 item test

 

DISCUSSION

The hypothesis for MedMyst evaluation focused on the efficacy of web-based technology in delivering substantive microbiology content to adolescents. The results reported here indicate that MedMyst was successful in supporting students‘ learning. The investment of 30 to 40 minutes yielded gains even with a three-day delay in testing and without accompanying classroom instruction. Finding efficacious materials for middle school students is particularly important since this is normally when interest in science begins to wane. Serving as an alternative teaching tool, MedMyst may even reach students not normally interested in science, as well as those who prefer more interactive and visual learning environments.

The diversity of the student sample and the differences in preexisting knowledge documented in this study represent the range of ethnic, socio-economic, and ability differences among students who might encounter the web adventures. In general, students from middle schools were able to improve their scores based on using the web adventure alone. The gains for the high school students sampled were not as consistent or as large as those of middle school students. This reaffirms that the designers‘ intended audience (middle school age students) is the best match for the materials. The genre and the format may not be as appealing to high school students or may be too repetitious of previously learned material.

The experimental design in this study tested the efficacy of MedMyst in a broad range of school settings and independent of any teacher intervention. The investment of time on task for MedMyst affords some clear learning gains for both genders, particularly with middle school students. With the addition of classroom instruction and relevant hands-on activities, gains in knowledge acquisition could be magnified. To further this end, there are now teacher materials available on the website. These consist of a magazine in pdf format to accompany each mission and hands-on activities from which the teacher may select those most appropriate for his or her classes. The other key feature of the MedMyst series is that it takes relatively little class time. In many instances, students could be assigned the web adventures as homework. With the many pressures of standardized testing and the requirements to march through a specific curriculum, teachers often have little time for —extra" material. MedMyst is a catalyst that can hopefully serve as an introduction or review to microbiology concepts that are part of the National Science Content Standards, leaving class time for hands-on activities. The authors would welcome the use of MedMyst by universities or individual microbiologists in their school outreach programs. A previous paper provides middle school science and health teachers with an overview of how to integrate MedMyst adventures into classroom instruction (18). Some of the obvious advantages of this tool are that it is free and available at any time or from anywhere on the web. Future plans include the creation of more missions to cover additional infectious agents and translating the website into Spanish. The comparison of web adventure learning with an equal investment of time on other presentation modes offers more research questions for exploration.

ACKNOWLEDGMENTS

This work was supported by a Science Education Partnership Award (R25-RR15295) from the National Center for Research Resources, National Institutes of Health. We would like to thank all the teachers and students who participated in the field tests. In addition, a debt of gratitude is owed to our scientific advisors: Major Bradshaw, C. J. Peters, Robert Tauxe, and Joseph McCormick for serving as vital members of our scientific advisory team. Thanks also to Lynn Lauterbach and Liliana Rodriguez for their help in preparation of the student and teacher materials.

REFERENCES

  1. American Association for the Advancement of Science. 1993. Benchmarks for scientific literacy. Oxford University Press, New York, N.Y.
  2. Annenberg Public Policy Center. 26 June 2002, posting date. Media in the home 2000: the fifth annual survey of parents and children. [Online.] http://www.annenber gpublicpolicycenter.org/05_media_developing_child/mediasurvey/survey7.pdf.
  3. Bloom, B. S. (ed.). 1956. Taxonomy of educational objectives: the classification of educational goals, by a committee of college and university examiners. D. McKay, New York, N.Y.
  4. Brown, J. S., A. Collins, and S. Duguid. 1989. Situated cognition and the culture of learning. Educ. Res. 18(1):32–42.
  5. Bruner, J. S. 1996. Frames for thinking: ways of making meaning. In D. Olson and N. Torrance (ed.), Modes of thought: explorations in culture and cognition, p. 93–105. Cambridge University Press, New York, N.Y.
  6. Center for Science, Mathematics, and Engineering Education. 1999. Designing mathematics or science curriculum programs: a guide for using mathematics and science education standards. The Center for Education of the National Academies, Washington, D.C.
  7. Cognition and Technology Group at Vanderbilt. 1990. Anchored instruction and its relationship to situated cognition. Educ. Res. 19(6):2–10.
  8. Cognition and Technology Group at Vanderbilt. 1993. Anchored instruction and situated cognition revisited. Educ. Technol.. 33(3):52–70.
  9. Cohen, J. 1992. A power primer. Psychol. Bull. 112:155–159.
  10. Kaiser Family Foundation. 15 November 1999, posting date. Kids & media @ the new millennium: a comprehensive national analysis of children‘s media use. Kaiser Family Foundation, Menlo Park, Calif. [Online.] http://www.kff.org/entmedia/1535-index.cfm.
  11. Kaiser Family Foundation. 24 October 2000, posting date. Children and video games. Kaiser Family Foundation, Menlo Park, Calif. [Online.] http://www.kff.org/entmedia/3271-index.cfm.
  12. Laurillard, D. 1998. Multimedia and the learner‘s experience of narrative. Computers Educ. 31:229–242.
  13. Laurillard, D., M. Stratfold, R. Luckin, L. Plowman, and J. Taylor. 15 August 2000, posting date. Affordances for learning in a non-linear narrative medium. J. Interactive Media Educ., 2. [Online.] http://www-jime.open.ac.uk/00/2.
  14. Miller, L. M. 2000. Middle school students and technology: habits and preferences. TechEdge 20:22–25.
  15. Miller, L. M., J. Moreno, V. Estrera , D. Smith, and J. Mayes. 2003. Constructing web-based adventure games: a model that works, p. 888–891. In World Conference Proceedings on Educational Multimedia, Hypermedia & Telecommunications. Association for the Advancement of Computing in Education, Norfolk, Va.
  16. Miller, L. M., H. Schweingruber, R. Oliver, J. Mayes, and D. Smith. 2002. Teaching neuroscience through web adventures: adolescents reconstruct the history and science of opioids. Neuroscientist 8:16–22.
  17. Miller, L. M., H. Schweingruber, and C. L. Brandenburg. 2001. Middle school students‘ technology practices and preferences: re-examining gender differences. J.Educ. Multimedia Hypermedia 10:125–140.
  18. Miller, L. M., V. T. Estrera, J. Moreno, and D. O. Smith. 2003. Solve medical mysteries. Science Scope, Middle School Teacher Journal for National Science Teachers Association, in press.
  19. National Center for Education Statistics. 2002. Internet access in U.S. public schools and Classrooms: 1994-2001. U.S. Department of Education, Office of Educational Research and Improvement, NCES 2002-018, Washington, D.C.
  20. National Research Council. 1996. National science education standards. National Academy Press, Washington, D.C.
  21. Plowman, L. April 1997, posting date. Getting sidetracked: cognitive overload, narrative and interactive learning environments. In Virtual learning environments and the role of the teacher, Proceedings of UNESCO/Open University International Colloquium, Milton Keynes, UK. [Online.] http://meno.open.ac.uk/lydiacog.html.
  22. Shields M. K., and R. E. Berhman. 2000, posting date. Children and computer technology: anaylsis and recommendations. The David and Lucile Packard Foundation, Los Altos, Calif. [Online.] http://www.futureofchildren.org/pubs-info3133/pubs-info.htm?doc_id=78956.
  23. Weller, M. 15 August 2000, posting date. The use of narrative to provide a cohesive structure for a web based computing course. J.Interactive Media Education 1. [Online.] http://www-jime.open.ac.uk/00/1/weller-00-1paper.html.
  24. Wild, M., R. Oliver,, R. Phillips, G. Rehn, and R. Dickinson. 08 August 2001, revision date. What is the problem to which interactive multimedia is the solution? Views on the nature, place and value of multimedia in education. Issues in Educ. Res. 4(2):57-79. [Online.] http://education.curtin.edu.au/iier/iier4/wild.html.
  25. Yager, R. 1991. The constructivist learning model. Science Teacher 58:52–57.

___________

*Corresponding author. Mailing address: Center for Technology in Teaching and Learning, Rice University, 6100 Main Street, MS 120, Houston, TX 77005. Phone: 713-348-5352 E-mail: lmm@rice.edu.

(Return to Top)


DOI: 10.1128/jmbe.v5.73
MICROBIOLOGY EDUCATION, May 2004 Vol. 5
Copyright © 2004, American Society for Microbiology. All Rights Reserved.



JMBE
ISSN: 1935-7885

American Society For Microbiology © 2014   |   Privacy Policy |   Terms of Use