TeleVISIONS (January 1980)

The Ongoing Role of Formative Evaluation By May of 1979 there were five test shows ready for evaluation by the true experts in all of this — 8-12 year-old children. In two phases, the research team set out for New Jersey, Tennessee, Mississippi, Texas, Illinois, and California, with the test shows and a set of research methods including the Program Analyzer technique, now computerized, the small-group interviews, post-viewing segment and cast appeal voting, program comparison voting, and a newly designed Teacher Case Study, aimed at observing the use of 3-2-1 Contact in actual classrooms, and at getting input from teachers themselves on the content and format of the series and its prototype Teacher's Guide. The show’s drawing power at home was measured as well, via Warner Communications’ 2-way QUBE cable system in Columbus, Ohio. A sample of 737 5-15 year-olds was recruited by telephone and invited to watch and respond to two 3-2-1 Contact shows on their home TV sets. Fifty-six percent of the recruited sample did tune in 1 show; 30% watched both. By summer's end the research team had seen over 2,000 children, bringing the grand total of children consulted to over 10,000. Together with a content staff of science experts, the researchers draw upon their intimate acquaintance with the target audience to provide continuing recommendations for the design of the series. Researchers provide insights about the target audience to producers and writers before film is shot and tape produced, and afterwards, as pieces are edited and assembled. There are difficulties inherent in providing such input, however. As the project expanded into its production phase, for example, dozens of new staff members were brought on board, many of whom had never worked with children, with science, or with formative researchers. The research team is continually faced with the challenge of communicating to team members two years of insights into the target audience. Sometimes we are successful, and sometimes creative decisions successfully take paths not fully endorsed by formative research. At this writing, the evaluation of the test shows has taken place, but the shows that will be aired are not yet complete. There remain a number of key trade-offs that have not yet been fully resolved. As we observed the reactions of children to the test shows, it seemed that a gain in the entertainment value of a piece often came at the expense of explicitness in the science content. The decision was made to structure the series around documentary film remotes that feature cast members talking to people out in the world of science, and exploring that world on behalf of the target audience. Such pieces have to be pulled together back in the studio, much as the diverse elements in Sesame Street are unified by the street sequences. However, the more serious tone of these film remotes was not always followed through in the studio treatment. Legitimate concern with appearing didactic sometimes gave way to a much ligher tone that was entertaining, but not consistent with the feeling established in the documentaries. It was also a well-motivated decision to offer children a diverse array of subjects and elements in the series, and the thirteen weekly themes, such as Communication, Growth and Decay, and Forces, provide the threads that bind the elements together. Yet there is a danger of appearing superficial within each piece, as we offer a rich and varied menu overall. A goal of the series is to encourge children to explore, to raise questions. But if, in the interest of variety and entertainment, we raise more questions than we can answer, will the show be satisfying to children? Thus, at the end of the test show phase, we were still faced with the fundamental problem of making coherent educational points in an entertaining way. The resolution is still in process as series production takes place. Ultimate considerations about emphasis, style, and talent, for example, as well as implications of budget and time constraints are, as always, left to producers. Yet when a producer will sit in an editing room and caution the assembled staff to, “Remember those 10-year-olds watching this at home alone,” it becomes clear that formative research has served an important purpose: to introduce the audience itself into the creative process. Barbara Myerson Katz is a graduate of the University of Pennsylvania and the Harvard University Graduate School of Education. She has taught English, film, and communications to high school students, and has developed programs, written scripts, and carried out field research for educational television projects in the Philadelphia schools. At the Children’s Television Workshop, she has been an independent researcher for Sesame Street, and since August 1977 has been Field Research Specialist for 3-2-1 Contact. Televisions — 31