Running Head: TESTING THE EFFECTS

Testing the Effects of Incorporating Tailor-Made

Multimedia Instruction on Television Production Skills

Joel R. Johnson

Illinois State University

This study is designed to be applied to the development of an educational interactive CDROM. The CDROM prototype has been created and is utilized in the current research to test various aspects of its design. Learning styles literature suggests the use of multiple teaching styles to maximize educational potential and training literature supports the use of interactive multimedia as a way to utilize multiple teaching styles. (Cannell, 1997; Shofield, Eurich-Fulcer, & Britt, 1992)

A limited focus group and an in-depth interview were used to acquire the data for study. Initial findings uncovered five main categories: resolution, access, tutorials, depth, and labeling. These categories point to changes needed in the mapping system and line art. They also point to a need for user tutorials and further consideration of format choices affecting accessibility. Because of the limited data acquired, the findings are tentative and further research will be required prior to the completion of the project.

Testing the Effects of Incorporating Tailor-Made

Multimedia Instruction on Television Production Skills

Today's students have been raised on a heavy dose of television. The information they receive is fast-paced, verbally succinct, and visually and acoustically stimulating. In contrast, today's classrooms rely heavily on readings and lectures to disseminate information. As a long-time assistant instructor of television production and one time student, director, and chief photographer in a student television production I have found the curricula to be lacking. Several textbooks provide overviews of the process of production in a manner that is accessible to students, but most equipment and technical manuals lack this accessibility.

What I am creating is a multiple learning style based approach to designing classroom curricula that will present information using an interactive multimedia CDROM. It is my hope that this type of approach will increase student satisfaction and this in turn will increase their competence in production skills. This study will include a brief literature review of research on learning styles, followed by an analysis of multimedia's role in a multiple learning style instructional approach. A prototype CDROM has been created and was tested during the course of this research. The CDROM used in this study was created specifically for this study and it is crucial to take into account certain design and aesthetic considerations. If multimedia instruction is to be effective, the methods employed in creating the images, audio, video, and electronic text must be of a quality that students are accustomed to. This point is of foremost importance to any instructional multimedia project's success. There will be extensive coverage in the methods section regarding the specifics of this multimedia project and its design process. Lastly I will reveal the results of the focus group and interview in the discussion section. It is important to note that this is applied research designed to advance a working prototype. It is also important to note that this is preliminary research and the results will be used to inform further field research prior to the completion of the CDROM project.

Review of Literature Kolb's (1985) Learning-Style Inventory was the initial basis of learning style research in this project. Kolb's inventory consists of 12 groups of sentences in which the respondent is asked to rank statements based on how they feel they learn best. These sentence groups contain sentences that relate to Kolb's four learning modes in the Cycle of Learning from Experience. Kolb describes the four learning modes (p. 4) as Concrete Experience (CE, "feeling"), Active Experimentation (AE, "doing"), Abstract Conceptualization (AC, "thinking"), and Reflective Observation (RO, "watching"). These can be used to determine which of the four "dominant learning styles" a respondent preferred. Each of these four styles represents different combinations of pairs of learning modes. The Converger "combines the learning steps of Abstract Conceptualization and Active Experimentation" (p. 7). The Diverger "combines the learning steps of Concrete Experience and Reflective Observation" (p. 7). The Assimilator "combines the learning steps of Abstract Conceptualization and Reflective Observation" (p.7). The Accommodater "combines the learning steps of Concrete Experience and Active Experimentation" (p. 7). Doris Mathew's (1996) review of Kolb's work presents dozens of studies supporting the idea that learning is related to educational achievement and her own study suggests that Kolb's learning styles can be accurately used as indicators of academic success. Robert Loo's (1996) study supported the concept that learning style remains constant. Loo, also supported the use of Kolb's LSI as a valid classificatory system. Kolb’s is one of several learning style inventories, profiles, and systems.

Rita Dunn (1988) offers an approach to learning styles also known as Learning Styles Inventory (LSI) that differentiates among auditory, visual, tactual, and kinesthetic strengths. Dunn claims young children and older students who do not achieve well learn better through a tactual/kinesthetic rather than an auditory or visual approach (Filipczak, 1995). Dunn's Learning Styles Inventory (LSI) is supported by research done in 36 studies (Given, 1997). Research done by Ganel P. Caldwell and Dean W. Gintheir (1996) reported that "variables on the LSI differentiating low from high achievers in math and reading, were used as predictor variables in a linear discriminate analysis. Predictor variables correctly classified 78.83% of the students in reading achievement and 80% in math achievement" (p. 141).

Training and development literature provided other approaches to understanding learning styles. Bernice McCarthy (1997) did a case study of four people who represent the four types of learners in the 4MAT. Jackie Taylor (1995) reduced the learning style preferences to visual, tactile, and auditory learners. Susan Boyd (Finley, 1996), divides personnel into Doers, Thinkers, Feelers, and Watchers. Mike Cannell (1997) describes people as preferring learning from books, traditional courses, and "new technology based methods" including interactive CDROM applications.

All of the academic research and professional strategy pointed to two things that might help improve the instruction of production students. First, while theories explaining the nature of the variables associated with learning styles differ, they all separate learners into specific and distinct learning style categories. This means that teaching in a single style could contrast with a large number of students' learning styles, decreasing their satisfaction, competence, and completion rates. Second, to improve performance they all suggest teaching in multiple learning styles. This allows students of various learning styles to learn some material in their preferred learning style.

Interactive multimedia is "making multiple learning styles available" (Hequet, 1997, p.20). Research done on the effectiveness of multimedia as an instructional tool seems to support further research into instructional multimedia effectiveness in the area of television production. Research findings show that "students using the tutors [interactive multimedia] had more control over the kind and amount of help they received from the teacher" (Shofield, Eurich-Fulcer, & Britt, 1992, p. 579). Norman K. Peterson and Barbara J. Orde (1995) report that instructors in multimedia classrooms were better able to accommodate individual student learning styles. Allowing this level of individual student choice has proven a useful way to approach student learning style differences.

Students with diverse cultural backgrounds may also benefit from the use of a multiple learning style approach. Understanding how multicultural students learn can provide instructors with a better understanding of cultural learning preferences that they might not have been aware of. This understanding can be used to create curricula with these diverse preferences in mind (Allen-Sommerville, 1996). Andrew S. Latham (1997) created the Kanachamch Early Education Project (KEEP) to address cultural learning styles. Latham found that it was highly effective with its original students, native Hawaiian elementary students. Latham's study also reported that replication of this study done using Navajo children suggested that the techniques used to treat one group may not be transferable to another.

There are also positive effects not directly related to learning styles research. James A. Kulik, Chen-Lin C. Kulik, & Cohen (1980) wrote an overview of 59 studies of computer based college teaching. They found computer based instruction made significant contributions to the course achievement of college students and also produced positive effects on the attitudes of students toward instruction and toward the subject matter they were studying. They also wrote that the most dramatic finding in this meta-analysis is related to time saving. They report that computer based teaching saved time over conventional teaching methods. Alexandrea Rand (1996) says one of the primary benefits interactive multimedia had over traditional instructional style was a timesaving of 50 percent or more. Mary Wise and Frank M. Groom (1996) reported that multimedia's "major effects may be in acclimating students for the technological world they are soon to enter as productive society members" (p.68).

Methods The focus group participants were all students of either basic or advanced television production courses for the Fall semester of 1998. All students were solicited during normal class periods. The basic course was comprised of approximately 50 students, 24 of whom filled out a form expressing interest in becoming involved. The advanced course was comprised of approximately 12 students 8 of whom filled out a form expressing interest in becoming involved. The forms included spaces to indicate name, phone number, and availability. Two focus group dates were set based on the availability of the students expressing interest. All students were contacted several days prior to the focus groups and those who could be were e-mailed the day prior to the focus groups. In addition the day of the first focus group was a class meeting day for the basic production class and a reminder was announced that day in class. The first focus group drew two basic production students and one advanced production student. The second focus group drew only a single student and had to be cancelled. The focus group sessions took place outside of class time and outside of the television production area at 7:00 p.m. on both evenings. Questions were pre-made and derived from personal experience and were open ended in nature. Additional questions were included as the participants moved into areas not on the pre-made question list. Coding of the focus group data followed the form outlined by Corbin and Strauss (1990) for open coding. There was no prior connection between myself and any of the participants in the focus groups. Open coding revealed ten categories: skills, access, communication, knowledge, tutorials, depth, labeling, interface, elements, and resolution. The items that were placed into the categories of skills, knowledge, elements and communication were collapsed into either the tutorial or the labeling categories. Interface is an important element but one which was not discussed and so it was dropped.

Delores James, Barbara Rienzo, and Carol Frazee (1997) utilized focus groups in research used to develop an educational video for high school students and achieved very good results with this technique. Focus groups "are stimulated by the experiences of other members of the group to articulate their own perspectives" (Lindloff, 1995, p. 174). Lindloff (1995) also guided issues of rapport such as self-disclosure demeanor, and active listening. The in-depth interview was added to increase the reliability of the findings through triangulation and to increase the amount of data available to work with (Lindloff, 1995).

The in-depth interview was conducted with the Television Production Coordinator, and sole instructor, John Fisk. Questions were loosely formulated based on the results of the data from the focus groups and also from personal experience. Many of the questions that were asked in this interview emerged from the interview as it progressed. This interview took place in an editing suite that was part of the television production facilities. I have known and worked with John for a number of years as a student, instructor, and as an assistant. Coding of the interview data also followed the form outlined by Corbin and Strauss (1990) for open coding. Categories revealed in open coding included all of the categories from the focus group interview plus immediacy, motivation, skills, transferability, computer experience, student/student interaction, curricula, and included elements.

The aesthetic and design considerations that go into creating an interactive multimedia curricula are crucial to its success. The interactive multimedia curricula format is a CD-ROM created using HyperStudio 3.1. This program was chosen due to availability and ease of use factors. The project is broken into five production-related areas called stacks. Stacks are composed of pages. Maps of ISU's television production facilities are provided (see appendix CELL 4 & 6) as well as QuickTime virtual reality images that allow a 360degree view of several rooms in the actual station. Line art was created for the video switcher (see appendix CELL 8), one of the most complicated pieces of equipment. Photographs, interviews and overviews from current and previous production students and instructors introduce and expound upon the various production positions with text and audio clips (see appendix CELL 2, 3 & 5). Two video segments produced at ISU's television production facilities will answer the questions "What is student television production?" and "What kinds of things are produced?" The videos will be button options presented at the introductory and exit points in the CD-ROM.

The project was organized spatially based upon the layout of the Television Production Department at Illinois State University (see appendix CELL 1). The idea is that the user enters the building through the introductory stack then moves to the basement hallway stack (see appendix CELL 2) where the direction you choose leads you to different elements of student television production. QuickTime VR movies of the building, basement hallway, studio and control room can be viewed firmly linking the multimedia with reality. Line art maps give users visual perspectives of the studio and control room in great detail (see appendix CELL 4 & 6).

There are five major elements considered in the design of this interactive multimedia project: text, audio, graphics, video, and interface. Each element has a significant effect on the usefulness of the project. The interface affects every other element. It should be as straight-forward and intuitive as possible. It must be consistent if the user is to successfully navigate the dense pages (Kristof & Satran, 1995).

The type elements should remain consistent within a piece for continuity and ease of reading. On a computer screen "twelve-point type is the smallest size that's comfortably readable" (Kristof & Satran, 1995, p.109) Generally for most on-screen applications, it can go much higher than that. Only scrollable text within the CDROM is as small as 12-point font. Contrast is also important in making the type readable. This project utilizes a neutral background color (beige) and blue type with a white drop shadow. San-serif fonts were chosen in order to facilitate reading ease on a variety of monitor sizes (Baird, McDonald, Pittman & Turnbull, 1993). Audio elements include music, sound effects and sound bites. Music was used for background sound on major screens. A "click" noise was inserted with each button click to verify to the user a button was, in fact, pressed. Sound bites were chosen from interview material gathered between February 1998 and May 1998.

Graphics were created using Adobe PhotoShop and Adobe Illustrator. Backgrounds were composed using PhotoShop. Text, maps and line art was created with Illustrator. Backgrounds are of a neutral color and were filtered to create a pattern making them look wall-like; these were generated in PhotoShop. Pictures were captured with the Kodak DC50 digital camera and processed in PhotoShop. HyperStudio supports JPEG compression to reduce the disk space "good" images take up. There are two video segments not yet incorporated into the CDROM. Both would be created in the facilities represented in the CDROM. These video segments would be created with short fast cuts, fast paced music, liberal doses of candid production sounds, and a charismatic announcer to do the off-camera voice work.

Categories from the focus group interviews pointed to one key problem with the prototype. The resolution category was used to describe issues of readability, recognition, and clarity or meaning. Resolution was a problem for all of the students in the focus group:

Comments of this nature help to point out the need for more detailed line art, particularly the maps (see appendix CELL 4&6). Pictures of equipment were exchanged for the line art to try and allow for clear identification (labeling) of specific components. More detailed line art could produce higher resolution while allowing for clear text to be placed appropriately. Photographs are extremely difficult to use and still maintain legibility of text, but perhaps combinations of photographs and illustrations may provide both.

Access was one of the students primary concerns and some overlap with the in-depth interview category of responsibility seems to be evident here:

I think that some of the value of utilizing computer-based training is coming out here and the literature review pointed to self-pace as one of the benefits of using this type of technology. It seems that this is one of the major benefits of an interactive CDROM.

Tutorials were a key element mentioned by both the focus group and the interview and this was an element that had not really been explored in the prototype product. This is an element that could be included through simple animations.

Labeling is a part of the program that students tend to gripe about. One requirement of the basic and advanced classes is the labeling of different pieces of equipment and operations. I believe that the complaints arise largely because this comprises a large portion of the written tests associated with these product-oriented classes. Labeling as I mentioned earlier can be associated with resolution in that higher resolution is likely to lead to more labeling information. Labeling is also associated with concepts, particularly to the basic students:

Many categories came up in interviewing the instructor that did not arise with the students. Some of these categories did not appear to have a related use to the creation of the CDROM; others pointed to clear directions that could be taken to improve it. Notable among these categories were transferability, computer experience, and included elements. Transferability was important to me because if this product was to be useful beyond ISU’s production facilities then equipment, concepts, and facilities had to be essentially the same. The interview strongly supported the idea that almost all of the elements could be completely transferable:

P1: …the names on the keys are kind of universal terminology and they’re going to find those anywhere.

Computer experience was an element that I had not given a lot of consideration when the prototype was created. Experience with basic functions can make the difference between a success and a failure as far as usability. I incorporated textual cues (i.e. click the pic to continue) but visual cues and basic instruction on what "clicking" meant was not included. This is an element that requires further research to determine whether or not it is significant and whether or not any measures (i.e. visual cues) would help those who are not computer literate.

The interview brought up one very important category that had not been dealt with at all and that was included elements, specifically audio. I had considered using audio for a variety of functions but it had not even crossed my mind to incorporate it into tutorial elements. This actually came up when I asked a question about using audio in the CDROM for elements like music and button clicks, fortunately this brought up a completely different idea to my interviewee

The size of the focus group drastically limits the value of the data acquired from them and further student testing of the prototype is necessary to get at some of the elements that were not well addressed. Issues of interface were extremely important to this study but the size of the focus group and perhaps my inability to adequately define the term made this an issue that was barely dealt with at all. Certain resolution changes will need to be tested on other groups to see if some of these problems can be resolved. Perhaps most importantly active tutorials need to be included and these need to contain multiple tracks of audio. Future research could utilize open-ended questionnaires or perhaps some kind of tangible benefits could be incorporated to try and improve the response rate of students and increase the validity of the findings.

References Allen-Sommerville, L. (1996). Capitalizing on diversity. The Science Teacher, 63(2), 20-23.

Baird, R. N., McDonald, D., Pittman, R. K., & Turnbull, A. T. The graphics of communication: Methods, media, and technology (6th ed.). Fort Worth TX: Harcourt Brace Jovanich.

Caldwell, G. P., & Ginther, D. W. (1996). Differences in learning styles of low socioeconomic status for low and high achievers. Education, 117(1), 141-147.

Cannell, M. (1997). Are you switched on? . People Management, 3, 23.

Dunn, R.(1988). Teaching Students through their perceptual strengths [commentary]. Journal of Reading, 31(4), 304-309.

Filipczak, B. (1995). Different strokes: learning styles in the classroom. Training, 32, 43-48.

Finley, M. (1996). What's your techno type - and why should you care?. Personnel Journal, 75(1), 107-109.

Given, B. K. (1997). How to deal with difficult principals: A learning styles approach [Special Section: Teacher Leadership]. Clearing House, 70(5), 257-260.

Hequet, M. (1997). How does multimedia change training?. Training, 34, A20-A22.

James, D. C. S., Rienzo, B. A., & Frazee, C. Using focus groups to develop a nutrition education video for high school students. Journal of School Health, 67(9), 376-380.

Kolb, D.A. (1985). The Learning Style Inventory: self-scoring inventory and interpretation booklet. Boston: McBer.

Kristof, R., & Satran, A. (1995). Interactivity by design: Creating & communicating with new media. Mountain View, CA: Adobe Press.

Kulik, J. A., Kulik, C. C., & Cohen, P. A. (1980). Effectiveness of computer-based college teaching: A meta-analysis of findings. Review of Educational Research, 50(4), 525-544. Latham, A. S. (1997). Responding to cultural learning styles. Educational Leadership, 54(7), 88-89.

Lindloff, T. R. (1995). Eliciting Experience: Interviews. In Qualitative Communication Research Methods (pp. 163 – 196). Newbury Park, CA: Sage.

Loo, R. (1996). Construct validity and classification stability of the revised learning style inventory (LSI-1985). Educational and Psychological Measurement, 56, 529-536.

Mathews, D. B. (1996). An investigation of learning styles and perceived academic achievement for high school students. The Clearing House, 69(4), 249-254.

McCarthy, B. (1997). A tale of four learners: 4MAT's learning styles. Educational Leadership, 54(6), 46-51.

Peterson, N. K., & Orde, B. J. (1995). Implementing multimedia in the middle school curriculum: Pros, cons and lessons learned. T.H.E. Journal, 22(7), 70-75.

Rand, A. (1996). Technology transforms training. HR Focus, 73, 11-13.

Schofield, J. W., Eurich-Fulcer, R., & Britt, C. L. (1992). Teachers, Computer Tutors, and Teaching: The Artificially Intelligent Tutor as an Agent for Classroom Change. American Educational Research Journal, 31(3), 579-607.

Strauss, A., & Corbin, J. (1990). Open Coding. In Basics of Qualitative Research (pp. 61- 74). Newbury Park, CA: Sage.

Taylor, J. (1995). Learning Styles - a practical tool for improved communications. Supervision, 56, 6-7.

Wise, M., & Groom, F. M. (1996). The effects of enriching classroom learning with the systematic employment of multimedia. Education, 117(1), 61-69.