Noelle Rouxel-Cubberly Assistant Professor of French Valeria Belmonti Director, Modern Languages Media Center College of Staten Island City University of New York
CHALLENGES CUNY realities: large classes, demographics Requirement Passive learning habits TECHNOLOGY ASSETS Digital Natives Increased speaking time Formative assessment Our Goal: ACTFL Intermediate Low level of oral proficiency by the end of 3 rd semester. Our Measurement: Course-embedded embedded assessment of oral proficiency.
1500+ STUDENTS PER SEMESTER Class appointments setup at the beginning of the semester 15-30 minutes at the beginning or end of the lecture ( formative time ) 2-8 8 oral assignments per class per semester
Content of exam posted on online message boards Students recorded audio answers using Audacity Response file posted on the message boards
No mechanism to control test progression Grading procedure Instructors used external grading tools No systematic way to distribute and analyze results Mean score for Spanish 114 oral assessment was 86.1% Students could hear questions and record answers unlimited times No time limitations to answer questions
OWL Testing Software OWL Testing Software www.owlts.com
Section 1 Item Item Item Item Test Section 2 Item Item Section 3 Item Item Item
Within the test: Order sections by number Randomize sections Within each section: Order questions by number Randomize questions
Instructor decides if questions are displayed simultaneously or not within a section
Instructor decides if and audio/video question can be played back or not Instructor decides if students can change previous responses
RECORD TIME: Test level Section level Item level THINK TIME: No time to think about answer Unlimited time to think about answer Set amount of time to think about answer
Students retrieve grade online
Specific corrections: efficiency, personalization, input in TL
Overall comments: visualization, references, proficiency-oriented, independent learning.
Spring 2011 -Mean score for SPN 114 midterm oral exam 73.5%
Film and Literature Activities Video clips Comments Half-guided Monologues Situations Question to Answer Pronunciation
Arrivia casa a Brooklyn e a casa a Brooklyn e tiricordidiaver lasciatole chiavineltuoufficioin New Jersey
Mi universidad es muy grande y hay varios edificios. Hay dos gimnasios, uno con piscina. Hay una biblioteca grande. Hay varios laboratorios de lenguas y de las ciencias. También hay una librería donde los estudiantes compran los libros de texto y plumas y cuadernos. Qué hay en tu universidad?
Hola Cómo está? Me llamo Teresa. Soy de Oaxaca, México. Mi estación preferida del año es el otoño. Durante los meses de otoño no hace mucho calor, hace fresco. Algunas veces llueve, pe Hola Cómo está? Me llamo Teresa. Soy de Oaxaca, México. Mi estación preferida del año es el otoño. Durante los meses de otoño no hace mucho calor, hace fresco. Algunas veces llueve, pero no siempre. También me gusta el otoño porque en esta estación celebramos una de las fiestas más tradicionales de mi país: el día de muertos. El día de muertos se celebra el uno de noviembre. En esta fiesta celebramos a nuestros parientes muertos. Hay muchas decoraciones típicas de esta celebración, como el papel picado o las calacas y más importante: los altares. Los altares se montan en honor de los parientes muertos. En ellos ponemos decoraciones y también ofrendas de comida. Normalmente ponemos la comida favorita de nuestros parientes y también la comida típica de la celebración Algunas de las comidas típicas de esta fiesta son: el pan de muerto Y las calaveritas de azúcar. Si algún día viene usted a México durante el día de muertos, pida estas comidas típicas, cómalas y visite los altares. Estoy deseando conocerla. Escriba pronto. Hasta luego.
Vocabulary and grammar knowledge in the context of specific cultural backgrounds was evaluated in exam 1. Students were shown a picture randomly selected from 3 movies among the 6 analyzed in the course
Oral exam 2 assessed both specific and global understanding of the film selected by each student. Each student only saw a clip related to his/her film.
FACULTY Group workshops 1-1 meeting Video and paper tutorials Faculty discussion board STUDENTS PowerPoint online Sample OWL Test in English Handout at computer desk Proctors during exams Spring 2010 Pilot Project 4 classes Fall 2010 All French and Italian Courses Spanish courses taught by full timers Spring 2011 All Spanish courses Chinese pilot project
Multiple choice, True/False, Fill-in, in, Short Answer are self-graded by OWL CHINESE Computerized midterm and final exams More tasks assessed than in paper exams Minimize grading AMERICAN SIGN LANGUAGE Finger spelling Receptive skills PLACEMENT EXAMS Adaptive tests System-graded Customized to our programs
STUDENTS Effective training but some students prefer audio instructions to written instructions on test screen. OWL is simple to operate and very intuitive. For some students, timing positively forces to think fast in the target language. Most students would prefer no timing and ability to re-record answers. Instructor s feedback very useful. FACULTY Effective training but need to share ideas and practices among instructors. OWL is easy to use but: Too many steps required to modify grades Somewhat difficult to view grades Too many steps required to print reports Verbiage not always intuitive Concerns over CSI network performance. Feedback time consuming.
noelle.rouxelcubberly@csi.cuny.edu valeria.belmonti@csi.cuny.edu We thank our colleagues, Jerry Lombardo, Jill Jegerski, Lucas Marchante, Gerry Milligan, Sarah Pollack, Kathryn Talarico, Paola Ureni, Oswaldo Zavala for their contributions. We thank Susan Holak and our VP for Technology Systems, Dr. Michael Kress for his ongoing support!