Information Technology and Student s Achievement: Evidence from a Randomized Experiment in Ecuador

Transcripción

1 Information Technology and Student s Achievement: Evidence from a Randomized Experiment in Ecuador Research Proposal for IADB - RES Research Center Stratega Ecuador Principal Investigators Paul Carrillo George Washington University Juan Ponce Facultad Latinoamericana de Ciencias Sociales - Ecuador Date: February 23, 2009 Improving education quality is a priority for most developing countries. Policy makers usually agree that such improvements could lead to structural shifts in productivity and boost long term economic growth. Governments face the challenge to identify efficient ways to use their scarce resources and raise the quality of education. In this paper, we propose to investigate the effects of Information and Communication Technologies (ICT) in the school environment on education quality. To quantify these effects, we propose to use an experimental design to evaluate the impact of a project run by the municipality of Guayaquil-Ecuador that provides ICT to secondary schools. 1

2 The use of ICT in the school environment can increase students human capital in at least the following two ways. First, exposure to ICT such as computers and other related equipment (printers, projectors, etc.) may provide students with valuable skills that are generally rewarded in the labor market. 1 Second, the use of ICT for learning purposes could shift student s outcomes (achievement) given the same level of other inputs (teachers and infrastructure). 2 Higher achievements should be reflected on better test scores but also on other outcome variables that measure the participation of young individuals on the civil society. For example, the use of the Internet for learning purposes could boost test scores but, in addition, increase students awareness about important national issues. Increased achievements should thus raise the individual empowerment with respect to both the government and the private sector. 3 Our proposed research is closely related to Barrow at al. (2009) and Banerjee et al. (2005). Barrow at al. (2009) analyze the effect of an instructional computer program for pre-algebra and algebra on student s test scores in the US. Their results suggest that computer-aided instruction has a positive effect on test scores (about 0.17 of a standard deviation). Similarly, Banerjee et al. (2005) suggest that computer-assisted mathematics instruction raised math scores of fourth-grade students in Vadodara, India. Our proposed study differentiates from this previous work in at least two important ways. First, our 1 See for exmple, Krueger (1993), Autor, Katz and Krueger (1997), and Entorf and Kramarz (1997). 2 Several studies have analyzed the effects of computer technology in the classroom. For example, some analyze the impact of subsidies to invest in computer technology (Angrist and Lavy, 2002, Goolsbee and Guryan, 2006, and Machin, McNally and Silva, 2007). Others provide direct evidence on the effectiveness of computer technology as an input in the education production function providing evidence of existing correlations (Wenglinsky 1998) or results from randomized evaluations (Barrow et al., 2009, Banerjee et al., 2005, Rouse and Krueger, 2004, and Ragosta et al., 1982). Barrow et al., (2009) and Banerjee et al., (2005) provide credible evidence that the effects of ICT use on test scores are positive. 3 This example illustrates how the use of ICT could empower individuals with respect to the private sector. Consider a student who uses ICT both to learn mathematics and to surf the web. Knowledge of mathematics and access to information allows him/her to understand, for example, that financial products based on Ponzi scheme s (piramides financieras) are a scam. 2

3 study focuses on young adults who, presumably, in addition to improved test scores may obtain other important effects from exposure to ICT in the classrooms. For instance, exposure to technology could affect outcomes of the labor market (labor force participation and wages) and participation in the civil society. Second, we aim to identify the effect of Internet access on outcomes of this segment of the population. For young adults, exposure to the Internet may boost their access to information and empowerment levels. To measure the impact of ICT in student s achievements we focus on a project financed by the city of Guayaquil in Ecuador. The program is called Más Tecnología Colegios and is scheduled to begin on May It targets 125 municipal secondary schools (grades 10 th to 12 th ) and will be managed by the NGO E-ducate. 5 The program has three main components. First, basic infrastructure for computer labs will be provided. Labs are expected to include four computers per school. Depending on budget availability, computer labs in some schools will be granted access to the Internet. Second, the program will install in all computer labs software specifically designed to facilitate student s learning of language and mathematics. The software is called APCI. 6 APCI personalizes the curriculum of each student based on the results of an initial evaluation test, and students are expected to use the software at least three hours per week. Finally, the third component is a comprehensive plan of teacher s training. The training includes general computer lessons as well as training to use the APCI platform. With the proper instruction, teachers should be able to track the academic progress of each student. Más 4 In Guayaquil, the academic school year runs from April to December. 5 Information about E-ducate can be found at 6 APCI stands for PERSONALIZED COMPLEMENTARY AND INTERCONNECTED LEARNING system. Details about APCI can be found at Educate s website 3

4 Tecnología Colegios is not a new initiative of Guayaquil s local government but rather an extension of an earlier program (Más Tecnología) that has successfully provided ICT services (the same three components described above) to more than 400 primary public schools since We propose to measure the impact of the Más Tecnología Colegios program on student s achievement. Measuring the effects of a public program on beneficiary s outcomes is challenging for well-known reasons. 8 It is also well-established that the ideal setup to evaluate the effect of a policy (such as the ICT program) on an outcome of interest is a randomized experiment. The implementation of random experiments to evaluate policy questions, however, is not always feasible. Data requirements, cost constraints and ethical concerns limit the implementation of this methodology. For impact evaluation of the Más Tecnología Colegios program, we have found three conditions that facilitate the use of an experimental design. First, the program has not been implemented yet. Thus, a base-line survey among a representative sample of the 125 selected schools in April 2009 can be performed. 9 This survey will include information about the student, its household, scores on standardized tests for language and mathematics, labor market outcomes and other measures of achievement such as knowledge of the political environment. 10 Second, due to municipal budget constraints 7 Details about the original program can be found at The initial program Más Tecnología was also financed by the Municipality and administered by E-ducate. Más Tecnología Colegios has exactly the same components of the initial program. The only difference is that it targets students of higher grades. 8 The RES call for proposals discusses some of these points. For the sake of brevity we omit further discussion. 9 The survey will be implemented in 40 schools (about 50 students per school). Thus, the total sample of students in both control and treatment groups is close to 20, To gather demographic data about the student and the household, the survey will include the same set of questions (and structure) as standard questionnaires of living standards surveys. An additional module will be developed to measure test scores and ask questions about current economic and political events. 4

5 there are not enough resources to implement the program in all schools in the current year. It is expected that, out of the 125 schools, 43 will receive the program in 2009, 41 in 2010, and the remainder in This provides an opportunity to compare outcomes between treated and non-treated students in a follow-up survey in April Finally, and perhaps more importantly, E-ducate can choose the set of schools that will receive the program in the upcoming year at its own discretion and has agreed to follow our recommendations and provide treatment to a list of 43 schools that will be randomly selected. 11 These three conditions provide us with an unusual opportunity to evaluate the effects of an ICT program on student s achievement using an experimental design. To conduct this study, we have gathered a solid research team. The project will be lead by Paul Carrillo, Assistant Professor of Economics in George Washington University, and Juan Ponce, Professor of Economics and Academic Director of the Facultad Latinoamericana de Ciencias Sociales FLACSO Ecuador. Both researchers have written on applied microeconomic topics and published in peer-reviewed academic journals. 12 The research center Stratega will coordinate research assistance and other administrative sides of the project. 13 To complete our research project, we estimate that $34,800 are required (details are provided in Annex 1). Finally, there is another practical consideration that makes our proposal highly competitive. To evaluate the impact of the Más Tecnología Colegios program, E-ducate will provide funding to conduct both surveys. They are enthusiastic to follow our 11 We would also like to randomize Internet access among the subset of schools that receive treatment. In certain schools, however, Internet access is unavailable (due to network constraints) and a true random assignment may not be possible. 12 Please find their CVs attached to this proposal. 13 Stratega and E-ducate are strategic partners. 5

6 recommendations and implement the experimental design to have a sound impact evaluation of this program. References Angrist, J. and V. Lavy, (2002), New Evidence on Classroom Computers and Pupil Learning, The Economic Journal, 112, Autor D., L. Katz, and A. Krueger, (1997), Computing Inequality: Have Computers Changed the Labor Market?, National Bureau of Economic Research, Working Paper Banerjee, A., S. Cole, E. Duflo, and L. Linden, (2007), Remedying Education: Evidence from Two Randomized Experiments in India, Quarterly Journal of Economics, 122, Barrow, L., L. Markman, and C. Rouse, (2009), Technology's Edge: The Educational Benefits of Computer-Aided Instruction, The American Economic Journal: Economic Policy, (1), Entorf H. and F. Kramaz, (1997), Does Unmeasured Ability Explain the Higher Wages of New Technology Workers?, European Economic Review, (41), Goolsbee, A. and J. Guryan, (2006), The Impact of Internet Subsidies in Public Schools, TheReview of Economics and Statistics, 88, Krueger, A., (1993), How Computers Have Changed the Wage Structure: Evidence form Microdata, , Quartely Journal of Economics. CVIII, p Machin, S., S. McNally, and O. Silva, (2007), New Technology in Schools: Is There a Payoff?, Economic Journal, 117, Ragosta, M. Computer-Assisted Instruction and Compensatory Education: The ETS/LAUSD Study Final Report, Project Report 19, Princeton, NJ: Educational Testing Service, Rouse, C. E., and A. Krueger, (2004), Putting Computerized Instruction to the Test: A Randomized Evaluation of a Scientifically-based Reading Program, Economics of Education Review 23, 4, Wenglinsky, H., (1998), Does it Compute? The Relationship Between Educational Technology and Student Achievement in Mathematics, Princeton, NJ: Policy Information Center, Research Division, Educational Testing Service. 6

7 Annex 1: Details About Project s Costs $ / Month Months Total Principal investigator # 1 4, ,000 Principal investigator # 2 4, ,000 Research assistant # 1 1, ,000 Research assistant # 1 1, ,000 Survey coordinator 1, ,000 Other administrative costs ,800 Travel funds for field visits 3,000 Surveys (funded by E-ducate) 0 Total 34,800 Notes: We expect that each principal investigator will devote approximately 2.5 months of full time work (50 days) to complete this project. Research assistance will be needed for both data handling and to make an exhaustive literature review. In addition, research assistants are expected to provide details about the implementation of the program. We expect to hire each research assistant for about 3 months. This project requires the hiring of a survey coordinator. This person will oversee the correct execution of the surveys and work full time for 2 months. Both lead investigators plan to visit schools before the base-line and follow-up surveys are implemented. Thus, travel funds for transportation to Guayaquil are required. Other administrative costs include printing and other communication costs. The full costs of the surveys will be financed by E-ducate. 7

8 Paul E. Carrillo Department of Economics, George Washington University 2115 G Street N.W. Suite 364 Washington, DC Phone : Fax: pcarrill@gwu.edu Web: Education University of Virginia, Ph.D. in Economics, 2006 University of Virginia, M.A. in Economics, 2002 Pontificia Universidad Católica del Ecuador, B.S. in Economics, 1998 Current Position Assistant Professor of Economics, George Washington University, August 2006 Present Research Interests Applied Microeconomics and Econometrics, Urban and Real Estate Economics, Empirical Applications of Equilibrium Search Models Publications Efficient Delivery of Subsidies to the Poor: Improving the Design of a Cash Transfer Program in Ecuador, with Juan Ponce, Journal of Development Economics, forthcoming. Stay Public or Go Private?: A comparative analysis of water services between Quito and Guayaquil?, with Orazio Bellettini and Elizabeth Coombs, Chapter 5 in Privatization for the Public Good? Welfare Effects of Private Intervention in Latin America, 2008, edited by Alberto Chong, David Rockerfeller Center for Latin American Studies, Harvard University, and Inter American Development Bank. The Effects of Different Types of Housing Assistance on Earnings and Employment, with Edgar Olsen, Catherine Tyler, and Jonathan King, 2005, Cityscape: A Journal of Policy Development and Research 8(2): Explaining Attrition in the Housing Voucher Program, with Edgar Olsen and Scott Davis, 2005, Cityscape: A Journal of Policy Development and Research 8(2): Las Diferencias Salariales Entre el Sector Público y Privado en el Ecuador, Banco Central del Ecuador, Cuestiones Económicas, 20(2), 2004, La Cuenta Satélite del Sector Petrolero Ecuatoriano, Banco Central del Ecuador, Cuadernos de Trabajo No 122, Quito, Working Papers An Empirical Two-sided Equilibrium Search Model of the Real Estate Market, invited to revise and resubmit International Economic Review. Alternative Measures of Homeownership Gaps Across Segregated Neighborhoods, with Anthony Yezer, invited to revise and resubmit Regional Science and Urban Economics. 1 / 3

9 To Sell or not to Sell: List Price and Time on the Market in the Housing Market. A New Geographical Housing Price Index for All Areas in the United States, with Dirk Early and Edgar Olsen. Information and Real Estate Transactions: The Effects of Pictures and Virtual Tours on Home Sales. Determinants of Cross-State Variation in Social Security Disability Rates, with Naoko Akashi, Bruce Dembling, and Steven Stern. Work in Progress Test Scores, Home Values, and Information Disclosure, with Stephanie Cellini and Richard Green. Zoning and Subway Stations, with Richard Green. How Desperate are Desperate Sellers?: Asking Price Changes and Marketing Outcomes. Chicken or Egg: The Effects of Foreclosures on Home Prices. Public Signals, Private Information, and Household Behavior: Evidence from a Natural Experiment, with Shahe Emran Teaching Assistant Professor of Economics, George Washington University, Intermediate Microeconomics Spring 2007, 2008 Urban and Regional Economics Fall 2006, 2007, 2008 Adjunct Assistant Professor, Facultad Latinoamericana de Ciencias Sociales FLACSO, Quito - Ecuador Graduate Econometrics Spring 2006 Adjunct Assistant Professor, Universidad San Francisco de Quito, Quito - Ecuador Introductory Microeconomics Fall 2005 Instructor, University of Virginia, Department of Economics Introductory Statistics Spring 2004 Teaching Assistant, University of Virginia, Department of Economics Graduate Econometrics I Fall 2001, Fall 2002 Graduate Econometrics II Spring 2002, Spring 2003 Principles of Microeconomics Fall 2000 Other Professional Experience Research Economist, Central Bank of Ecuador, August 2004 July / 3

10 Research Assistant for Dr. Steven Stern, Quantitative Health Care Solutions, March 2002-August 2003 Consultant, United Nations Children s Fund (UNICEF), Quito-Ecuador, April June 2000 Analyst, Central Bank of Ecuador, July 1998-March 2000 Awards University Facilitating Fund Award, George Washington University, GWIPP Policy Research Scholar, George Washington University, Bankard Pre-Doctoral Fellowship, University of Virginia, July 2003 Dupont Fellowship, University of Virginia, August 2001-May 2003 Doctoral Fellowship, Banco Central del Ecuador, August 2000-May 2003 Best academic performance, Department of Economics, Pontificia Universidad Católica del Ecuador, Fall 1996 Presentations George Washington University, Department of Economics, October 2008 American University, Department of Economics, October 2008 University of Maryland, Department of Economics, October 2008 AREUEA Mid-year Conference, May 2008 George Washington University, Department of Economics, April 2008 ASSA Meetings of AREUEA, New Orleans, January 2008 Facultad Latinoamericana de Ciencias Sociales, Quito, Ecuador, November 2007 AREUEA Mid-year Conference, May 2007 George Washington University, Urban Seminar, May 2007 Inter-American Development Bank, Research Department, May 2007 George Washington University, Department of Economics, March 2007 International Atlantic Economic Conference, March 2007 Indiana University, Econometrics Seminar, Department of Economics, March 2007 Inter-American Development Bank, Research Department, October 2006 Inter-American Development Bank, Research Department, February 2006 George Washington University, Department of Economics, February 2006 University of Cincinnati, Department of Economics, February 2006 Queens University, Department of Economics, January 2006 The New York Fed, Research Department, February 2006 University of Virginia, Department of Economics, October 2005 Ifo Institute, Kiel-Munich Workshop on the Economics of Information and Network Industries, Munich, Germany, August 2005 The Society for Computational Economics, 11th International Conference on Computing in Economics and Finance, Washington D.C., June 2005 Other information Languages: Spanish, English, German (basic) Citizenship: Ecuador / US permanent resident Last Updated: Jan / 3

11 Juan Ponce Jarrín Hoja de vida Título académico más alto obtenido Título Universidad Año Mención Institute of Social Phd Studies. The Development 2008 Hague. The Studies Netherlands Reconocimientos académicos internacionales Premios y honores Fecha Institución y lugar Premi I Ajuts a la Investigació Becas y afiliaciones (scholarships) Experiencia docente Antigüedad en la FLACSO, a tiempo completo y con dedicación exclusiva Profesor Asociado Profesor Investigador Coordinador Programa Experiencia en postgrado a tiempo completo, fuera de la FLACSO Abril 30/2004 Fecha Fundació Càtedra Iberoamericana. Universitat de les Illes Balears. España Institución y lugar Tipo de reconocimiento Premio Investigación Tipo de reconocimiento Fecha de ejecución Programa Fecha de ingreso Cargo actual Fecha de inicio Economía Economía Economía Enero 2000 Enero 2004 Enero 2007 Subdirector Académico Universidad Facultad Período Agosto 2008 Seminarios de postgrado dictados fuera de la FLACSO (de una duración superior a 30 horas) Experiencia de docencia en pregrado a tiempo completo, antes de ingresar a la FLACSO Universidad Facultad Título Período Universidad Facultad Período 1

12 Publicaciones académicas Libros de autoría de edición nacional (1) Fecha Título Editorial(es) y lugar de publicación 2001 Empleo y Economía ILDIS y Abya-Yala del Trabajo en el Ecuador 2004 Informe de Secretaría Técnica Desarrollo Social del Frente Social y 2008 Segundo Informe Nacional de los Objetivos de Desarrollo del Milenio en el Ecuador Libros de autoría de edición o coedición internacional (1) 2008 Education Policy and Performance. Evaluating the Impact of Targeted Education Programs in Ecuador 2006 Más allá de los promedios: Afrodescendientes en América Latina Desarrollo social y económico de la amazonía ecuatoriana basado en el ecoturismo: emprendimeintos populares como alternativa a un desarrollo excluyente (1) Obra realizada por un autor o en coautoría con una sola persona. SIISE PNUD- SENPLADES- FLACSO Shaker Publishing. The Netherlands The World Bank Fundación Cátedra Iberoamericana de la Universidad de las Islas Baleares. España. Páginas Libro editado o dirigido de edición nacional (2) Libro editado o dirigido de edición o coedición internacional (2) Libros compilados nacionales o internacionales (3) (2) Incluye un estudio introductorio y una selección de textos originales reunidos alrededor de un proyecto editorial. (3) Incluye una selección de textos originales alrededor de un evento (foro o taller) y las antologías de textos. Capítulos publicados en libros de edición nacional (4) Fecha Título y título del libro 1997 La educación: a la espera de la equidad y la calidad. En Pobreza y capital humano en el Ecuador 2 Editorial(es) y lugar de publicación Secretaría Técnica del Frente Social Páginas

13 1998 A educación en el Ecuador: problemas y propuestas de solución. Una tipología de cantones para la intervención. Cuaderno de Trabajo No Ecuador: Tipología de la Intervención Social. Cuadernos de Trabajo 1999 La eficiencia interna del sistema educativo en el Ecuador: un análisis de cohorte. Cuadernos de Trabajo 2001 Propuesta de indicadores sociales para el centro histórico de Quito. Serie Foro, Desarrollo Cultural y Gestión en Centro Históricos 2003 Cómo hacer más eficiente el gasto en educación en el Ecuador: un análisis de los determinantes de la matrícula primaria y secundaria. Cuadernos de trabajo 2003 Quién se beneficia del gasto social en el Ecuador. Cuadernos de trabajo 2003 Los efectos indirectos del gasto social en Ecuador. Cuadernos de trabajo 2004 Household proxy means in Ecuador: a trade off between targeting and poverty reduction gains 2005 Mujeres y Educación. En Mujeres Ecuatorianas entre las crisis y las oportunidades El tratado de libre comercio y el desarrollo humano. SEDES SEDEH-SIISE SIISE SIISE SIISE SIISE Mimeo FLACSO ILDIS-FLACSO 3

14 En TLC más que un tratado de libre comercio 2006 Políticas sociales y programas de transferencia monetaria condicionada en América Latina. En Cohesión Social 2006 The Impact of Decentralization of Student s Cognitive Achievements. The case of Redes Amigas of Ecuador. Documento de Trabajo. Parte de mi disertación doctoral 2006 Notas para la discusión. Estrategia Nacional de Desarrollo: HaCIA UN Acuerdo para alcanzar los ODM en Ecuador 2006 Notas para la discusión. Estrategia Nacional de Desarrollo: Indicadores y disparidades cantonales. Ministerio del Trabajo, NNUU, UNESCO-ILDIS FLACSO-Ecuador CISMIL CISMIL Capítulos publicados en libros de edición o coedición internacional (4) 2002 Determinants of The Hague. ISS secondary enrollment in Ecuador: some policy simulations 2004 Ecuador Creating Fiscal Space for Poverty Reduction. Chapter 3: Education BID y Banco Mundial (4) A exclusión de todo capítulo en un libro editado o compilado por la misma persona. Artículos publicados en revistas arbitradas e indexadas de nivel A (5) Fecha Título y nombre de la revista 2008 The Impact of Cash Tranfers on School Enrollment. Policy Research Working Paper Efficiency Delivery of Cash Transfers to the Poor: Improving the design of a 4 Editorial(es) y lugar de publicación The World Bank Journal of Development Economics. Revista JCR Páginas

15 Conditional Cash Transfer Program in Ecuador Remittances for Development? A case study of the Impact of Remittances on Human Development in Ecuador Ex_ante simulations of Direct and Indirect Effects of Welfare Reforms Meeting the Millennium Development Goals in Education: a cost-effectiveness analysis for Ecuador. ISS Working Paper Series No Simulating progressive social transfers: gas subsidies and solidarity bonds in Ecuador. Discussion Paper Series. IZA. DP No The Impact of a Cash Transfer Program on Cognitive Achievement. The Bono de Desarrollo Humano of Ecuador. Documento de trabajo. Artículos publicados en revistas arbitradas e indexadas de nivel B (5) Journal of Development Studies. Revista JCR Review of Income and Wealth. Series 53. Number 4. December Revista JCR. Journal of Development Studies Revista JCR ISS working paper No. 393 Economics of Education Review (5) Revistas indexadas en el Journal Citation Reports (JCR) o en las bases Ebsco, Blackwell, ProQuest y JStor. (6) Revistas indexadas en el catálogo Latindex, CLACSO, HAPI, CIAO o índices equivalentes. 5