Causal Effects of Dual-Language Immersion on Student Achievement
Leveraging randomized and quasi-experimental designs, this set of work examines causal effects of dual-language immersion education on student achievement in core subject areas. It also examines program costs and partner-language proficiency acquisition.
Effects of Dual-Language Immersion on Student Achievement in Portland, Oregon
Funder: Institute of Education Sciences, State & Local Education Programs & Policies Grant #R305E120003 Years: 2012-2016
Steele, J. L., Slater, R. O., Zamarro, G., Miller, T., Li, J., Burkhauser, S., & Bacon, M. (2017). The effects of dual-language immersion programs on student achievement: Evidence from lottery data. American Educational Research Journal, 54(1S), 282S-306S.
Using data from seven cohorts of language immersion lottery applicants in a large, urban school district, we estimate the causal effects of immersion programs on students’ test scores in reading, mathematics, and science and on English learners’ (EL) reclassification. We estimate positive intent-to-treat (ITT) effects on reading performance in fifth and eighth grades, ranging from 13% to 22% of a standard deviation, reflecting 7 to 9 months of learning. We find little benefit in terms of mathematics and science performance but also no detriment. By sixth and seventh grade, lottery winners’ probabilities of remaining classified as EL are 3 to 4 percentage points lower than those of their counterparts. This effect is stronger for ELs whose native language matches the partner language.
Article Preprint Online Appendix Bibtex RAND Research Brief
Steele, J. L., Slater, R. O., Li, J., Zamarro, G., Miller, T., & Bacon, M. (2018). Dual-language immersion education at scale: An analysis of program costs, mechanisms, and moderators. Educational Evaluation and Policy Analysis, 40(3), 420-445. doi: 10.3102/0162373718779457
Using input and outcome data from a randomized study of dual-language immersion programs in an urban district, we examine the mediating relationships of dosage, expenditures, and classroom characteristics to students’ academic performance, and the moderating role of students’ race/ethnicity. Differential costs of immersion were concentrated at the district level and were modest, at about 2% to 4% of per-pupil spending annually. We estimate that an additional US$100 spent per immersion student in a given year was associated with an additional 8% of a standard deviation in language arts performance in English, which was just over one third of the causal point-in-time enrollment effect of 22% of a standard deviation. We find no generalizable evidence of differential effects by race/ethnicity.
Article Preprint Bibtex Twitter summary
Burkhauser, S., Steele, J. L., Li, J., Slater, R. O., Bacon, M., & Miller, T. (2016). Partner-language learning trajectories in dual-language immersion: Evidence from an urban district. Foreign Language Annals, 49(3), 415-433.
Research has demonstrated that students in dual-language immersion programs perform as well as, or better than, their peers in core academic content areas by late elementary school. However, the extent to which immersion education fosters bilingualism has received less attention in the literature. Using data from a four-year efficacy study of dual-language immersion education in the Portland Public Schools in Oregon, this study reports the skill levels that 1,284 dual-language immersion students achieved in their classroom partner languages (Spanish, Japanese, and Mandarin Chinese) between third and eighth grades. The authors found that by eighth grade, the average dual-language immersion student, regardless of language, performed at least at the Intermediate Low sublevel, and often higher, on Standards-Based Measurement of Proficiency assessments of nearly all language skills tested (listening, reading, writing, and speaking). In comparison, eighth graders in the Portland Public Schools who began taking Spanish as an elective in upper elementary or middle school scored only at about the Novice Mid sublevel. After four years of immersion learning (grades K–3), fourth-grade students whose home language was Spanish scored similarly in reading and speaking to their immersion peers whose home language was not Spanish; however, they outperformed their immersion peers in listening and writing.
Li, J., Steele, J., Slater, R., Bacon, M., & Miller, T. (2016). Teaching practices and language use in two-way dual language immersion programs in a large public school district. International Multilingual Research Journal, 10(1), 31-43.
Many educators and policymakers look to two-way dual language immersion as one of the most promising options to close achievement gaps for English learners. However, the programs’ effectiveness depends on the quality of their implementation. This article reports on a large-scale study of the implementation of dual language immersion across a large, urban school district. Using classroom observations, we examined teaching practices and language use by teachers and students in dual language immersion classrooms across an entire school district. We found strong implementation of teaching practices consistent with sheltered instruction, and strong adherence by teachers to partner language use as prescribed by the district’s guidelines. This article provides a descriptive view of what is happening in two-way dual language immersion classrooms in a large, urban U.S. school district. While other studies may examine classroom practices, few have done so on this scale.
Article Preprint Bibtex RAND Research Brief
Anderson, K., Zamarro, G., Steele, J. L., Miller, T. (2021.) Comparing performance of methods to deal with differential attrition in randomized experimental evaluations. Evaluation Review, 45(1-2), 70-104.
This paper uses state and local data from the Portland dual-language immersion study to examine statistical methods for addressing differential attrition, which happens when randomized treatment- and control-group participants exit the sample at different rates. In randomized controlled trials, attrition rates often differ by treatment status, jeopardizing causal inference. Inverse probability weighting (Hirano et al, 2003; Busso et al., 2014) and estimation of treatment effect bounds (e.g. Lee, 2009; Angrist et al., 2006) have been used to adjust for this bias. We compare the performance of various methods within two samples, both generated through lottery-based randomization: one with considerable differential attrition and an augmented dataset with less problematic attrition. We assess the performance of various correction methods within the dataset with problematic attrition. In addition, we conduct simulation analyses. Within the more problematic dataset, we find the correction methods often performed poorly. Simulation analyses indicate that deviations from the underlying assumptions for bounding approaches (Angrist et al., 2006) damages the performance of estimated bounds. We recommend the verification of the underlying assumptions in attrition correction methods whenever possible and, when verification is not possible, using these methods with caution.
Article Preprint Bibtex Online Appendix
Effects of a Statewide Dual-Language Immersion Scale-Up in Utah
Funder: Institute of Education Sciences, Research Practice Partnership Grant #R305H170005 Years: 2017-2019
Steele, J. L., Watzinger-Tharp, J., Slater, R. O., Roberts, G., & Bowman, K. Achievement effects of dual language immersion in one-way and two-way programs: Evidence from a state scale-up in Utah. (Working Paper.) Washington, DC: American University and American Councils for International Education.
The rising U.S. demand for dual-language immersion (DLI) programs, which offer core instruction in two languages from early grands onward, has raised questions about program design and access. We leverage the rapid expansion of DLI schools across Utah to estimate effects of DLI program availability on the academic achievement of native English speakers and English learners (ELs) in programs that serve mainly the former (one-way) and those comprising one-to-two thirds of the latter (two-way). Adjusting for school fixed effects, cross-grade intent-to-treat estimates in one-way programs are largely null, but those in two-way programs reach 0.10-0.11 standard deviations in math and English and show higher EL reclassification rates by grade 5. Estimates suggest an advantage of cultural adjacency in program design.