Competency-Based and Adaptive Instruction

These studies examine implementation and effects of student-centered teaching interventions, including competency- or mastery-based learning and grading, as well as computer-based adaptive instruction that adjusts to meet students’ individual levels of skill.

Study of Competency-Based Education in Five States

Funder: Bill & Melinda Gates Foundation
Years: 2012-2014
Steele, J. L., Lewis, M. W., Santibañez, L., Faxon-Mills, S., Rudnick, M., Stecher, B. M., & Hamilton, L. S. (2014). Competency-based education in three pilot programs: Examining implementation and outcomes (RR-732-BMGF). RAND Corporation.
In 2011, the Bill & Melinda Gates Foundation created the Project Mastery grant program to support competency-based education initiatives in large school systems that serve a high proportion of disadvantaged youth. Competency-based education meets students where they are academically, provides students with opportunities for choice, and awards credit for evidence of learning, not for the time students spend studying a subject. The Foundation asked RAND to evaluate these efforts in terms of implementation, students’ experiences, and student performance. This report presents final results from that evaluation, offering a literature review and conceptual overview of the competency-based education movement and describing the implementation of competency-based features under each project. The report concludes with six lessons for policy, partnerships, and practice.
Key Findings: (1) Not All Competency-Based Programs Look Alike: Each Project Mastery grantee emphasized different aspects of competency-based education. Some placed greater emphasis on student choice and project-based learning, while others focused on flexible pacing and evaluation for proficiency. (2) Tensions Exist in Implementing the Approach: Sites faced challenges in determining how to provide credit for out-of-school activities, including afterschool activities and travel-based learning experiences. Sites experienced tension in holding all students to a common definition of proficiency and evaluating students strictly based on performance, not effort. Sites faced challenges in providing enough computer hardware so that students could use new competency-based curricula or in identifying funds that would permit expansion of new curricula beyond the pilot initiatives. Sites reported that students with weak academic backgrounds may require extra support under competency-based models. (3) Student Experiences Were More Similar Than Expected: Because the programmatic details of the Project Mastery sites were varied, one striking finding was the similarity of students’ self-reported experiences across sites. The highest student reports of engagement, flexible pacing, and choice came from a site in which respondents were focused on yearlong, self-directed projects that applied academic content to real-world contexts. (4) Student Performance Varied Across Sites: Effects of competency-based models on student learning appeared most positive in programs that put primary emphasis on student choice with project-based learning. However, the research design did not permit causal inference, and these findings could be at least partially due to students’ or teachers’ selection into the programs.
Monograph Bibtex RAND Research Brief

Evaluation of Cognitive Tutor Geometry

Funder: U.S. Department of Education Institute of Education Sciences Years: 2007-2010
Pane, J. F., McCaffrey, D. F., Slaughter, M. E., Steele, J. L., & Ikemoto, G. S. (2010). An experiment to evaluate the efficacy of Cognitive Tutor Geometry. Journal of Research on Educational Effectiveness, 3(3), 254–281.
This randomized, controlled field trial estimated the causal impact of a technology-based geometry curriculum on students’ geometry achievement, as well as their attitudes toward mathematics and technology. The curriculum combines learner-centered classroom pedagogy with individualized, computer-based student instruction. Conducted over a 3-year period in eight high schools within an urban fringe district, the study found that students assigned to the treatment curriculum scored 19% of a standard deviation lower on the geometry posttest than their counterparts assigned to the district’s standard curriculum, but found no statistically significant impact on students’ attitudes toward mathematics and technology. Researchers also collected observation and interview data on teachers’ instructional practices. These data suggest that many teachers had difficulty implementing the treatment curriculum’s learner-centered pedagogy. In fact, observed levels of learner-centered practices were only modestly higher in treatment classes than in control classes. In both treatment and control classes, however, higher levels of learner-centered pedagogy were associated with higher student achievement in geometry.
Article Preprint Bibtex

Ikemoto, G., Steele, J., & Pane, J. (2016). Poor implementation of learner-centered practices: A cautionary tale. Teachers College Record, 118(13), 1–34.
Many school systems are adopting new curricula in response to more rigorous standards that require higher-order thinking skills. This article presents implementation findings from a randomized, controlled trial of the Cognitive Tutor Geometry curriculum. We found a significant negative effect on student achievement despite the curriculum’s focus on learner-centered learning strategies that have previously been found to improve students’ ability to meet high mathematics standards. Our research confirms prior research that finds learner-centered instructional practices are correlated with higher student achievement. However, our findings also suggest that learner-centered curricula can actually do more harm than good when implemented poorly. We found that the cognitive demands of the curriculum coupled with teachers’ poor implementation of learner-centered instructional practices seemed to limit students’ ability to engage with the mathematical ideas. Teachers struggled to implement the curriculum because they lacked prior experience with learner-centered teaching strategies, had limited exposure to the curriculum, and were not provided with job-embedded support from principals or instructional leaders within their school. They also worked with students who were reluctant to collaborate and had low prior math achievement. Findings from this study suggest that curriculum adopters should be careful to ensure strong implementation of cognitively demanding curricula. In particular, districts and school leaders should provide intensive job-embedded professional development and support to assist teachers in achieving high implementation.
Article Bibtex