by Molly Hanrahan, Monitoring & Evaluation Intern, and Merritt Frey, Executive Director

In this post, we explore this year’s DEF exam results and their fluctuation from year to year. In a companion post coming next week, we’ll discuss the wide variation in results among our schools on the DEF exam.

The DEF exam in Mali tests 9th grade students as they graduate from middle school. If students pass the test, they are eligible to continue on to higher education. If they fail, they must repeat 9th grade until they succeed. This reality combined with the fact that the DEF test results are the only quantitative measure we have of how much our students are learning means that it’s incredibly important to us to have strong pass rates.

We're pretty happy with this year's overall pass rate at our schools -- 59 percent. However, we're less happy with our ability to understand what drives that pass rate and, more importantly, pass rates at individual schools.

Frustratingly, our DEF pass rates fluctuate greatly from year to year. See the chart to the right to see this fluctuation in action. Although our overall trend shows improving DEF scores, each year sees quite a swing – ranging from changes of 3 to 21 percentage points over the years! This fluctuation often occurs at the individual school level as well. For example, one school went from a 75% pass rate last year to a 0% pass rate this year. Another increased from a 10% pass rate in 16-17 to a 98% pass rate in 17-18.

Even more dramatic swings are seen in the national DEF pass rate…so at least we know the problem isn’t just us! This makes us wonder if the problem isn’t outside of our schools – perhaps the DEF exam itself is dramatically easier or harder in different school years?

Looking closer to home in our schools, we analyzed DEF pass rates against some of our other data to see if we could find a pattern. So far, we have not yet found any strong correlation with class size, school age, location, etc. It does appear that there may be correlation between the number of textbooks available per student and DEF pass rates.  There is also a modest connection between school age and pass rates -- new schools (i.e., those 3 years old or  younger) tend to slowly ramp up their pass rates.

We’re doing some additional analysis to see if those correlations stand up to further testing. (More on that in next week’s blog post about variation in DEF pass rates among our schools.) However, any correlation should point more toward variation among the schools (i.e., a school with more textbooks would perform better) not fluctuation between one year and the next (i.e. if there is a correlation a school with more textbooks would likely do better each year, not do great one year and terrible the next).

While our pass rates fluctuate, we have many schools with amazing results this year. We are proud of our students who passed, and they’ve opened doors for themselves in the future—we just need to find out how to stabilize and increase the number of students experiencing that success.