We actually don't think assigning one number or grade to a school is a great practice. Schools are complex and multi-faceted ecosystems that cannot be encapsulated by one number. One number cannot capture all that makes a child love her school. There are no good or bad schools. Schools have strengths and weaknesses.
However, there is a ubiquitous rating system that IS assigning one number to schools, and unfortunately, these scores are everywhere and they influence the decisions of parents. But not only that, this system is assigning points in a way that is hurting communities across the nation.
That's why we've developed a rating system that assigns numbers in a more fair and equitable way that exposes more truth about school quality. Test scores are one minor detail in the vast ecosystem of a school, and it is absolutely not the only thing to consider. But if you are going to consider test scores, they have to be considered appropriately. Our method is a better way to consider test scores.
In the future, SchoolSparrow will move away from assigning one number to a school. We will incorporate all the data that is available including suspensions, survey data, growth data, etc. Eventually, parents will be able to customize what is important to their family when it comes to assessing school quality. For example, one parent might focus on attendance and graduation rates for a high school, while another might focus on growth and suspensions for a middle school. SchoolSparrow will rank schools differently for every parent depending on the customized relative weights of these factors. And if a parent wants to consider test scores, they will always be displayed in the context of parent-income using our methodology.
Public schools that have higher populations of students with Limited English Proficiency (LEP) and/or Children With Disabilities (CWD) are likely to have lower overall test score averages because these populations tend to have lower scores. In order to have a substantive comparison of test scores between schools, these test-takers have to be accounted for.
We take the reported overall Reading/Language Arts (RLA) score for the each school, and we solve for the percentage of NON-LEP/CWD students that are deemed proficient on the RLA section of the standardized test.
For example, let’s say a school has a reported overall RLA score of 50. That means 50% of the kids at this school are deemed proficient on the RLA section of the test for that year. The school reports that 340 students took the test, 50 LEP students took the test with a score of 25% proficient, and 34 CWD students took the test with a score of LE10 (less than or equal to 10% proficient). Notice that we publish the raw data as reported by the Department of Education for every school in the country on our website.
We assume that the % of LEP students that are also CWD is the same for the overall school. %CWD = 34 CWD students / 340 Total Students = 10%. Then we apply this % to reduce the number of LEP students so we aren't double counting. 50 LEP students x (1 - 0.10) = 45 LEP Students. The total number of Non-LEP/CWD test takers is then calculated as 340 total - 45 LEP - 25 CWD = 270 Non LEP/CWD students took the test.
Now we need to find the # of non LEP/CWD students that were deemed proficient on the test. We start with the overall school, so of the 340 students that took the test, 50% were deemed proficient, which is 170 total students deemed proficient. The # of LEP students deemed proficient is 45 LEP students x 25% = 11.25 LEP students proficient. CWD: 25 CWD Students x 10% proficient = 2.5 CWD Students Proficient. The number of non LEP/CWD students that are proficient is calculated as 170 total proficient - 11.25 LEP proficient - 2.5 CWD proficient = 156.25 non LEP/CWD students proficient.
So, this school's reported test score on the RLA section of the test is adjusted up to reflect the % of non LEP/CWD students that were deemed proficient on the RLA section of the test: 156.25 / 270 = 57.9% proficient.
Clearly this method has some assumptions and there is a certain amount of error, but one thing is certain: making these assumptions is better than doing nothing which is what the other guys are doing.
Research has proven that test scores are 70-80% attributed to parent income, not school quality. This is why the existing ratings are biased towards schools where parents have high incomes, and they unfairly underrate quality schools where parents have diverse or low incomes.
We use data science to normalize test scores for parent income, and in this way, we expose more truth about how the school is influencing student performance. The graph below shows an example for the Boston Metro Area. For every public school in the Boston Metro Area, we plot the % of students who took the test and are deemed Economically Disadvantaged vs the schools overall RLA test score (adjusted for LEP/CWD students as described above). For comparison purposes, we have also shown the graph that results when using the unadjusted RLA performance scores as reported by the US Dept of Education.
The resulting scatterplot has a trendline that describes the average test score at every parent income level. Clearly there is a relationship: as income falls, so do the average test scores at each school. The correlation between these two factors can be measured by the R-squared of the trendline. An R-square of 0.2 means two things are correlated. The R-square of test performance vs parent income is very high at 0.72 for the Boston area. This mirrors research by Sean Reardon and others which concludes that test scores are 70-80% attributed to parent income.
We can also observe the difference between the adjusted and unadjusted scatterplots. The adjusted data moves the trendline up at every income level, but not in equal measures. At the extremes, the highest income schools move up about 18 points, while lowest income schools move up over 38 points! This tells us that the impact of LED/CWD students is critical across the board, but also that it impacts the scores of lower income schools to a much greater extent than higher income schools.
We call the trendline the average or expected score. On our system, schools are then scored by the extent to which they have a departure from the trendline. For example, a school near the lower right quadrant (low income, low test scores) that scores 5 points above the trendline will have the same score as a school near the upper left quadrant (high income, high test scores) that is also 5 points above the trendline.
Even though these two schools have wildly different scores, they both exceed expectations by the same amount, so they get the same score. Schools on the trendline are roughly a C, and as they move above the trendline their scores get better, and the scores go down as performance falls below the trendline.
Research has shown the students that are part of a diverse environment (if it's done right) get certain cognitive benefits that cannot be replicated at a homogenous school, such as better teamwork and decision making, more empathy, greater acceptance of others, possibly even higher IQ's. These are the kids we want leading the next generation.
Each school has a diversity score based on the racial make-up of the school. If a school has at least 2 races represented by 10% of the population, then the school scores a 10. If 3 races are represented by at least 10%, then the school scores a 20/40, all the way up to at least 3 races represented by at least 20% of the population is a 40/40.
The school ratings are bumped up according to their diversity score. Schools with a 10 go up one grade (C to C+). Schools with a 20 go up 2 grades (C to B-), and schools with a 30 or 40 go up 3 grades (C to B)
So how does this compare to what the dominant ratings site is doing in regards to Diversity? The other guys have what's called an equity score. This compares the test outcomes of kids that are considered economically disadvantaged with the rest of the school. If there is a large gap, the school gets dinged. This method unfairly dings urban schools again. So not only are the dominant ratings dinging schools where parents have diverse or low incomes through test scores, but also they are doing it again with the equity rating. There is an achievement gap between children from low vs high income families, and while it is important to provide supports to help close this gap, the fact that it exists is not necessarily the schools fault.
Consider the extremes: a wealthy family vs a family experiencing poverty.
The experience of the child in these homes from 0-5 years old is wildly different when it comes to education. The more privileged child is read to every night, hears more words, gets educational games, trips to the museum, likely has two college educated parents, education is a pillar in the value system of the home, also expensive daycares and/or preschools that actually have a curriculum, or one parent is more likely to be at home. In addition to a 5-year head start, the additional resources for tutors, test prep, etc. for the child of the wealthy family make the contrast even more stark. In the home of the child experiencing poverty, the need for stable food, water and shelter is the focus and education takes a back seat to these basic needs.
The point of the comparison is this: the gap in test performance is not the school's fault, yet school's where parents have diverse or low incomes are being underrated for one of the very things that makes the school great: it's incredible diversity.
On our rating system, an A+ is a 10/10, 9/10: A or A-, 8/10: B+ or B, 7/10: B- or C+, 6/10: C, 5/10: C- and so on.