A Symbol of hope vs the Hawthorne Effect
Via Joanne Jacobs: Philadelphia Inquirer 10/30/2006 Symbol of hope:
An elementary school in the heart of North Philadelphia has grabbed the national spotlight for dramatically improving its test scores.I double checked their results and overall they are amazing, but I did notice one thing.
The initial jumps were so large the Philadelphia school district verified them by having some students retake the tests.
Over four years, the number of fifth graders scoring at the highest level - advanced - soared from 1.2 percent in math and reading to 42.1 percent in math and 29.8 percent in reading.
In reading their results improved dramatically from a 12.2% proficiency rate in 2002 to a 72.5% in 2005, but all of the sudden in 2006 they dropped back down to a 47.5%. The same pattern is reflected in math scores.
I have noted this trend in several benchmark schools that I have looked up over my brief blogging career. Could it be that the initial rises in performance are a result of the "Hawthorne effect", and that once a system becomes established there is a leveling out of performance? Unfortunately the article doesn't mention the drop off at all. Perhaps its a statistical blip, maybe there were some changes not mentioned in the article, but its something that I might keep an eye out for in the future.
Update: I found a similar but not quite as dramatic drop off in scores at Amistad Academy of Achievement First.
2006 scores weren't available, but there was a drop off in 2005 scores from 2004 after several years of improvements.
Are there any studies that look at whether this is a common phenomenon?