Cross-posted from the blog. By Sarah D. Sparks
As the first grants issued under the federal Investing in Innovation, or i3, program wind down, we鈥檙e taking stock of what we鈥檝e learned from this politically popular program. Here at Inside School Research, I鈥檓 also exploring some of the quirkier results coming out of these grants. You can dive deeper into the program with an explanation of the Investing in Innovation structure and political legacy, a round-up of some of most recent i3 evaluations, and on-the-ground-looks at what鈥檚 changed in schools in Maine and North Carolina as a result of the program.
The Investing in Innovation development grants were billed as a way to quickly test ideas and interventions with a lot of potential, to find the ones worth expanding to other schools. But as schools in Forsyth, Ga., and Corona-Norco, Calif., discovered, educators can learn a lot even from programs that don鈥檛 become superstars under i3.
Forsyth and Corona-Norco each won in 2010. Forsyth schools won $4.74 million to implement a digital content and data-management system called EngageME P.L.E.A.S.E., while Corona-Norco won $5 million to implement an online student writing program called But both programs ran into problems during implementation and by the end of the grant, neither program had found evidence of significant benefits for students participating.
In that, the districts are in the same boat as many of the i3 grants, particularly those in the category of 鈥渄evelopment"鈥攖he smallest grants designed to test ideas with the least initial evidence. In an interview with 91制片厂视频 Week, Nadya Chinoy Dabby, the U.S. Department of 91制片厂视频鈥檚 assistant deputy secretary for innovation and improvement, estimated more than 1 in 4 i3 projects found significant results, higher than the average for venture capital projects, but still leaving many ideas on the cutting-room floor.
But that doesn鈥檛 mean the educators and researchers who participated didn鈥檛 find any benefit from making the attempt. Jason Naile, Forsyth鈥檚 project director, said the district 鈥渁bsolutely鈥� gained from the i3 planning and research process. 鈥淲e鈥檝e had a number of things as a result of the grant to accelerate what we are doing or bring in new things we wouldn鈥檛 have tried.鈥�
Implementation Issues Slowed Evaluation
Forsyth originally envisioned building EngageME from scratch as a data system to track students鈥� progress through a variety of academic and behavioral indicators, and match it to a comprehensive digital library that would suggest potential remedial or enrichment activities targeted at the student鈥檚 interests and learning style to provide 鈥渇lipped classroom鈥� options.
It was an ambitious task, and in the first year of development, the district had to drastically cut back and change vendors three times in the search for someone who could do the work. By the time the system was up and running, the district had only a half-year, rather than a year, of implementation time with the treatment group before adding the control group for the evaluation. 鈥淟ike anything else rolled out midyear, it can be difficult to get going,鈥� he said.
Looking back, Naile said said he wished the 91制片厂视频 Department had provided more guidance for small-scale grantees on how to change the plan in response to early problems. 鈥淲e had a change midstream or otherwise we would not have met the goals of the grant, and there wasn鈥檛 a lot of guidance on how to do that,鈥� he said. 鈥淣avigating [the change in vendors and scope] within a grant environment, with private sector and public school鈥� was challenging.
For other small grantees, Naile recommended getting involved in the networking community that has built up around i3, including discussion groups, meetings, and coaching calls. 鈥淢aybe if we had gotten involved in some of those activities earlier, some things would have gone smoother,鈥� he said.
Similarly, the Corona-Norco Unified school district planned a hybrid online system for giving students immediate feedback and support on their writing. The plan was to use the written part of state assessments to evaluate whether the program helped students, but in the third and fourth years of the grant, California suspended student testing as it switched over to tests alligned with the Common Core State Standards. The district was left with only baseline data to draw from in time for the grant evaluation.
That was a huge difficulty. The lack of data was one of the biggest issues, and it was out of our control,鈥� said Charla Capps, Corona-Norco鈥檚 director of educational services for curriculum and instruction. 鈥淭he high schools were telling us they were instantly able to tell which kids had come from treatment schools, because their writing was dramatically better. The soft data we had was amazing, the improvement teachers were seeing in writing was amazing鈥攂ut it was all anecdotal.鈥�
Schools Persevere Beyond i3
Both Corona-Norco and Forsyth continue to focus on building on the evidence they did glean.
Forsyth now has an individualized education plan for every student, and a digital content library tagged with metadata that allow teachers to search by the content standard, grade, description, and media type, as well as to provide ratings and feedback. For any given assignment, a teacher can get recommended content from the system for each of her students, and assign it to them with one mouse click. The library now has 15,000 items built by district teachers or collected from open resources online and vetted by the district. 鈥淎 lot of teachers want to use good content, but they don鈥檛 have time to search for it,鈥� Naile said. 鈥淲e鈥檙e hoping this lets them get it without having to go to page 35 of a Google search.鈥�
鈥淚n our grant, we focused on flipped classes and the technology in personalized student learning,鈥� Naile said. 鈥淣ow we are taking that back and seeing that 鈥榩ersonalized learning鈥� is such a buzzword right now, but there are so many things involved in personalized learning beyond technology.鈥�
For example, 鈥渢he personalized task button is being used, but not as much as we would like,鈥� Naile said. 鈥淭eachers don鈥檛 trust the content. For one assignment they could potentially have 100-200 pieces of content recommended to those kids, so they say, 鈥楬ow do I know it is meeting their needs?鈥欌��
Still, he said the platform developed for the grant has come into everyday use. In fact, during a series of snow days last year, the district continued to hold class, as teachers pulled together online classes from the digital library and assigned them to students at home.
Based on its school-level results from teachers, Corona-Norco decided the WriteUp! program was worth pursuing for elementary and middle school students, as well as for remediation for 9th graders. It has expanded WriteUp! to all of its treatment and control schools and all of its high poverty Title I schools, serving about 26,000 students in grades 4-8 out of 54,000 students in the district.
鈥淭he i3 project really helped us to be more thoughtful and more strategic in how we roll out technology throughout the district,鈥� said Barbara Wolfinbarger, Corona-Norco鈥檚 administrative director, who wrote its i3 grant in 2010. 鈥淎s we look at moving forward tech to implement [Smarter Balanced testing], our professional development model [created for WriteUp!] has served us well in classroom coaching, make sure the teachers know what we are planning and have some time to plan how to implement it.鈥�
And now that the state has settled into its new assessment, 鈥渨hen our students had to take the [Smarter Balanced] assessment, they were head and shoulders above the other kids because they had been writing to prompts [on WriteUp!] that ended up being a lot like those on the test,鈥� Capps said.
It鈥檚 worth it for districts to try to use i3 or other grants to pilot and study new programs, even if they don鈥檛 produce results within the grant period, Capps said. 鈥淓nsure you have multiple measures, because things will happen, ... and really document the growth that you are making, both the statistical and anecdotal,鈥� she said.