|
March 15, 2003
Digging for Data
By Todd McIntire
No Child Left Behind's challenge: collect, analyze, and use information to improve student performance. Here, we offer three different approaches.
Follow the Leader
 |
| Project Achieve lets schools keep track of each student's mastery of specific standards |
Michael Lane, superintendent of the Virginia CUSD #64 in Virginia, Ill., is excited. "We now have the tools and capability to do the data-driven decision making required by the No Child Left Behind Act," he says. Lane's district is one of 75 participating in the first phase of Following the Leaders, a major initiative to support states as they respond to NCLB's requirements. Funded by the U.S. Department of Education and coordinated by the Education Leaders Council, the venture provides the technology and expertise necessary to ensure that standards and assessments are aligned at local, district, and state levels.
The project is based on the premise that to truly make an impact on individual student performance, parents, teachers, and administrators must have access to accurate, timely information. All too often, however, the data most needed to make informed decisions presents the biggest challenge to capture and analyze. Lane explains, "Prior to Following the Leaders, we relied on manual observation of test results. We would look at paper reports of class composite and individual scores. It was a time-consuming process. Now a teacher can go online and immediately find out where the class is as a whole or what each individual did."
Results from standardized tests can provide valuable insights, but these data offer only snapshots of student performance at a particular moment. Given the growing need to closely monitor student achievement, it is important that schools capture and use the daily flow of data that measures performance against state standards. Pearson Education Technologies, Plato Learning, Scantron, and many others offer management software for organizing such data. Following the Leaders schools are using the online Achieve 3.0 from Project Achieve to manage the volumes of data produced by students and teachers every day.
Teachers use the software's lesson planner component to organize the content, skills, and standards they will teach. As they plan a lesson, key concepts and skills are linked dynamically to the relevant state standards. The software then tracks how many times a standard has been taught, and individual student results are captured, linked to standards, and displayed in one place. At any time, teachers can see how a student is performing against any standard and then revise lesson planning, adjust pacing, and intervene before too much time is lost. In addition, schools in participating states have access to Project Achieve's Resource Center, a database of thousands of instructional activities. Lane believes this is a tremendous benefit to his teachers. "There are pools of lesson plans geared to each standard, so teachers can find resources to tailor their instruction to the needs of their students," he says.
One challenge Following the Leaders schools face is that many teachers are not initially eager to make the jump to a fully electronic planning and grading system. To ease these fears and build skill levels, project organizers arrange for teachers to have access to live consultants and online tutorials. "Teachers have been receptive because they see how much simpler it is to do standards-aligned lesson planning," says Lane. "It helps them find students' weaknesses and strengths." The result is instruction that is more targeted and efficient.
Virginia CUSD Strategies for Success
- The project can be used at several levels-local, district, and state-to align curriculum, instruction, and assessment with state standards.
- All student achievement data, including class assessments and standardized tests, are linked to the appropriate state standards and instantly available for analysis.
- A combination of on-site support and online tutorials support teachers as they adapt to new methods of planning, record keeping, and data analysis.
Building a District Data Warehouse
New York is taking on data-driven decision making in a big way. Each of the state's 12 Regional Information Centers-nonprofits that provide cooperative technology services to school districts-is offering data warehousing services to its constituent districts. A data warehouse brings together data from a variety of disparate information systems in a school district (e.g., attendance, grades, demographics, and test scores) to a single location and makes that data accessible for complex analysis and reporting. Maureen McCarthy, manager of data warehousing for the Lower Hudson Regional Information Center, explains, "We believe that by helping districts standardize and integrate student performance information, we will provide administrators, teachers, parents, and students with tools to improve student and school performance."
The RICs have adopted the eScholar data warehouse, a platform specifically designed to accommodate the data sources available in K-12 school districts. The RIC typically hosts the warehouse, provides technical support in project implementation, and offers professional development to school and district administrators on how to use the new tools.
Building a data warehouse is a daunting task for any organization. School districts are particularly challenged because of the wide variety of data sources used, from student demographics, grades, and standardized test scores, to staff certifications, attendance, and spending. In addition, this information is usually found on different computers in different databases with different data formats. As a result, before the data can enter the warehouse, it must be transformed into a common format. The idea is to "cleanse" and standardize the data so that it can be combined in new and different ways to answer questions and provide comparisons. In New York, the RICs use eScholar's analysis and data cleansing applications to complete this process.
Once the data is cleansed, it's stored in a relational database on a centralized RIC server. However, collecting the data in the warehouse is only the beginning. In order to get a return on their investment, districts need to know how to find information they can use to improve student achievement. For most teachers and administrators, this is an unfamiliar task. The Regional Information Centers help by developing reporting tools using products such as Cognos and offering professional development on how to make data-driven decisions. In the Lower Hudson region, the RIC uses a team approach. "We partnered a technical person with someone who knows the classroom curriculum," says McCarthy. "These teams work very effectively with principals and teachers to help them know both what the data says, and more importantly, what they can do with the information."
Results from this initiative are promising. In addition to identifying threshold students-those on the verge of the next performance level-and providing targeted interventions, districts are using the data warehouse to correlate class grades and individual teacher assessments with test results to find predictors of student performance. "Ultimately, the data is used to improve instruction and raise success rates," says McCarthy.
LHRIC
Strategies for Success
- The technically complex aspects of the project are centralized where top experts are available to provide support.
- Establishing technical standards allows for collective product development, professional development, and tech know-how.
- Developing tools that streamline the process of performing complex data analyses lets any teacher or administrator easily identify strengths and weaknesses.
Data Analysis at the local level
 |
| Renaissance Academy teachers use Scholastic Reading Inventory to gauge students' reading
abilities. |
In only its third year of operation, Renaissance Academy Charter School in Phoenixville, Pa., has seen standardized test scores rise every year. The school attributes some of this success to the expectation that every teacher become an expert in analyzing student performance data. Using traditional paper reports and computer-based systems, teachers and administrators at Renaissance dig through every element of student performance looking for details that they can use to improve student achievement.
The Renaissance staff spends significant time analyzing the results of once-a-year standardized tests. They break down annual test data by class, teacher, and content strand. They compare results by teacher and by same cohort of students (e.g., all second-graders in one year to all first-graders in previous years). But they do not stop there. They continue to analyze data throughout the school year using a combination of electronic and online systems.
Every eight weeks, students at Renaissance take the electronic version of the Scholastic Reading Inventory as part of the Success for All reading program. SRI is a comprehension test that assesses students' reading levels. Students take the SRI on a networked computer, the results are automatically recorded, and reports are generated for the teacher to review. Renaissance teachers use the SRI data to adjust instruction according to students' needs, track students' reading growth over time, and match readers to texts. Most notably, SRI results are used to reorganize students into reading groups that are most appropriate to their stage of development. The key is that Renaissance not only collects student performance data, but acts on the data that it collects by adjusting instruction.
In addition, Renaissance has a management agreement with Edison Schools, which provides its Benchmarks system to diagnose students' academic strengths and weaknesses. Each month Renaissance students in second grade and above take online tests in reading, language arts, and mathematics. The tests are composed of 15 to 25 multiple-choice questions formatted to match the high-stakes assessment of a particular state, in this case the Pennsylvania System of School Assessment. Every question is accompanied by a set of teaching notes that provide instructional strategies for the tested skills and objectives. Reporting forms give teachers and administrators immediate feedback on student-, class-, and school-level proficiency in particular strands and skills.
Renaissance teachers go beyond the ready-made reports available in the Benchmarks system. Each month they scrutinize the results to identify high- and low-performing students, content strands that need additional attention, and common mistakes and misunderstandings. Teachers are required to submit a monthly written review of their students' results along with a set of action steps for the next month. The result is a staff that has extraordinarily deep knowledge of their students' knowledge and skill levels. These insights lead to interventions that help students succeed at higher levels.
Finally, each month Renaissance's curriculum coordinators and achievement advisors complete online surveys on the implementation of their school's student achievement plan. Edison Schools combines the results of these surveys into a Monthly Achievement Profile, a comprehensive report that describes progress and calls attention to areas in need of additional support.
The bottom line: Renaissance is demonstrating great success as measured by standardized tests and parent and student feedback. A big reason for this is their commitment to knowing every detail of student performance and then acting on the information gathered.
Renaissance Academy Strategies for Success
- Student achievement data is collected and analyzed throughout the school year, not just in conjunction with annual high-stakes tests.
- Each teacher is responsible for analyzing student performance data and developing action plans for immediate response to identified needs.
- Sophisticated online assessment tools are used to ease the process of collecting data and reporting on student performance.
Todd McIntire is the director of achievement for Edison Schools.
Read other articles from the March Issue
Send a letter to the Editor in response to this article.
|
