Preliminary Evaluation Results

 

Chancellor’s Summer Academy Preliminary Evaluation Results :

Report to Principals and Teachers

Katherine Ryan and Maurice Samuels

November, 2005

 

The Chancellor's Summer Academy (CSA) is a professional development program designed to enhance the professional development of teachers and other personnel from Champaign and Urbana School districts. The CSA university team and the districts worked together in Spring 2005 to design the first summer academy, which was held from July 25-August 5.  The 2005 Academy offered personnel in each participating school the opportunity to (1) learn about and focus on strategies intended to enhance language and literacy development of students in elementary grades 3-5 and middle school and (2) to develop a purposeful, teacher-directed plan for professional development around literacy for each school to be carried out over the 2005-2006 school year. The University will provide assistance in carrying out these professional development plans.

 To provide information about how to improve CSA, the Chancellor commissioned a formative evaluation. Three questions framed the evaluation: a) How effective are the sessions for helping teachers learn and apply new language and literacy enrichment strategies? b) How has the relationship between the University and the C-U school districts been improved as a result of the CSA? c) How can the CSA planning and implementation process be improved?             

To evaluate the CSA, we prepared and field tested the Teachers’ Literacy Practices Inventory, developed and administered CSA session questionnaires, observed sessions, conducted focus groups and semi-structured interviews with CSA teachers, C-U school district administrators, and the UIUC CSA team. (See Table 1 for descriptions of data sources and instruments.) A summary of preliminary findings is presented below.

Teachers’ Literacy Practices prior to CSA Participation

The Teachers’ Literacy Practices Inventory was administered to teachers (N=33) to assess how they were engaging in teaching literacy before participating in the CSA.

Content Area Reading

Teachers frequently use several important instructional strategies when teaching literacy such as a) teaching students to use informational text structures to support student learning (4.3) and b) using anticipation strategies to help students build meaning as they read. (4.3) However, teachers report there are some strategies or resources they do not use as often.

• Teachers report they do not regularly use materials that reflect the cultural background of the students they teach. (3.5)
• Teachers could promote student engagement in higher order comprehending during (4.1) and after reading informational text (4.1) more frequently.
• Teachers do not regularly check on the amounts of informational text being read by students. (3.5) 

Differentiation in Instruction

Teachers report they do not differentiate instruction on a regular basis (3.8) and they do not collaborate frequently with their colleagues about students’ literacy progress. (3.6).

Assessment to Inform Instruction

In general, teachers do not use assessment to inform literacy instruction very often. Teachers do not regularly use:

• pre-assessment data to inform instruction (3.4);
• standardized procedures for conducting informal assessment (2.8).
• assessments such as informal vocabulary assessments (3.0) and comprehensions (3.8);
• student assessment results to influence planning in areas such as fluency activities (3.5), vocabulary instruction (3.4), vocabulary activities (3.4), comprehension (4.0), and grouping arrangements (3.6). 

Technology in the Classroom

Teachers report technology is not a significant part of their literacy instructional practice. They are not comfortable using it to teach literacy (2.9), to assess student performance (2.8), or working with other teachers to develop technology-enhanced lessons (2.1).

English Language Learner (ELL)

Teachers do not incorporate instructional strategies aimed at ELL students regularly.  They do not adjust their assessment practices very often. (3.3). Teachers seldom plan lessons that include language objectives for ELL students (2.7) or have native language resources available. (2.3) Students rarely have the opportunity use their native language in class. (1.7)

Quality of the Chancellor’s Summer Academy (CSA

Based on questionnaire (N=37) and focus groups results (4 groups with N=6-8), teachers characterized the CSA to be the best professional development experience of their professional lives and rated nearly 90% of the sessions they attended as “high quality.” (The average item rating for the session questionnaires ranged from 4.3-5.0). Working together, teachers learned about what to do and how to improve literacy instruction. Selected findings are highlighted later in the report.

Speaker Sessions. 

• Speakers shared and modeled sound strategies that could be used in the classroom.
• Diversity. Multi-cultural speakers in particular enlightened their thinking on incorporating multiple ethnicities into curricula and classroom instruction. [One participant stated that speakers, “Opened my eyes.”]. 
• Speakers effectively translated educational theory into ideas about effective classrooms.
• The information was relevant to teachers’ school’s instructional challenges. (item ratings of 4.4-5.0) and useful for improving their instruction. (item ratings of 4.45-5.)

Literacy Instruction Sessions.

• The sessions provided helpful and concrete examples that captured classroom learning.
• Teachers reported the production of the lesson plans a “very useful product.” 
• Focusing on a small number of literacy strategies more deeply was very effective.
• These sessions presented new literacy strategies and showed them how to implement the strategies in the classroom. (item ratings of 4.45-5.0)
• Teachers wanted to participate in all the literacy sessions not just a few.

Strategic Action Plans (SAP)

Developing the Strategic Action Plans.

• Teachers expect the strategic action plan they developed to be useful in improving their school. (4.5)
• The plans they developed addressed their school literacy issues (4.4) and fits with what they are already doing. (4.4)
• Developing the plan as a school team was effective (4.4) and makes them “hopeful” about implementing the plan

Challenges to Implementing the Plan

• CSA teachers are concerned about their lack of content knowledge. According to district officials and teachers, they will be relying on UIUC expertise.
• Realistically, CSA teachers are concerned that not all staff will be interested in adjusting their instruction to incorporate the CSA strategies.

Summary

Based on the convergence of the evidence, the CSA was a rewarding experience. Many of the participants have left the CSA with new appreciations for the university as well as strategies that will help improve their pedagogical practice in the classroom.  Furthermore, the CSA has initiated conversations that have opened the door for future collaborations between the school district, state, and the University. One key to CSA success is that materials and information shared were practical and useful to teachers in the classroom. Also, instrumental to the success of the academy was the planning committee’s treatment of the participants and the resources allocated by the Chancellor’s office.  Thus, the evidence indicates that the academy is working towards the achievement of it first goals of providing support to enhance children’s literacy learning and forging a partnership between the University and local school districts. 

Future Concerns

1. The focus of future academies. CSA planning members, CSA teachers, and district officials are concerned about depth versus breadth. One district official said, “All our teachers need to know how to help students read--elementary and middle school. Students need to read for social studies, to do story problems in math, to learn about science materials, and to take the tests in each of these content areas.”  
2. Principals’ participation. Teachers, district administrators, the CSA team members,

and educational research all suggest that then “instructional” leader” is critical to school improvement. How can principals participate in the CSA?