Doug Lemov's field notes

Studying implementation with network leaders

As we’ve shared in our first and second posts, we were fortunate to partner with Harmony Public Schools as they endeavored to deliver high quality professional development systems focused on Lesson Preparation for 300+ instructional leaders across 60 campuses.  In this post Dan Cotton and Dillon Fisher discuss how they worked with Harmony to develop and refine systems to constantly assess how professional development was shaping outcomes in the classroom:

One key to successful organizational change (in organizations both small and large) is to build strong systems for studying progress.  Successful professional development is more than a sequence of events; it also has to include a process for self-study and learning. 

For example, school leaders need to be able to answer:

• What’s going well? What are the opportunities?
• Where are early Bright Spots that can be used to set the bar and build momentum? 
• What are we learning about what it takes to make change in our organization?
•  

One key action we undertook with the help of some incredible leaders at Harmony, was to co-build an Artifact Study System to provide us with this source of feedback. We’ve broken down our analysis of this tool into two pieces: 1) the Artifact Collection System itself 2) lessons learned from studying artifacts leaders submitted.

Artifact Collection System

Before Harmony could study the reality of Lesson Preparation, they first needed an effective way to gain regular access to the preparation and execution of their leaders without visiting 60 campuses each month. The Artifact Collection Form is a digital form that allows instructional leaders to submit their meeting preparation, a video of the coaching meeting, and the final Lesson Preparation that they co-produced with their teacher. Here are three lessons we learned about the system itself:

1. Keep Artifact Collection Predictable: Instructional leaders at Harmony were regularly engaging in Lesson Preparation coaching meetings but remembering to film and submit them was a new practice. Harmony built a Scope & Sequence at the beginning of the year to keep submissions predictable and clear for leaders. One set of artifacts, due monthly, seemed to be a cadence that felt possible for leaders. How do we know? A sample month led to 734 submissions.
2. Make The Process As Simple As Possible For The People Submitting: While the number of artifacts turned in monthly means the system was large and complex, Harmony succeeded by keeping the process as simple as possible for leaders submitting their work. When it was time to share evidence of their implementation, instructional leaders accessed a simple Google Form that asked leaders for their name, campus, and role, and then an upload button to help them connect the leader preparation, video, and teacher lesson prep from their Google Drive to the form. It looked like this:
   
3. Make It Accessible for Senior Leaders: While the form appeared simple for instructional leaders, each month the form collected as many as 800 artifacts into one large spreadsheet. We learned quickly that if the system creates more work than the value it creates. you have a problem! Stavroula Rojo, a system-leader on Harmony’s Performance Reporting team, added filters to make it easy for senior leaders to access what they needed: Assistant Superintendents could filter by region, Principals could filter by campus, Curriculum Directors could filter by content area, etc. All of this made it easy to find what one needed quickly to study artifacts. Additionally, Stavroula was able to create a data-tracking tool that supported accountability and follow-up across the district.

Study & Respond:

As the artifacts rolled in, Harmony shifted to the most important part of the process: How can we effectively study the evidence of implementation to accelerate progress?  It was relatively easy to ensure organization-leaders had access to the evidence of Lesson Preparation execution, but figuring out how to best use the system to drive action was a learning process. Our work with Harmony helped distill two lessons we think are widely applicable:

1. Collaborative Study is Critical: When embarking on a new instructional practice (or a newly prioritized one), system-wide leaders can’t expect to drive change on their own. The goal of gathering data is to enable meaningful collaboration. With Harmony, we co-led bi-monthly “Study Meetings” with Assistant Area Superintendents (who were responsible for coaching principals). We utilized these meetings to collaboratively study 2-3 videos and artifacts from different regions that exemplified common and high-leverage challenges. Assistant Area Superintendents used these meetings to build shared understanding of effective implementation, further refine their shared vision for Lesson Preparation coaching and to practice developing specific praise and narrow action steps they could share with leaders to accelerate their practice. The collaboration was critical: often Assistant Area Superintendents left those meetings with shared tools (ex: Feedback Cheat Sheets, checklists for studying video) that allowed them to be more effective as coaches-of-coaches. Additionally, we were able to spot trends that informed the focus areas of upcoming leader development sessions so that sessions consistently felt responsive and relevant.
2. Feedback Drives Buy-In: Perhaps most importantly, we witnessed how much leaders crave feedback when they submit evidence of their hard work. Campus leaders, just like system-leaders, were invested in Harmony’s goals and wanted to be as effective as possible. The more feedback they received, the hungrier they became to improve and show evidence of their growth. Of course, “winning” on individual feedback when there are 700+ artifacts is no easy feat. Harmony focused on a coaching-of-coaches model: Assistant Area Superintendents gave feedback to principals, principals gave feedback to instructional coaches, and all leaders were in role-specific Professional Learning Communities that leveraged peer-feedback (with feedback cheat sheets to support strong feedback) on a bi-monthly cadence.

The Artifact Collection System was central to our success. Following the launch of the Harmony project, we have replicated the use of an Artifact Collection System in other TLAC projects to give our team and our school partners a window into implementation and a source for uplifting bright spots. We’d encourage any school or school system launching a new initiative or a new curriculum to build and use a similar system.