01.05.14For Your Feedback: A Rubric for Coaches (Draft)
As many of my friends and colleagues know I am a big soccer (i.e. football) fan. Love the game and love thinking about teaching the game… because I find it fascinating, because my kids all play, and because there’s always vice versa- Thinking about how the things I’ve learned about teaching from observing classrooms do or don’t apply in training often provides further insights.
A few weeks ago I had the pleasure of doing a couple of workshops for coaches affiliated with the US Soccer Federation. I spent an afternoon with an elite group of technical directors from top clubs and then three days with A and B license instructors, most of whom were also coaches of top programs or college teams. To state the obvious it was fun, fascinating and humbling. The people in that room are one of the reasons I love the game so much– it’s full of people who are insightful and passionate and even after making it to the top of their field are still hungry to learn. I hope i was able to share some useful insights about what teaching looks like on the soccer field; I definitely learned a lot about it from them.
Anyway, I tried to put together a coaching rubric for the workshop- A one-pager you could keep on your clipboard to help you think about the quality of teaching when observing or developing a coach. I identified six important areas in which they could help coaches “teach” well.
Check for Understanding was one. It’s the hardest challenge of teaching–making sure students are learning what you are teaching. As a result there are of course practices where the players get it wrong for most of the drill and the coach doesn’t notice. Result: players practice doing it wrong and get better at doing it wrong. Let the coach who hasn’t had that happen stand now and take a bow….
Another was the presence of Systems and Routines to ensure that down times and transitions lasted the absolute minimum and that the coach had a healthy obsession with efficiency.
There’s a bit in there on Structure and Design— a reminder that “getting it right is the mid-point of mastery” i.e. that to really be skilled at something you can’t stop practicing when players get it right once or twice. You want them, to paraphrase one of my son’s coaches, not just to be able to do it right but to be able to do it right every time, while thinking of something else. That means a lot of repetition in a variety of situations.
My favorite point on the rubric is the idea of “Success Points,” that is, to me a great coach always knows what excellence in versus mere completion of an activity looks like. He or she sets players to work after having described and modeled not just know how to do it, but how to do it at the highest quality.
So..here’s my rubric: Criteria for Quality Instruction at Training.Revised.
Please know that this is an early draft, which I’m posting as much to get feedback on as to be useful. So, coaches, please feel encouraged to weigh in: Does it help you to assess teaching quality among coaches you observe? In your own work? What could be better? What could I add or drop? Is there a better way to structure it? How does it work for sports other than football/soccer?
Looking forward to your thoughts!
This rubric is very comprehensive and easy to follow. Keeping it to a one-page is key and you’ve done that here. I really think the indicators of sucess are very powerful as coaches can readily have a model of what “excellent” looks like. “Normalizes error” is under two criteria and I believe its a better fit under culture. The question to ask yourself when repeating sucess indicators is: “Where will this indicator have the greatest leverage?”. I hope this feedback is useful to you. Thank you for sharing and keep up the great work!
Thanks for your feedback, Claudia and for the note about my double booking “normalize error”… you’re totally right (i just think it’s so important that i try to put it everywhere) but your guidance that I should put it where it will have greatest leverage is really helpful. It shall be done. 🙂
I think this is really good and tremendously helpful. As someone who coaches, I would love to be evaluated with a rubric like this. This type of eval. is lacking at the college level. Nothing like this is used. The other problem at the college level is that there is currently a trend to hire administrators who’ve never coached. So not only are we not effectively evalulated in my opinion, we’re also hearing from people who’ve never walked the sideline. Perhaps there is a marketplace for this type of eval/consulting/coaching coaches?
Hello Doug Lemov- I am fan of your education work and I am also a huge soccer fan. I played since I was 3, and until I was about 25 – then coached for about 7 years.
My business career has been focused on assessment of ability and remediation toward mastery – to start with – in manufacturing and production environments and then in technology training and Professional Development. When I was young, my best soccer coach told me to select the skill that I was worst at, and work on it until it became something that was a strength. I look for ways that people learn and help provide scaffolding for their individual learning.
Having a rubric to focus observation on is absolutely essential to create a meaningful measuring stick that is common for assessment of any skill – including coaching soccer, or teaching a classroom. I recognized the need for a systematic way to track progress toward mastery on non standardized tasks about 7 years ago.
In 2007, I reasearched the University Technology Transfer Departments and found that Dr. Behrmann with George Mason University (and the Helen Keller Institute for Human disAbilities) had developed a progress monitoring tool called the KIHd System (Keller Instructional Handheld Data System) to track the educational progress of kids with Autism.
In 2008, I licensed their work for commercialization of a method to manage individualized teaching instruction from numerous instructors with a longitudinal view.
My logistics background led me to build a derivative work of the KIHd System that would measure observation and individual progress toward a target of individualized mastery, while creating the ability to compare Subjects on Common Targets. When Common Core Standards were being developed for education we incorporated the Rubric of Common Core into the KIHd System so that users who need academic support can have longitudinal tracking of their progress and a portfolio of what types of remediation have been successful in the past.
In building this tool – I strove to make it accessible to any instructor who is evaluating a Subject doing some defined Target so that the lessons learned by one teacher of coaching can be capitalized on by other support coaches or teachers. I launched this KIHd System in 2008 and after seeing the progress with sales and data being collected – I discovered that even collecting the data in real time and putting it into a rubric is not trusted by the adminstrators or the Subjects of Observation (if they are not what they expect).
It was a painful lesson to learn but the fact of the matter is that a note on a piece of paper is just not enough of a insight to make an accurate adjustment to coaching method. The research from George Mason with people with Autism was to use 5 data points before a trend line was ‘trusted’ to modify the approach of instruction – but in practice – even 5 data points on a piece of paper or entered into a system to graph it – is not trusted a year later. What is missing is the video to go back to and actually see the occurrence. Then it is trusted. Video does not lie or forget.
When coaching for high level sports competition, “viewing tape” in preparation for a game is standard practice. During the Game – the refs going to the instant replay to see the event in slow motion and zooming in on specific areas during review eliminates uncertainties. Slowing the tape down to micro analyze how the opponents tendencies ‘fit’ with your strengths will usually give you a competitive advantage. But when the critical eye of observation is focused on the things that you can control, and the things you can influence the result is usually much greater and able to be tracked over time toward mastery. This sort of tracking enable a trajectory of change – if you have a rubric to standardize around.
I looked at your rubric for Soccer Coaching and I decided to prepare my tools – wrapped around your rubric to demonstrate several benefits of this technology enabled approach.
First tool – Dartfish EasyTag – this is a FREE app from the google play or iTunes market. This app allows the viewer to “tag” observations on a handheld device (ipad, iphone, android phone) and then email the tags to a Licensed version of the Dartfish Video Platform to merge with the video. In this case I have populated a video of Arsenal’s (Barclay Premier League) on a Dartfish TV channel that I created for you and your users to see how the tagging panel flags events that correspond to the rubric you created.
As one of the comments from a coach indicated – coaches wish they were evaluated on a rubric like this. I agree. The only way that really has use is if the standard is applied across all coaches in a similar role. To demonstrate this capability and interconnectivity – I have created a KIHd System Account for you to:
a) see your custom rubric in the KIHd System (Stored in Parameters)
b) see and adjust the relationships for Instructors and Learners
c) see how a single example video with no tags can be used to normalize the inter-rater reliability and enable the fine grained differences to be identified for each Subject on each learning objective that is relevant for them.
The untagged video of the Arsenal Practice I uploaded is available here: http://www.dartfish.tv/Dispatch.aspx?target=collection&CR=p97911c82070m1757552&sh=li&aid=caf8d94d-f7c0-4af2-9d34-a28082f5773f (A limited Collection on your Dartfish TV Channel) There is other Uncommon Schools and Video that was populated for demonstration purposes.
If you do not have the Dartfish EasyTag app, you can download it with this link: https://itunes.apple.com/us/app/dartfish-easytag/id369126115?mt=8&ign-mpt=uo%3D4
The KIHd System Account I created for you is available at: https://login.adeeperview.com/public/login.aspx
I have included as Subjects – the coaches from the Barclay’s league and developed a set of credential for your users to try it out. Barclay’s has 4 divisions: League One, League Two, Premier League, and Champions League.
I have made 4 users – whose Username and Password are the Same for this demonstration:
UN: Coach.One Pass: Coach.One
UN: Coach.Two Pass: Coach.Two
UN: Coach.Premier Pass: Coach.Premier
UN: Coach.Champion Pass: Coach.Champion
I have not made ANY Targets or identified any Reinforcer Variables or Secondary Behavior Variables for any of the coaches. In my opinion, an initial evaluation of each coach should be done from a video that is able to be referenced. The Tagging Panel would identify the metadata in the practice that is relevant from your rubric. Subject Measure of 1-5 ratings can be applied to each example from each coach to build a library of useful tagged content.
I would be happy to arrange a call and show you how all of that works (it is pretty straight forward as Dartfish has been doing this with WorldClass Athletes for 15 years. In the meantime check out the Tagging Panel from Dartfish EasyTag and watch the video of Arsène Wenger of Arsenal at a recent Open Practice.
In past introduction work with KIPP Academy and Uncommon Schools I created a Dartfish.TV Channel for you that contains some of your publicly available education video also.
My email is mdarden@adeeperview.com is you wish to discuss.
Honestly I think the KIHd System and Dartfish tools are an essential set of tools that need to be incorporated into the observation and coaching fields – what ever the discipline. It is how progress is made!
I am looking forward to hearing from you.
Michael Darden
A Deeper View
mdarden@adeeperview.com
http://www.adeeperview.com
http://www.dartfish.tv/adv