A Quick Note- the eCOVE Software Blog

It's come to my attention that folks didn't know that they needed to sign up for both the Data-Based Observation blog (this one) and the accompanying eCOVE Software blog separately. 

The eCOVE Software blog describes various tools, techniques, bug fix announcements, etc related to the software. If you'd like to keep up to date (it's not a daily blog), just go to the blog and enter your email address.

Peace,  John

"Tenured Teachers Costly to Fire"

This title is from an article in the Teacher Magazine where they report some astounding figures.  In New York City it costs $250,000 to fire an incompetent teacher (they give examples of law-breakers as well as those not competent in the classroom). Last year they fired 10 teachers - that's $2,500,000 worth. It's also one out of every 5500 teachers or 0.00018 percent, so it's not a condemination  of the profession. The cost, however, is significant and unnecessary. I believe that there are heavy legal costs, quite often due to a conflict in definition and judgments about what constitutes good teaching. Those conflicts, and the associated anguish and costs, can be greatly reduced using the Data-Based Observation Model (made even easier with the eCOVE Software).

The full process would include all vested parties making the language of their standards clear and in observable terms (I'd love to help with that). Once that's done,  objective data collection is implemented using eCOVE Software or pencil/paper/stopwatch, and professional discussions begin. When the criteria is clear and the evaluation is based on objective data, reaching consensus on the quality of the teaching is much, much easier.

Where there is conflict about the data collection, the two parties agree on a neutral third party to observe and gather the objective data. That removes the observer bias or agenda (perceived or real). These decisions are hard on everyone involved, and doubly hard on the kids until they are made. The tools are available to make the entire operation more equitable and less costly.

Teachers and Reflection

I wrote this in response to a thoughtful posting on another blog, The Speculum, where the author, Lennie Irvin, presents a draft of a pre-dissertation proposal. It's about writing and reflections, and presents interesting definitions of reflections.

-----------------

A practitioner's response:

I'm the retired (ha!) director of the Willamette University School of Education where I gave numerous assignment for student/student teachers to 'reflect' on either what they saw in a observation of another teacher or on their own practice as a student teacher. Our program was heavy in requiring reflective tasks. From that experience comes this caution - the reflective practices of your students may not be done under the same definition you are proposing. If I use hindsight to define reflection as used by the vast majority of our students, it was to describe what they saw or remembered, with a cautious evaluative conclusion.

Again in hindsight, I don't think the faculty had a clear definition of reflection, certainly not a common one, and I don't believe, based on the outcomes, that we make the task of reflection clear to the students.

As I 'reflected' on my career in teacher education (reviewing what I had done, what was effective/not effective, identifying a central theme to my practices, and what I would change were I to start over) - my post retirement definition that I wish I had used with students - I concluded that my efforts to improve their teaching skills through praise, criticism, and offering of various solutions to their teaching problems was largely ineffective. It wasn't until I took a more objective role in observation/evaluation that I found success, both in the depth of teacher reflection and in effective change in teaching practices.

I found that when I shifted from feedback in the form of anecdotal notes to providing objective data on what was happening in the classroom, the students/student teachers shifted from a defensive/deflective or accommodative response to one of independent reflection (definition above), problem solving, and change.

In my now failed retirement, and initially as a mental exercise, I developed the Data-Based Observation Method and then wrote a software program to support it. The software is not necessary, but make it much easier.

In the Data-Base Observation Method the focus of an observation is collaboratively determined between observer and observee - "What do you want to know about your classroom?". Guiding questions can be used to focus the process on categories such as content delivery, class management, student relations, etc. Once the focus has been determined, the observer will gather objective frequency and duration data on classroom behaviors of teacher and/or students.

Here's where the reflection come in. The data is provided to the teacher with these questions "Is this what you thought was happening in your classroom? Is a change needed? If so, what will you change?" When this external, objective picture of what had happened in their classroom (part one of my definition - reviewing what I had done) is presented, the teachers make the determination of whether the teaching/learning was effective/not effective (part two of the definition).

Professional level discussion ensues, based on the objective data, to determine if and/or what changes should be made (part four of the definition - part three, identifying a central theme, comes after a number of the data-based discussions).

There now has been five years of implementation of this method by me, school administrators, and peer coaches and I have found that it builds the skill of reflection in teachers, a skill I am coming to see as a critical element in becoming a long-term successful teacher. I believe that 'reflection' is shallow and surface when the person does not have the factual basis for understanding what occurred and when the reflective task is to be either self or externally evaluative in nature. The question is not "How did I do?" (Answer: good, poor, passed, a 6 out of 10, I liked it, etc), but "What happened and was it effective?"

Teacher Evaluations From The Teacher's Viewpoint

In a May, 2008 report by the Education Sector titled Waiting To Be Won Over there is an interesting and revealing question: exactly what do teachers think of formal evaluations. The graphic here illustrates a significant problem. In this chart 41% of the teachers viewed formal observations as a 'formality'. When I look at the evaluation forms and the standards they are based on, it's quite obvious why they would think so. The language in the standards/indicators and on the forms is very frequently so vague as to be unverifiable.

Only 26% viewed the evaluations as 'useful and effective' - just 1/4 of the teachers!

But what I found most interesting was the 32% that saw formal evaluations as 'well-intentioned but not helpful'. To me that indicates that both parties - administrators and teachers - had hoped that the evaluations would serve a good purpose, but were disappointed.

The reason for these disappointing numbers is twofold: the problematic language in standards and the subjective nature of most evaluations. To illustrate, here are three standard/indicator statements taken from a large district's teacher evaluation/professional development document:

"Students initiate and take responsibility for their own learning."  Not only is this a stretch to view this as within the responsibility and power of the teacher to invoke, but this is not a behavior that is observable through the series of classroom observations nor through a paper-trail of student work.  It's a wonderful value, but unusable as a basis for teacher evaluation.

"Teachers connect new knowledge to prior learning."  This can be observed by looking for statements or assignments that directly make the connection between a students prior knowledge and the lesson at hand. Data can be collected on both the teacher's statements and the student's statements and questions.

"Teachers make the lesson objectives and learning targets clear for students in advance of the lesson."  This too can be observed by tracking statements made by the teacher; questions, statements, and errors made by the students; and the response by the teacher to student errors. However, the phrase '...in advance of the lesson' make this a bit more narrow - this is a standard to be accomplished before the lesson begins, and does not account for the changes that need to be made during the typical lesson. Of course, making the goals clear before the task begins is very important, but there will also be a need for objectives and targets to be adjusted as a result of feedback during the lesson.

The key to all this is that the language in standards/indicators were initially written as general guidelines, and are now being used as the basis for evaluations. There is a serious need to revisit the standards and rethink/rewrite them into evidence that can actually be gathered. It's not fair or reasonable for either the administrator nor the teacher when the objectives and goals of their behavior isn't clear. 

With clear standards, written in observable terms, and objective tools for collecting the data, the subjective nature of the process is diminished. When the objective data is presented to teachers, I predict the numbers in both the 'formality' and 'well-intentioned, but....' categories will significantly decrease. 

Aligning Tools To Standards

I've been doing extensive work aligning (and creating) eCOVE tools that align with various district and state standards for teacher performance. As I create more tools, needed to provide objective data in line with the language in the standard/indicator, I am continually amazed at the uselessness of many of the standards.

It's not that the standard lacks value as a broad guideline, but they are often worded so that the interpretation of the standard can be so broad to lack any form of reliability -- or that the standard/indicator points at some internal value such as 'caring' for the students or 'believing' in the students.

An example from Danielson's latest version of Framework for Teaching: "Teacher's plans and practice reflect accurate understanding of prerequisite relationships among topics and concepts." What would one observe, in print or in a live situation, that would serve as evidence that the teacher understood a prerequisite relationship? And what knowledge and skills would the observer need to adequately judge this indicator for a variety of content areas? Nonetheless, I am sure that there are many evaluation sheets being checked on the basis of  'a feeling' or a vague interpretation of this language. No wonder 40% of teachers view observations as a formality.

The solution is not to rely (solely) on eCOVE to identify ways to gather data that might be conceived as supporting this indicator, but to REVISE the standards language. In fact, I'm setting a high priority for myself and eCOVE to create a set of standards that have observable behavior indicators attached. If you'd like to help, let me know. Stop by the website - www.ecove.net - and email or call me. If you have a set of standards that have been revised and have observable indicators, PLEASE contact me -- you'll be doing the profession a big service.

If we can't identify the evidence that would support the achievement of a standard, then the standard is useless at best, and frequently dangerous and misleading. When the standard lacks the possibility of a clear line of evidence, it can be interpreted and applied by each observer - and thus is no standard at all.

Self-Directed Professional Growth

I've been in numerous discussion with Teacher Leaders recently, and come across a common frustration. It seems that no matter how hard they try to convey that they are 'just there to help', there lingers a resistance and trust issue. When I did deeper, I find that the observation process is one of identifying 'good' practices, and sometimes 'bad' practices; or it's one of taking notes on 'what occurred during the observation'.

I think that when the process has, in any way,  judgment or valuing language involved there will be a defensive resistance. Even the notes are a problem as the choice of what to record is a judgment made by the observer.

Since this doesn't occur in the Data-Based Observation Method, it's really easy to get past the initial trust concerns - you just have to follow the system and prove that you are not there to hammer them with the data.

-------

Key concepts:   Don't Praise, Don't Criticize, Don't Provide Solutions!

-----

Follow this sequence of interaction:

Pre-conference: Centered around determining what data to collect - "What do you want to know about your classroom?". This an be in light of a teacher's individual goals, some teacher-perceived problem, or building/district/state standards.

-----

During the observation, gather data without making any comments reflecting praise, criticism, or solutions.

-----

In the post conference, ask these questions when presenting the data:

• Is this what you thought was happening in the classroom? (teacher reflection and interpretation)
  
• Do you think a change is indicated? (teacher and observer professional discussion about the interpretation of the data)
  
• If so, what will you change (teacher ownership and empowerment; enhanced professional discussion)
  
• How can I support you? (professional collaboration)
  
• When should follow-up data be collected to see if the change is effective? (making the entire process not one of pleasing the observer, but in implementing effective change)
  

This approach shifts the dynamic from defensiveness to empowerment, from judge to colleague. There is no observer, Teacher Leader or Administrator, who can solve every classroom problem. It's far better to develop the teachers' skills in reflection and problem solving. This can be accomplished by basing the discussions on data rather than opinion.

-----

I welcome comments about any of my thoughts.

Teacher Pay, Portfolios, and Best Practices

I just read an article in Ed Week about the Houston and Denver districts' efforts in teacher pay for performance. Both programs are broad implementations of the pay-for-performance system and are struggling with enrollment and acceptance. The most interesting quotes were by Gayle Fallon, President of the Houston Federation of Teachers. Both quotes, "It's better than last year. Still, they are handing out money and getting nothing in return."  and "What we hear from teachers consistently is that they have no clue what they did to get the money", point to the black-box nature of using student test scores as a primary determiner in awarding pay or other rewards. While student learning is the primary goal, the connection between the teacher's direct influence on student scores (as they indicate learning) is very difficult to determine. I can understand the teachers not 'having a clue' when the results of their efforts (the test scores) are calculated and revealed sometime in the future and those results also include the influence of a large number of other variables.

The Sherwood School district in Oregon has recently received a large grant from the Chalkboard Foundation to improve student learning. Part of their efforts include a bonus pay system based on a teacher portfolio of evidence, which can include student scores as well as other strong evidence of exemplary teaching and professional conduct. They contacted me to discuss the use of data-based observation data on best practices as a part of that process. We haven't finalized anything, but it sure makes sense to me (naturally).

We have credible research about teaching practices that result in increased student learning. We have a tool (eCOVE Software) that will track the implementation of those practices in an individual classroom. Now we have the opportunity to reward teachers who are implementing those researched best practices. The process is not difficult to manage - identify the behaviors that everyone is confident in as directly influencing student learning (Class Learning Time, Time on Task, Wait Time, Level of Questions {as answered by students, not just asked by the teacher}, etc, etc), train observers (teachers, aides, paid data gatherers, administrators) to competently use the data collection tools, and determine the appropriate data collection procedures (number of data points, length of individual data collection events, etc).

The result of this, I predict, will be interesting and engaging. Not only will teachers know immediately that they are using the researched best practices in their classroom, but they will have a running record of that. It's that running record that is the greatest benefit - it can provide feedback to the teacher who has a goal of becoming an exemplary teacher in a timely and useful manner. They can immediately see if they are moving toward a higher level of proficiency instead of waiting for months to find out if they 'won'. As I've said before, teachers are deeply dedicated to effectively teaching their student in the best manner possible. Bringing the objective feedback to the classroom level in real time will build more effective teachers; then we'll know why those scores went up as well have the 'clues' we need.

As a side note, I'm a bit concerned that the student performance/more money is a strong extrinsic motivator and will shift the focus on why one becomes/continues to be a teacher. I think the immediate, objective, and over time feedback that eCOVE provides will not only reinforce the skills of teaching  but will also reinforce the teacher's perception of their skills. Since we love doing what we do well, the data and teacher reflection become the intrinsic motivator. As teachers become/continue to be successful in their craft, and are clearly aware of their successes, they will keeping doing what they love - helping kids.