Final Exams and the Art of Percentage Marks

It's exam time again. High school students around the province are anxiously preparing for their final exams. Many students would be very happy with the idea of abolishing examinations. I would too, they are the ultimate teacher make work program. Not only do I design examinations for my students, I take care to create a fair environment where my students can write them, then I spend hours marking all of them, and recording the grades in order to come up with final marks and informed comments on learning.

It's work all around, I tell you. I'm not complaining, it's a real art form to come up with a single percentage mark that summarizes student learning. That's what I want to talk about today, the art of a percentage mark. That single mark. You know, on these.

Percentages marks are always based on evidence of student learning. Each teacher gathers evidence of student learning in four KICA categories over time,  and then creates unit tests or projects that summarize the key learning across these four categories. (The KICA categories are part of the Ontario Achievement Chart and stand for knowledge, inquiry/thinking, communication and application. These are province-wide performance standards.)

Without evidence of learning, a teacher cannot determine if learning has taken place, and therefore cannot assign a mark. In these cases, the student will be left with an incomplete course on their high school transcript at best. That's why teachers are such sticklers for due dates and assignments being handed in. And this line of thought leads us to the concept of "most consistent, most recent" bit that teachers talk about.

Most Consistent, Most Recent
The Ontario Growing Success policy document makes it clear that teachers should use the “most consistent, most recent” evidence of student learning when calculating percentage marks (Growing Success, page 39).

A lot of teachers I know use Markbook (Asylum Software) to record and calculate grades. This year, in accordance with the Growing Success policy document, the Markbook developers changed the default mark algorithm to Blended Mode (away from Weighted Average).

This led to some very interesting situations for myself and colleagues when the progress reports came out last October. I myself handed out two separate printouts to my students because I had not yet mastered the new output format of Markbook. Some student marks differed by more than 5% on the two reports. This led to several great conversations with students about what a percentage mark means, and I want to summarize the message of these conversations below.

Mean, Median and Mode

MarkBook has five distinct ways of calculating a student’s percentage grade which can produce five different marks for each student:

• Weighted Average - the arithmetic mean 
• Weighted Median - arithmetic median
• Weighted Mode - arithmetic mode
• Blended Mode - a combination of modes, levels and percentages, explained below 
• Blended Median- a combination of medians, levels and percentages, explained below 

There is a brief description of mean, median and mode in the image above. The mean is the sum of all the numbers divided by the quantity of numbers in the set. The median is the middle number, or the average of the two middle numbers. The mode is the most commonly occurring number. And the range, is the highest number minus the lowest number. The range is simple for mark sets, it's 100-0 = 100.  (I'll define blending and weighting in a few seconds)

The Art of Percentages

The intention of the Ministry is that each teacher considers the mean, median and mode simultaneously, and considers a student's entire mark set in order to make  an informed judgement about each student’s acquisition of the curriculum. There is a diagram to describe this below.

1. When there strong agreement among all three calculations, e.g. a range less than 5 % between the mean, median and mode, the teacher can be quite confident that marks are not being skewed by any one calculation and the average of the three measures strongly indicates the student's progress. This is the symmetric distribution seen in the image below, with strong agreement between the mean, median and mode.
   
2. When there is not strong agreement among all three calculations, as in the skewed distribution above, the teacher is to look at the most consistent and recent levels of achievement in conjunction with the mean, median and mode.(Growing Success, page 39). For instance, if the student aces  the final exam, the mark on the exam will strongly weigh into the teacher's professional judgement because it is the most recent evidence of learning.

Weighting and Blending
Two additional terms that you need a definition are weighting and blending.

Weighting refers to assigning some student scores a greater weight than other scores, because they come later in a unit (a final project) or are more significant (such as a unit test). For example if a teacher weights a vocabulary quiz at a 1 and unit test at a 10, it means that the unit test counts for 10 quizzes, and that the unit test is 10 times as significant as the quiz.

Blending  refers a three-step process starting with a weighted median or mode in each KICA category (knowledge, inquiry, communication and application) then:

1. Converting this number to an achievement level (0, 1, 2, 3  and 4).
2. Converting this level back to a percent using the mid-range value of that level (e.g. if the mode is level 3 with a base of 70 and the  base of level 4 is 80, then the percent assigned to a level 3 result is 75%), and finally,
3. Finding the weighted average of the mode or median  of each KICA category.

Why so complicated... well,  the fact of the matter is blended mode is the measure that most closely reflects policy, “ most consistent, more recent” (p 39, Growing Success). All deliberations around determining a final term mark/grade are intended to begin from the blended mode as a point of reference.

My Personal Art of Percentage Marks
I started this semester influenced by my personal comfort with using the arithmetic mean as my preferred  representation of central tendency.

Most teachers I know will immediately reference this comfort, most teachers I interact with take the mean as a valid interpretation of a mark set. However, I began to see problems, especially because the mean can be influenced by outliers. I saw that when I left only the summative "assessment of" marks in my calculations, the arithmetic mean tended to swing by 1-5 %. My observation was that it is really not fair to penalize a student for failing to learn quickly enough, when there is evidence they did indeed learn the material by the end of the course.

As the semester progressed, I saw and experienced first hand that weighted modes and weighted medians are a much better representation of central tendency in a student mark set. These values are are not greatly influenced by outliers. You can see this in the image below. The mode and median describe this mark set more accurately than the arithmetic mean.

It is my intention that a student can have a 'bad hair day' mark-wise early in the unit, without it shattering their final mark. Similarly, acing a few quizzes early in the unit before slacking off should also not unduly influence a final mark. Consistency is the key.

What really matters for the percentage mark is that student demonstrates achievement of curriculum expectations by the end of the course. (Students are also required demonstrate of learning skills and work habits, and these are an integral part of a student’s learning but these are separately evaluated where possible. That is a whole other discussion.)

This past midterm,  I started with the arithmetic mean, and then used both the weighted median and weighted mode, to inform my efforts to leave the mark the same, or adjust it upwards or downwards in order to consistently represent the student mark set.

This process of mark evaluation puts the ‘professional judgement’ of the teacher back to the forefront of  learning. Growing Success asks teachers to use professional judgement in order to:

• determine which specific expectations should be used to evaluate achievement of the overall expectations 
• determine which specific expectations will be accounted for in instruction and assessment but not necessarily evaluated. 
• weigh all evidence of student achievement in light of these considerations. 

"Percentage marks represents a student’s achievement of overall curriculum expectations, and should reflect the student’s most consistent level of achievement, with special consideration given to more recent evidence."(Growing Success, page 39)

As Growing Success states "Teachers’ professional judgements are at the heart of effective assessment, evaluation, and reporting of student achievement." (Growing Success, page 8)


I’m very much for it.

References:
Provincial Report Card 
Growing Success
Markbook 
Ontario Achievement Charts

Authority with Grace - stolen cell phones, cheating on tests and misbehaviour

The most common reoccurring ethical issue that I have experienced in the classroom have been concerns regarding student attitudes to school and work. When I first started teaching, I confess that I thought that the most effective way to deal with a disruptive student in my classroom was to send them to the office. And as I discovered from the first student that I sent to the office for acting out in class, that this type of discipline was not discipline at all. The tone of  this interaction was not appreciated by the student and in fact it was ineffective in shaping the classroom environment that I was trying to create.

I remember a fellow new teacher suffered from the same misconception that I had, and in her case the office was more direct, they sent that student right back to the classroom, telling her to manage the misbehavior herself. 

Sending students out of the classroom is often not the most ethical way of dealing with student misbehavior. Instead, our most ethical response depends on our ability to decipher the cause of the misbehavior. This ability depends in a large part on emotional intelligence. For example, students can motivated to misbehave:

• Because they do not know what the proper behavior is
• For no other reason than they are having a bad day
• They are bored with a certain subject
• They feel uninvolved with the classroom or classroom decisions and feel it is an unfair environment
• Their teacher seems to expect negative behavior from them
• They want attention, power, or revenge
• They have given up on themselves and feel anxious or inadequate.

Since I have a lot of control over what goes on inside my classroom and little control over what goes on outside, I began to focus on how I interacted with students while instructing them.

First, I started by incorporating the philosophy of inclusion in the classroom (using the Tribes philosophy).

The Tribes philosophy focuses on incorporating activities that are designed to create an atmosphere of respect and responsibility in the classroom.

From the first day of school, I discuss the concepts of respect, responsibility and participation, and develop both curriculum and community learning goals for each lesson. Immediately I noticed that the students responded very positively to these changes, the quality of our discussion improved, and the quantity of disruptions dramatically decreased over time.

An important part of the Tribes  process is setting up four key classroom rules which govern our interactions. These are:

• Mutual Respect
• Attentive Listening
• Appreciation - no Put-downs
• The Right to Pass 

The idea is that through discussion, the students and teachers can reach the stage of community where students and teachers can work together creatively to contribute their talents to the classroom.

 

Restorative Practices

My second step was becoming trained in restorative practices. Restorative Practices has deepened my practical understanding of classroom leadership.

Restorative practices focuses on the art of seeking to solve problems collaboratively, instead of seeking to solve problems by placing blame.  The premise of restorative practices is that students are happier, more cooperative and productive and more likely to make positive changes when those in authority do things with them, rather than to them or for them. Restorative practices is proactive, rather than reactive, and it works to improve and strengthen human relationships by repairing harm. Students feel fairly treated and are less likely to re-offend. 

Since I started using restorative practices in my classroom, a lot of what I choose to do in terms of classroom management has to do with involving the student into the problem solving process and re-affirming our relationship, rather than relying on punishment as a social regulator.

When I encounter an behavioral dilemma, the main tool in my arsenal are the restorative questions. These questions provide an opportunity for everyone involved to tell about the incident from their perspective and how it affected them. Everyone gets to speak, and because they feel heard, I frequently get apologies from students and the process ensures they do not repeat their actions that caused the harm. The their behavior has changed and our student-teacher relationship has strengthened. My classroom feels much friendlier.

The restorative questions are purposely scripted, and provide all involved with an opportunity to speak and listen to one another in an atmosphere of safety and equality.

Restorative practices has helped me to successfully mediate a variety of classroom conflict: from stolen textbooks, cheating on tests, finding missing cell phones, dealing with poor word choices and excessive lates.

Teachers face ethical dilemmas on a daily, weekly and monthly basis. Our focus as teachers is to provide equal opportunity for education for all. One size does not fit all. In teaching, what is considered the most ethical often comes down to determining what is in the best interest of the particular student involved with the situation. Both the Tribes Philosophy and Restorative Practices have been critical in shaping my understanding of classroom leadership.

References:

Tribes Learning Communities: http://tribes.com/about/research-based-components/

Restorative Practices: http://www.iirp.edu/pdf/beth06_davey2.pdf

Personality Colours in the Classroom

Myers Briggs
The Myers Briggs Type Indicator (MBTI)  measures the preferences in how people perceive the world and make decisions.  It's a comprehensive test and quite involved since there are 16 different personality types defined in this test. You can see these 16 types in the image to the right. 

It's complicated. I took a leadership course in which we determined our Myers Briggs personality type.  We spent two of the four sessions just getting a handle on comparing and contrasting the four aspects:

• Extraversion with Introversion
• Sensing with Intuition
• Thinking with Feeling 
• Judging with Perceiving

By far, the  most rewarding part of the course was working with purposefully diverse teams and experiencing first-hand the interaction styles of all personality types.

Personality Colours
I use Personality Colours test extensively in my classroom. The four colours in the test simplify the sixteen types of the Myers Briggs:

• Blue for the Harmonious
• Orange for the Adventurous temperament
• Green for the Curious
• Gold for the Responsible

Right now I am teaching one very ‘orange’ class. More than 80% of my students in this science class of mine are ‘orange’. This means that they tend to be extroverted, outgoing, active, and kinesthetic learners.  

I am a green/gold personality, so the enthusiasm and natural inclusiveness of my students came as a welcome surprise to me. I am continuously warmed by their sheer willingness to participate in activities and their extremely positive response to our classroom lessons of inclusivity and respect.

Knowing that this class is orange helps me find solutions to their natural busy-ness in science, a subject that works on developing the green/gold personality traits. For example, it helps me to be understanding when an ‘orange’ student asks yet again to be let out for a drink of water (really a walk and a destressor). It also helps me create assessments that involve their kinesthetic tendencies, such as setting up laboratories as stations activities.
 
In addition, knowing their personality colours really helps my students.  It helps students understand why they like to do the things a certain way,  and why sometimes a different choice makes more sense for them than their peers. It helps them understand why some subjects come naturally to them to see that even if another skill or subject does not seem as natural, it is still important and worth the effort to learn. Personality colours also helps me to create a purposeful teamwork in the class. I purposefully pair complementary colours or diverse colours, so that my students can gain a practical understanding of their own strengths in relation to the strengths of their peers.

I feel that I as a teacher have a great role in teaching personality colours. I think the tests of the personality colours are remarkably accurate for me. They reflect how I think about me, my preferences and what I make a priority. Personality colours provides a framework in which my students can reflect on their strengths and develop strategies for their areas of need.

Try them:
Myers Briggs - Test, YouTube video
Personality Colours - Test, YouTube video
A Personality Colours handout for the classroom handout

Classroom climate and intercultural understanding

I have had the question, on occasion from one or two people who ask me how I compare creation stories to scientific evidence. I've been asked because I am a science teacher and I also am interested in multiculturalism. Last week I helped to organize and host and Educator's Tour of the Mississaugas of the New Credit First Nation.

Well, to be honest I think this question arises from a misunderstanding. Comparing creation stories to scientific inquiry is like comparing apples to oranges.

I want to first highlight the intention of Ontario's curriculum documents. What I teach in science class is scientific literacy. Scientific literacy is "the scientific knowledge, skills and habits of mind required to thrive in the science-based world of the twenty-first century." (Ontario Ministry of Education. (2008), The Ontario Curriculum Grades 9 and 10: Science).

Science is responsible for much of the prosperity and technological advances we have achieved in the past 100 years. For instance, the scientific method of analysing data is used extensively in business, marketing, science and engineering and is responsible for:

• discovering life-saving pharmaceuticals
• developing vaccines to prevent global pandemics
• determining from the evidence that the climate is changing
• creating sustainable and renewable energy sources
• improving the productivity of our agricultural sector to provide sustainable sources of food
• protecting the environmental health and economic vitality of the oceans, among others.

I teach scientific literacy every day in the classroom, I teach students how to observe the world around them and to distill conclusions from data.

Our curriculum document also says that  "the overall intention is that all graduates of Ontario secondary schools will achieve excellence and a high degree of scientific literacy while maintaining a sense of wonder about the world around them."  (Ontario Ministry of Education. (2008), The Ontario Curriculum Grades 9 and 10: Science).

I communicate the wonder of our world  every day in my classroom. I enjoy the wonder of creation stories, and I leave it to various religions to explain how their version of the creation story fits with the scientific evidence that we all equally share.

It is my goal to create a classroom environment that is safe, positive and equitable. I include creation stories in my teaching to provide context for my lessons and to broaden intercultural understanding of my students . It is my goal to celebrate individual differences and to teach students to enjoy the diversity of cultures in our world.

TEDTalks - the importance of relationships

There has been some discussion in my department surrounding  TEDTalks. As I mentioned in a previous post, last month I attended the Ignite Mississauga, a yearly youth TedTalks event.

The first TEDTalk I want to share will make you all warm and fuzzy, the speaker is Rita Pierson and she says that every student needs a champion. Her best line is "Kids don't learn from people they don't like." This is especially true for kids with low achievement records. The way to build relationships starts with simple things:

• Trying to understand first before judging
• Apologizing when things are done wrong and,
• A persistent belief in the capability of the students. Yes, even to the point of nagging our students to be the best they can possibly be.

 The second TEDTalk is a bit long with good points at the end. Geoffrey Canada talks about the slowness of our school system to adapt to the needs of their customers. We all know the modern school model:

• Five days a week
• 8 to 3 pm
• Five periods a day
• September to June

Geoffrey's point of view is shaped by the poverty-stricken neighborhoods he serves in Harlem, NYC. Geoffrey offers a customized school model offering services that his community's specific needs:

• Extended-hours charter schools
• A chef who prepares healthy meals
• Health and parenting clinics
• Community centers for children and adults during after-school, weekend and summer hours

He makes good points, especially regarding meeting our students where they are at.

The greatest influence on student learning - John Hattie

What has the greatest influence on student learning?

Yesterday at a professional learning day we looked at the work by John Hattie. Hattie, a Professor of Education at Auckland University in New Zealand has published impressive papers involving meta analysis on student learning. Meta-analysis is commonly found in medicine and epidemiology, where for example, it is used to synthesize the effects of drugs in clinical trials. Hattie uses meta-analysis to provide a birds-eye view of effective education across the world. Hattie’s meta-analysis is fascinating in its sheer breadth. His work draws on "a total of about 800 meta-analyses, which encompassed 52,637 studies, and provided 146,142 effect sizes [...] these studies are based on many millions of students" (Hattie, 2009; 15).

Hattie's bottom line is the "effect size". An effect size of "1" indicates that a particular approach to teaching or technique advanced the learning of the students in the study by one standard deviation above the mean. So an effect size of "1" is very good indeed.

• Reverse effects in red are self-explanatory, and below 0.0
• Developmental effects are 0.0 to 0.15, and the improvement a child may be expected to show in a year simply through growing up, without any schooling. (These levels are determined with reference to countries with little or no schooling.)
• Teacher effects "Teachers typically can attain d=0.20 to d=0.40 growth per year—and this can be considered average"  ...but subject to a lot of variation.
• Desired effects are those above d=0.40 which are attributable to the specific interventions or methods being researched.

You can see in the normal distribution below, that the mean effect size is 0.4 and the desired effect size is therefore greater than 0.4. Very few effects are above 1.0, called the 'very good' category.

I included a detail of the top 20 or so areas with the most pronounced effect on students learning below. Formative assessment (assessment 'for learning' and 'as learning') matters as much as I thought it would, with an effect of 0.9. I know that effective feedback, delivered at the right time and in the right way enables me as a teacher to change I am doing AND helps the student understand their achievement of the learning goals.




What surprised me in the list above, was that student self-expectations rank the highest in effect, with an impact of 1.44, and that study skills appear more than 20 lines down the list with an effect size of 0.59. 


References
Hattie's two research papers, Influences on Student Learning and Teachers make a Difference, explore what it is that really makes a difference to learning in our classrooms, and the difference between experienced teachers and expert teachers. His book Visible Learning synthesised these results. Hattie's webpage brings together the freely available online resources related to John Hattie’s Visible Learning research  http://visible-learning.org/.