Tag Archives: evaluation

When She was Good…

image by Marie Rayner

she was very, very good, and when she was bad, she was horrid! This a line from a nursery rhyme I like to say to my daughter and it always pops into my mind when talking about what it means to be a good instructor.

The purpose of evaluation of teaching is to identify good teaching practices and weaknesses to be improved on. This raises the question of “what are good teaching practices?” and “what makes an instructor good?”

The definition of good is going to be subjective, which makes evaluation more challenging. My definition of a good instructor is going to be different from a student which is going to be different between students which is going to be different from an administrator.

This is no small problem when it comes to evaluating our teaching, especially if we are asking for feedback from students. In order for the evaluation to be valid, the assessment tool must measure what we want to measure. If I want to measure my effectiveness as an instructor based on my definition of a good instructor, then I first have to determine what is good and then determine if the assessment tool gives me the information needed to reach a conclusion on how good I am. This would all go out the window if the students who are doing the evaluation do not share my vision on what makes a good instructor.

In 2005, Edith J. Cisneros-Cohernour has published a paper in “The Quality of Higher Education” called “Validity and evaluations of teaching in higher education institutions under positivistic paradigm“. In this paper, Cisneros-Cohernour applies the aspects of construct validity proposed by Messick to student evaluation of teaching. I found this application very insightful in thinking about the research surrounding student evaluation of teaching. The second part of the paper explores the limitations of using student evaluations to make decision about the effectiveness of teaching. This paper provides fodder for all kinds of blog posts, but let’s start at the first step. What is my definition of a good instructor?

I’m doing this one first because it is what I can control. The definition of good held by my students and administrators would just be speculation. I’m also not sure if I should care about what others think of my teaching if I am happy with my teaching. I suspect that is naive, and I should consider the other stakeholders point of view sooner or later, but for now, this is my blog so I’m starting with me.

If my goal is to learn how to be a good instructor, I think I should apply the same principles I use for designing a course of study for my students. I have a goal, so what are the desired outcomes?How would I know if I’m a good instructor? The measurable characteristics of a good instructor for me would include:

  • empathize with the difficulties of student learning.
  • design learning activities that help students overcome these difficulties.
  • evaluate the effectiveness of supplied learning activities.
  • modify learning activities as needed.

In his book, “The Skillful Teacher”, Brookfield explains 3 core assumptions of skillful teaching that I think match my outlook. I would equate skillful teaching with good teaching. All 3 core assumptions are interrelated with each other. Brookfield’s first assumption is that “skillful teaching is whatever helps students learn.” The challenge here is for the instructor to give up any preconceived ideas of what is helpful and be open to anything that helps students learn, even if these methods seem counter-intuitive or fly in the face of tradition. The second assumption is that “skillful teachers adopt a critically reflective stance towards their practice”. The critically reflective process strives to use informed pedagogic actions that are based on research. Skilled teachers should be able to explain why they do what they do and provide evidence to support that process. The second part of the critical reflective process is to view our practice from different perspectives or the “4 complementary lenses”: students’ eyes, colleagues’ perceptions, literature, and our own autobiography. The third assumption is “the most important knowledge skillful teachers need to do good work is a constant awareness of how students are experiencing their learning and perceiving teachers’ actions”. Brookfield also highlights the difficulty of evaluating how students are perceiving their learning experience. He suggests classroom assessment techniques like the one minute paper or the muddiest point to reveal student difficulties in learning the course concepts. He also uses an instrument he calls “critical incident questionnaires” to become aware of how students feel about his course.

So as I explore student evaluation I’m finding all kinds of tangents that are related to this concept. So many paths to take such as:

  • student definition of a good instructor
  • administrator definition of a good instructor
  • methods to evaluate teaching such as critical incident questionnaires
  • additional reflection on the content of Brookfield’s book

I like how inspiring this type of exploration can be, but I also feel like I’m running around like a chicken with her head cut off. I’m constructing a mental map of each area where I’ve been to see if at the end of the journey, a coherent picture starts to emerge.

 

Leave a comment

Filed under Evaluation

Feedback from Students

The evaluation process not only includes evaluation of the student learning but also evaluation of instructor teaching. My institution has a form that is given towards the end of the course called “Student Report on Teaching” (SRT). SRTs are given to students during the probationary period and then periodically after that. I haven’t seen an SRT since my probationary period over a decade ago. The infrequency of evaluation on the institutional level has left me interested in doing my own SRT.

My experience with the SRTs during my probationary period has left me with some concerns which suggests changes to the current form. My major concern involves what to ask on the SRT in order to gain useful feedback. Our institutional questions include questions like “Were tests and assignments graded and returned promptly”, which lead me to stay up all night grading exams so that I could return them during the next lecture. I was shocked to be rated negatively in response to this question! How could I possibly get exam results back any quicker than the very next class which was sometimes less than 24 hours after they wrote the test? Another question asks about whether course objectives were clear. I included the course objectives in the course outline and at the beginning of each lecture. I specifically asked at the end of the lecture if the course objectives were clear. Again, I was shocked to be rated negatively in response to this question on the SRT. If the course objectives were not clear, why didn’t students speak up during the course? I did not get the results until after the course was over and without student names so I could not follow-up on the concerns raised by the SRT. What is the point of doing evaluations if the information is untimely and of such poor quality, improvements cannot be made?

I found unsolicited student evaluations on a site called “Rate My Professor”.  While it was nice to receive positive feedback in such a public place, I found the negative feedback frustrating. Students leaving negative feedback often had the course number wrong which made it hard to take their comments about the course seriously. The students would also express dissatisfaction with my personal attributes without explanation as to why they felt I was rude or mean or evil or whatever else they didn’t like about me. One student even accused me of being a racist which I consider to be quite a serious charge and absolutely something I do not want to be. However, the student never said anything to me or filed a formal complaint. I have no idea what I did to deserve such a comment, and thus, I have no idea how to rectify the situation or even verify if a situation actually exists to rectify.

Despite this bad taste in my mouth, I still believe there is a need to gather student feedback. Thus I was interested in the Faculty Focus newsletter in my email this morning. In her article, “How to Get Better Feedback from Students“, Maryellen Weimer offers some tips:

  • Ask questions “about the impact of a policy, practice, behavior, technique, assignment, or instructional approach on students’ efforts to learn” rather than what students liked/disliked about the course and/or instructor.
  • Ask these questions immediately after the activity was experienced by the student during the course rather than a global survey at the end of the course when it is too late to do anything.
  • Teach students how to give constructive feedback. This instruction would include what is relevant and appropriate feedback by explaining what can be changed and what can’t as well as modeling the desired behaviour when giving feedback to students.
  • Show students that you take their feedback seriously by discussing the feedback with them. If feedback cannot be incorporated, explain why and invite other solutions to identified problems. I would even go farther and suggest not asking for feedback on anything you have no desire to change.

The most difficult tip to incorporate is asking the right questions and this needs to be explored further in future blog posts. I have several tabs open, so stay tuned.

Leave a comment

Filed under Classroom Management, Learning Environments

Setting Achievable Goals with Rubrics

Rubrics are generally considered a good thing since they simplify grading for instructors and provide feedback for students. There are a lot of resources on rubrics and how to construct them. This is one example from Harvard University and this is a link to an index page for more resources on rubrics. From the index page, we can even find a rubric for rubrics!

I like rubrics as a motivational strategy since it conveys expectations. In her book “Student Engagement Techniques”, Barkley says, “Students must have confidence that, with appropriate effort, they can succeed. If there is no hope, there is no motivation” (p. 11).

This blog is a class assignment and it is graded using a rubric.  This is the expectation for the highest level of achievement for this blog:

Level ofMastery Level 4 A superior, consistent performance; beyond expectations
Blog Project60 % The Blog reflects a professional product that could be used as an educational resource. Content, links, resources and media are substantive and reflect breadth and depth. A wide range of media is evident throughout the Blog.
Organization AndLayout

35 %

Organization and layout requirements are met. The document has a professional look and feel
Writing skills5% Writing is free of grammatical, spelling and punctuation errors.

 

The only grade here is 10/10. Perfect. Who is perfect? But it is worse than that, because the level of mastery here says “beyond expectations”. If the expectation is perfection, how can we exceed that? Furthermore, we are only expected to spend 8 – 10 hours week working on this course, which includes other activities besides this blog. I can spend 8-10 hours on one blog post alone. How am I to achieve substantive content, links, resources and media within that time frame? For organization and layout, the document should have “a professional look and feel”. Well, thank goodness WordPress makes a professional layout easy, but I’m still not a professional writer. Besides, what does “professional” look and feel like anyway? The last criteria is that “writing is free of grammatical, spelling and punctuation errors.” Yeah, right. As if that is going to happen without a copy editor and even with a copy editor, mistakes still happen.

So if we go back to our rubric about rubrics, the blog assignment rubric gets a poor since “expresses goals that are unclear, unattainable, or unrealistic“, “offers judgments that are merely opinions“, and “describes characteristics that are not age appropriate “. So how would I fix this, if I was the instructor? The description of criteria for each part of the assignment needs to be more objectively defined. For example, how many posts should be included? What constitutes depth and breadth (for me depth would be using the higher levels of Bloom taxonomy and breadth would involve using 3 or more sources in the post)? What format should the posts be in (text, video, audio, cartoons)? How many of each? Should these formats be self-generated or just include sharing the work of others? How long should the posts be?

I think the description of perfection is useful to convey the instructor’s vision of the assignment in terms of goals but not as an evaluation category. I see value in striving for perfection and knowing what that looks like helps, but as evaluation category perfection is unattainable. There is no hope, so why even try? Indeed, I had a mini-meltdown when I read this rubric at the beginning of the course, until I told myself I didn’t need to be perfect. Level 3 is good enough.

Levelof

Mastery

Level 3 A solid consistent performance; demonstrated competency of knowledge and skills
BlogProject

60 %

The Blog demonstrates a comprehensive perspective of the topics, however, more attention to detail could have been demonstrated in regards to content, links and resources.
OrganizationAnd

Layout

35 %

Organizational requirements met (i.e., thinking and content). Issues exist re structure and layout of material.
Writing skills5% Writing contains minor grammatical, spelling and punctuation errors.

 

The blog to me is a continuing project which helps me evolve as an instructor. The blog gives me a place to collect ideas and techniques, explore my thoughts, take risks, modify my approach and document this journey. A daily post forces me to keep thinking about my teaching and allows me to convey my dedication to my craft. The blog will never be done.

More importantly, I decided this blog is for me, not some grade. I’m also not out to seek fame. If people want to come along for the journey, they are most welcome and I appreciate all comments and shares. I find the thought of self-promotion repulsive in its arrogance. If a fellow instructor asks me a question, I will feel compelled to share what I have learned but I just don’t think that I’m all that interesting or brilliant to promote this blog without prior solicitation.

Leave a comment

Filed under Motivational Strategies