The Literacy of Critical Evaluation: Why Rating is Not Good Enough

Essential Question: How Do We Teach Critical Evaluation in ELA?

source

Realization phase: I realized today that the average book rating on Amazon is over 4 stars.  I am fairly sure that the average app rating in iTunes is also over 4 stars. Zappo boots have an average overall rating of over 4 stars as well.  How can these ratings possibly be useful?  Yet they are embedded in today's concept of "smart shopping." 

The concept of an overall rating is dangerous.  Consumers - of products or information - who rely upon the "star reports" are not thinking critically or evaluatively, as they believe they are doing.  They are thinking quantitatively and shallowly.  Unfortunately, the ubiquitous nature of crowd-sourced rating systems today means that our students are participating in, and taking advice from, a thinking habit that is dumbing down their own ability to think and evaluate critically.

Insight phase: Critical thinking habits can be and need to be addressed in the classroom, or we run the risk of raising a group of satisfied sloppy thinkers.  [And those thinkers will, in some states, be rating teachers. In all states, they will be voting.]

Parenting groups are actively promoting the teaching of critical thinking.  For example, the issue is confronted directly in recent articles:

• How to teach your kids critical thinking,  
• Teaching Critical Thinking Skills to Children, 
• Teaching Critical Thinking - a Parenting Science Guide,
• and Critical thinking in children: Are we teaching our kids to be dumb?.  

Aside from psychology professors, parents are the most prevalent online writers about this topic. They want their children to be "brainy" and successful.  They are not seeing that schools will do the job.

And they are right to be concerned.  Although critical thinking is identified as a key 21st century learning target,  and it is probably going to be embedded in new standards-based assessments, it is clearly difficult to pin-point how to develop it, much less to get good advice from experienced educators.  There is even sigificant disagreement about whether or not critical thinking can and should be addressed before the middle level years (clearly, the parent-educators have no such qualms - and I agree with that).  So what is the target for developing critical evaluation as a habit of mind?

source

Learning phase: My learning journey today in this area leads me to identify these elements of critical thinking:

• Writing - Persuasive/Arugmentative especially
• Questioning - questions asked before intereacting with text/media; questions asked during and after
• Debating
• Connection to authentic experience (life)
• Creativity
• Investigation or research
• Systematic truth-seeking
• Fair-mindedness (consideration of the opinions and insights of others, consideration for point of view)

In the ELA classroom, these elements need to be applied to literacy tasks, specifically reading text (including visual and media text) and writing about this text.  These are habits of mind, not skills.

As spelled out in Paul and Elder's The International Critical Thinking Reading & Writing Test (.pdf sample pages from The Critical Thinking Community), close reading and substantive writing require skills that enable students to take ownership of the ideas in a text.  These skills are:

1. Clarify purposes: an author’s purpose(s) (when we read), and our own purpose(s) (when we write).
2. Formulate clear questions: those that an author is asking (as we read) and those we are pursuing (as we write).
3. Distinguish accurate and relevant information from inaccurate and irrelevant  information: in texts that we read and in preparation for our own writing.
4. Reach logical inferences or conclusions: based on what we read, and in preparation for writing.
5. Identify significant and deep concepts: those of an author and those we want to guide our thinking while we write.
6. Distinguish justifiable from unjustifiable assumptions: that an author is using, or that we are using in our own thinking as we write.
7. Trace logical implications: those of an author’s thinking, and those that may follow from our written work.
8. Identify and think within multiple viewpoints: those that an author presents (or fails to present when relevant) and those relevant to the issues in our written work (and that we need to include).

Critical evaluation (phase) is the end product of the application of these elements.  The materials available at The Critical Thinking Community site (all for sale, some freely available) will provide the district or school with clear guides to improving student critical thinking.  If this is not an option, I recommend that all ELA teachers in the US should study the sample Critical Reading/Writing "text" - with anchor answers - for The Declaration of Independence (it is contained in the .pdf file).  This would be a great professional development or PLC activity.

Also of great use to ELA teachers are the following tools provided on The Critical Thinking Community site:

• an overview of The Elements of Thought - an expansion of the pie above,
• 35 Dimensions of Critical Thought,
• remodeled lessons, which can be accessed through the navigation bar at the 35 Dimensions page (lessons reference the strategies) - I especially liked the remodeled Journal and Argumentative Writing lessons in the Junior High School section.

source

Simplification: All of this might be to complex for some teachers, however.  For these, a focus on Bloom's Taxonomy Revised (.pdf) is in order.  The top two tiers apply directly to critical evaluation:

• Evaluating: Making judgments based on criteria and standards through checking and critiquing.
• Creating: Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning, or producing. 

The above explanation is from Mary Forehand - Bloom's Taxonomy.  Notice that producing is only one element of creating.  ELA teachers should not focus overly on creative projects or products - this emphasis is a flaw in most information about the Creating element.  Yes, it is the most fun part for students - the whole benefit of critical thinking is lost if the evaluative step is not embedded in the creative step.

For example, if the Essential Question for an ELA unit is the one I posed in my last post, Why are zombies compelling?  students might create a zombie movie as a culminating project.  Does this demonstrate critical understanding of the genre or the question?  Nope.  On the other hand, a video interview with the director, cast, and screenwriter of that imagined movie would be able to demonstrate a critical understanding of the question.

The key, therefore, is twofold (no surprises here, but reminders are in order):

1. The texts must be worthy of critical evaluation - complex, challenging, accessible on multiple levels, question-raising
2. The student tasks must (post-Analysis) address the Evaluating and Creating tiers of Blooms.  Paul and Elder strongly support the argument that the best of these tasks are also social in nature.  In other words, the culminating topic or assignment must be evaluative in nature. 

Conclusion: Which brings us back to ratings.  Critical shoppers ignore the overall rating and go directly to the reviews, which they read critically (Is there an agenda here? Does the review address the product? Does it provide me with real information? etc.).  Or goal is for students to always do the same.

Practical applications:

• Taking a page from my Zombies unit, students could critically evaluate one of the texts by Brooks, a children's book, and one of the web texts - does it get better than that?
• Detailed rubrics can provide students with some language for critical evaluation. Use them often and honestly, and insist that students read the evaluation text! Unfortunately, they are not readily available. 
• It would be interesting to design a year-long critical evaluation focus with the thematic frame of Ratings Defended.  How would this be done?  Students would have to not only rate texts, responses to text (each other), models, etc. - they would be required to defend and explain each rating.  I can't think of a better use of:

• journaling
• wiki or blog book reviews
• classroom debates
• essays - even the 5-paragraph essay can be reframed in this model - this idea is explored in remodeled lessons at The Critical Thinking Community .  I have written about it also in this post on iSearch  essays. (also called iLearn)

I think this would eventually improve the whole value of social rating.  And I personally would begin as early as pre-K, where "thumbs up" is still a valid rating system. 

Resources to read and intrigue you:

• Critical Thinking: Why is it so hard to teach?  (Daniel Willingham) (.pdf download)
• Try this new widget - it is supposed to provide a critical thinking question every day (as well as to model the best forms of questioning):

triskaidekaphobia

Why 13? That is going to be my 1st 1LQ  (1-laptop-question).  Actually, we are doing this without laptops, as I will not be using them until students have completed their "responsibility training."

There are 13 sections in Seedfolks, representing 13 main characters, 13 points of view.  Fleischman could have selected any number, but he selected 13. I have no idea why, although I will be making my own guesses (sharing "Thirteen Ways of Looking at a Blackboard", the power of the prime number, the power of the shape of the number itself). The point is: We should celebrate these questions and their answers.  Why NOT 13?

With the 1LQ sessions (5-8 minutes, group-work), we will begin to use my Question Matrix as the core tool for LA 7.  As we move beyond tomorrow's discussion into a focused reading of the book (Reader's Workshop model - we will begin by reading for Characterization and How It is Developed), the matrix will be used to develop questions that will make our reading deeper.  This upcoming week is for practice, so although we have already crossed out "Who is...", we will leave the other cells untouched until next week.  After this, students will be generating, and answering, questions for HW and during class, "using up" cells as they do so. It is my anticipation that comprehension and deep-reading skills will develop, as will the ability to recognize, highlight, and discuss pertinent supporting detail. 

Question Matrix

This is test-prep.  It is also prep for a life of thinking.  I have decided on this very simple core tool this year because there is not enough time in the teaching year to be complex.  Learning is not about one question-one answer. It is as complex as the simple question, Why 13?  The complexity of teaching is helping each individual student find a path through the content, processes, and ideas of the year.  This is the most important walking stick I will offer.

On Watching the Game

The Jets are winning 14 to 7, and the Bengal offense is talking on the bench. That is what catches my interest. The QB and the line are talking about what is happening and what it means.  They are not watching the Jets move down the field another time - they are discussing and considering and arguing about what they need to do.

We think football players are dumb (as in mute). These are not. These players know what it means to use language to solve a problem, to use language to move forward.  To win (hopefully).

Isn't this dialoguing a major goal of education?  (rhetorical question).

How can we bring our students to this point if we do not require them to have/practice these discussions about the situations, jobs, readings, experiences, activities, units which we arrange for them or arrange for them to arrange for themselves? 

I reflect on myself as both an athlete and a student (I was very good at both before college hit me for a loopy loop), but I did not talk much and no one required me to do it.  This was a problem.  In today's world, I would have been a Facebook all-nighter and had an iPhone in my pocket.  But I would still be avoiding f2f communication.  I learned to use and appreciate language by longing for it and studying it, I think.  But even today I often short-change myself f2f.

Where I am going is this: We need to put our students on teams and make success depend upon active, focused, informed communication.  I am tired of the pedagogy that says that students should all be doing or reading their own things, then sharing their learning.  This model is not a jig-saw puzzle or an engagement bullet.  It is a mush that more often that not is intensely boring to students and teachers alike.  Yet we are told to smile and nod and complement the individual, who learns nothing from feedback and who is engaged no further than she has engaged herself - which is too often too little.

We all learn by sharing ideas about the SAME content. That is what blogging, wikis, social networks, and football teams are all about.

Why can't education get it right?