The Literacy of Critical Evaluation: Why Rating is Not Good Enough

Essential Question: How Do We Teach Critical Evaluation in ELA?

source

Realization phase: I realized today that the average book rating on Amazon is over 4 stars.  I am fairly sure that the average app rating in iTunes is also over 4 stars. Zappo boots have an average overall rating of over 4 stars as well.  How can these ratings possibly be useful?  Yet they are embedded in today's concept of "smart shopping." 

The concept of an overall rating is dangerous.  Consumers - of products or information - who rely upon the "star reports" are not thinking critically or evaluatively, as they believe they are doing.  They are thinking quantitatively and shallowly.  Unfortunately, the ubiquitous nature of crowd-sourced rating systems today means that our students are participating in, and taking advice from, a thinking habit that is dumbing down their own ability to think and evaluate critically.

Insight phase: Critical thinking habits can be and need to be addressed in the classroom, or we run the risk of raising a group of satisfied sloppy thinkers.  [And those thinkers will, in some states, be rating teachers. In all states, they will be voting.]

Parenting groups are actively promoting the teaching of critical thinking.  For example, the issue is confronted directly in recent articles:

• How to teach your kids critical thinking,  
• Teaching Critical Thinking Skills to Children, 
• Teaching Critical Thinking - a Parenting Science Guide,
• and Critical thinking in children: Are we teaching our kids to be dumb?.  

Aside from psychology professors, parents are the most prevalent online writers about this topic. They want their children to be "brainy" and successful.  They are not seeing that schools will do the job.

And they are right to be concerned.  Although critical thinking is identified as a key 21st century learning target,  and it is probably going to be embedded in new standards-based assessments, it is clearly difficult to pin-point how to develop it, much less to get good advice from experienced educators.  There is even sigificant disagreement about whether or not critical thinking can and should be addressed before the middle level years (clearly, the parent-educators have no such qualms - and I agree with that).  So what is the target for developing critical evaluation as a habit of mind?

source

Learning phase: My learning journey today in this area leads me to identify these elements of critical thinking:

• Writing - Persuasive/Arugmentative especially
• Questioning - questions asked before intereacting with text/media; questions asked during and after
• Debating
• Connection to authentic experience (life)
• Creativity
• Investigation or research
• Systematic truth-seeking
• Fair-mindedness (consideration of the opinions and insights of others, consideration for point of view)

In the ELA classroom, these elements need to be applied to literacy tasks, specifically reading text (including visual and media text) and writing about this text.  These are habits of mind, not skills.

As spelled out in Paul and Elder's The International Critical Thinking Reading & Writing Test (.pdf sample pages from The Critical Thinking Community), close reading and substantive writing require skills that enable students to take ownership of the ideas in a text.  These skills are:

1. Clarify purposes: an author’s purpose(s) (when we read), and our own purpose(s) (when we write).
2. Formulate clear questions: those that an author is asking (as we read) and those we are pursuing (as we write).
3. Distinguish accurate and relevant information from inaccurate and irrelevant  information: in texts that we read and in preparation for our own writing.
4. Reach logical inferences or conclusions: based on what we read, and in preparation for writing.
5. Identify significant and deep concepts: those of an author and those we want to guide our thinking while we write.
6. Distinguish justifiable from unjustifiable assumptions: that an author is using, or that we are using in our own thinking as we write.
7. Trace logical implications: those of an author’s thinking, and those that may follow from our written work.
8. Identify and think within multiple viewpoints: those that an author presents (or fails to present when relevant) and those relevant to the issues in our written work (and that we need to include).

Critical evaluation (phase) is the end product of the application of these elements.  The materials available at The Critical Thinking Community site (all for sale, some freely available) will provide the district or school with clear guides to improving student critical thinking.  If this is not an option, I recommend that all ELA teachers in the US should study the sample Critical Reading/Writing "text" - with anchor answers - for The Declaration of Independence (it is contained in the .pdf file).  This would be a great professional development or PLC activity.

Also of great use to ELA teachers are the following tools provided on The Critical Thinking Community site:

• an overview of The Elements of Thought - an expansion of the pie above,
• 35 Dimensions of Critical Thought,
• remodeled lessons, which can be accessed through the navigation bar at the 35 Dimensions page (lessons reference the strategies) - I especially liked the remodeled Journal and Argumentative Writing lessons in the Junior High School section.

source

Simplification: All of this might be to complex for some teachers, however.  For these, a focus on Bloom's Taxonomy Revised (.pdf) is in order.  The top two tiers apply directly to critical evaluation:

• Evaluating: Making judgments based on criteria and standards through checking and critiquing.
• Creating: Putting elements together to form a coherent or functional whole; reorganizing elements into a new pattern or structure through generating, planning, or producing. 

The above explanation is from Mary Forehand - Bloom's Taxonomy.  Notice that producing is only one element of creating.  ELA teachers should not focus overly on creative projects or products - this emphasis is a flaw in most information about the Creating element.  Yes, it is the most fun part for students - the whole benefit of critical thinking is lost if the evaluative step is not embedded in the creative step.

For example, if the Essential Question for an ELA unit is the one I posed in my last post, Why are zombies compelling?  students might create a zombie movie as a culminating project.  Does this demonstrate critical understanding of the genre or the question?  Nope.  On the other hand, a video interview with the director, cast, and screenwriter of that imagined movie would be able to demonstrate a critical understanding of the question.

The key, therefore, is twofold (no surprises here, but reminders are in order):

1. The texts must be worthy of critical evaluation - complex, challenging, accessible on multiple levels, question-raising
2. The student tasks must (post-Analysis) address the Evaluating and Creating tiers of Blooms.  Paul and Elder strongly support the argument that the best of these tasks are also social in nature.  In other words, the culminating topic or assignment must be evaluative in nature. 

Conclusion: Which brings us back to ratings.  Critical shoppers ignore the overall rating and go directly to the reviews, which they read critically (Is there an agenda here? Does the review address the product? Does it provide me with real information? etc.).  Or goal is for students to always do the same.

Practical applications:

• Taking a page from my Zombies unit, students could critically evaluate one of the texts by Brooks, a children's book, and one of the web texts - does it get better than that?
• Detailed rubrics can provide students with some language for critical evaluation. Use them often and honestly, and insist that students read the evaluation text! Unfortunately, they are not readily available. 
• It would be interesting to design a year-long critical evaluation focus with the thematic frame of Ratings Defended.  How would this be done?  Students would have to not only rate texts, responses to text (each other), models, etc. - they would be required to defend and explain each rating.  I can't think of a better use of:

• journaling
• wiki or blog book reviews
• classroom debates
• essays - even the 5-paragraph essay can be reframed in this model - this idea is explored in remodeled lessons at The Critical Thinking Community .  I have written about it also in this post on iSearch  essays. (also called iLearn)

I think this would eventually improve the whole value of social rating.  And I personally would begin as early as pre-K, where "thumbs up" is still a valid rating system. 

Resources to read and intrigue you:

• Critical Thinking: Why is it so hard to teach?  (Daniel Willingham) (.pdf download)
• Try this new widget - it is supposed to provide a critical thinking question every day (as well as to model the best forms of questioning):