Asbestos and Extinctions

Over the Xmas break various family members were engaging in an interesting conversation whose starting point was the way many people are excessively, indeed sometimes hysterically concerned about the dangers of asbestos fibres from nearby demolitions or renovations.

The background theme was how poorly people understand risks, especially small risks, and how they misplace their anxieties about risks.

I suggested that the emotional energy people put into obsessing about floating asbestos fibres would be better invested doing something about much larger dangers such as, say… global warming.

To put things in a bit of perspective, consider:

“This end-Permian extinction is beginning to look a whole lot like the world we live in right now,” Payne said. The end-Permian extinction (mentioned in an example in the previous post on this blog) was a catastrophe 250M years ago when the great majority of land and marine species were eliminated.

Payne is “assistant professor of geological and environmental sciences at Stanford University… a paleobiologist who joined the Stanford faculty in 2005, studies the Permian-Triassic extinction and the following 4 million years of instability in the global carbon cycle.”

Payne says: “The good news, if there is good news, is that we have not yet released as much carbon into the atmosphere as would be hypothesized for the end-Permian extinction. Whether or not we get there depends largely on future policy decisions and what happens over the next couple of centuries.”

See The Day the Seas Died: What Can the Greatest of All Extinction Events Teach Us About Climate Change?

On Buying Cheese

The current issue of Choice Magazine (the Australian “Consumer Reports”) has a report on cheddar cheese.

They had five experts blindly rate 28 cheddar cheeses, ranging from your cloth- or wax-wrapped special deli cheddar at $50+ dollars per kilo down to the supermarket brands, sometimes less than $10 per kilo.

Eyeballing the results table, it seemed that price wasn’t a reliable guide to quality – some good cheeses were quite cheap and vice versa.

In the results table, they listed overall quality (score out of 20) and price per kg. They didn’t offer a “value for money” rating, so I copied the table into Excel and had it compute “value for money” as quality divided by price.

Now that the data was in Excel, we could probe a little further.

Turns out the correlation between quality and price was -.05. In other words, the quality of the cheese you buy, on average, has virtually nothing to do with price. If anything, as you go up in price, it gets worse.

Consequently, the correlation between quality and value for money was abysmal: -.8. In other words, on average, the more you pay, the more you’re getting ripped off.

Some cheeses had long names with lots of fancy-sounding words, such as “Devondale Special Reserve Premium Aged Vintage.” That must be a good cheese, right?

I used Excel to count the characters in a cheese’s name. Running the correlations showed that length of name bears little if any relation to price, quality, or value for money.

Conclusions: buying cheddar cheese is a lottery. If you haven’t tasted the cheeses, and are just trying to guess which ones are good, ignore price and fancy names; these have nothing to do with quality. If you want value for money, go for the cheaper cheese.

In short: when buying cheddar cheese in Australia, it just isn’t true that “you get what you pay for.”

PS – the cheese I’ll buy: South Cape Vintage Black Label. Nearly the top in quality, but only $15 a kilo.

Powered by ScribeFire.

Pre-structured maps of legal arguments

Peter Tillers discusses why DNA can never be regarded, on its own, as conclusive evidence of guilt or innocence. 

This post makes me wonder about the possibility of a kind of schematic argument map showing how the argument from say a DNA match to guilt would have to go in some more-or-less general version.  This map would display the numerous inferential steps, assumptions etc. – i.e., the numerous points at which the inference might fail.

John Burns, who at the time was quite senior in the Hong Kong police and had experience in training detectives, proposed this kind of idea in masters dissertation.  He called them “pre-structured argument maps”.  You would have such a map for each typical situation in which a detective might be trying to make the case for guilt, e.g., one for shoplifting.  The pre-structured map would embody (a) a good understanding of the overall structure of the case that would have to be made out, and (b) the accumulated wisdom of experienced detectives as to all the bases that need to covered – e.g., the detective would have to have evidence to rebut the defendant’s claim that he already owned the item. 

Then, rather than piecing together a case (whether in argument map form, or more traditional format) from scratch, the detective would check off the various aspects of the case on the pre-structured map, removing parts which are inapplicable to the particular situation, etc.. Along the way the detective would be learning what a good case looks like, being exposed to the myriad ways in which the case might be defeated by a clever lawyer, etc.

Powered by ScribeFire.

Support is transitive?

On the news tonight there was coverage of protests in Washington against the Iraq war. There was a soundbite of an Iraq veteran saying “You can’t support the troops and oppose the war, because the troops support the war.”

These thoughts flashed through my mind in quick succession:

1. Argument.
2. Argument, very concisely expressed.
3. Bad argument.
4. Bad, but interesting.

Why interesting? Well, consider what this fellow must be assuming. Put another way, what co-premise would, if true, make this argument strong?

Presumably something like, “if you support somebody, you have to support what they support”.

This is similar to the technical notion of transitivity: if A supports B and B supports C, then A supports C. Conversely, if A doesn’t support C, then A doesn’t really support B.

So we get:

Now it seems to me that this assumption is obviously wrong as a general principle. For example I can support my child without thereby being obliged to adopt whatever ill-considered attitude they might adopt.

From a “critical thinking” perspective, the argument is really a “fallacy of equivocation” – i.e. an argument that is fallacious because it “equivocates” on a key term, meaning that it uses a key term in different ways in different places.

The term “support” means one thing when you talk about supporting the troops, and another thing when you talk about supporting or opposing the war.

But there is a deeper issue here – the idea that allegiance to a group requires allegiance to the beliefs of that group. Something profound (and presumably of evolutionary origin) in the human psyche makes us tend this way. Many human organisations promote the idea and owe their continued existence to its power. But it is of course a dangerous idea.

Critical Thinking – Where to start?

“Just Some Guy” wrote today:

I recently stumbled on an excellent online article authored by yourself entitled “Teaching Critical Thinking“. I was wondering if you could take a moment of your valuable time to suggest a couple of books on the subject. I would like improve my critical thinking skills so I suppose the focus sought would be adult learner skill(s) acquisition with emphasis on techniques and (lots of) practice. I have been trying to develop said skills on my own (without much success). I would really like to have find a proven program to apply. As you know there is tons of information available online however I am getting lost trying to sort out all the wheat from chaff. Thank you in advance for your consideration.

I used to be a regular academic, and one reason for heading off in a different direction was the experience most academics know all too well, which is that you’ll slave for months on a paper, have it published, and then… nothing happens. It seems you may as well not have bothered. So it is gratifying when some paper you wrote, and which seemed to have vanished without a trace, starts to get picked up, read, and perhaps even appreciated. In the case of the paper mentioned above, in past month I’ve heard that it is the subject of a faculty discussion group at the University of Pittsburgh (where I did my PhD), and read by administrators at a startup university campus in Singapore. Now it seems to have helped Just Some Guy. Maybe it was worth the effort that went into it.

Anyway, regarding JSG’s query, in workshops I used to hand out brief annotated “further reading” list. Here it is:

There are hundreds of books on thinking and how to improve it, ranging from airport junk to turgid academic treatises. Here is a short list of some of the best, focusing on critical thinking. All are accessible, entertaining, and contain many valuable insights. Listed in alphabetical order, so don’t necessarily start at the top.

Cialdini, R. B. (1984). Influence: The Psychology of Persuasion. New York: William Morrow and Co. Classic, eye-opening description of the tricks, ruses and deceptions others use to manipulate us into doing what they want.

Giere, R. N. (1996). Understanding Scientific Reasoning (4th ed.). Fort Worth: Holt, Rinehart and Winston, Inc. Very clear overview of the fundamentals of scientific reasoning. Basic literacy in scientific methodology.

Heuer, R. J. (1999). Psychology of Intelligence Analysis. Center for the Study of Intelligence, CIA. Although intended primarily to assist intelligence analysts, there is a lot of good stuff here, on both the descriptive (how our minds work) and normative (rules for better thinking) sides. Plus, available free online!

Kepner, C. H., & Tregoe, B. B. (1997). The New Rational Manager. Princeton: Princeton Research Press. These are the people who first brought “critical thinking” to the business world and built out of it a multinational consulting firm. Very practical orientation.

Minto, B. (1996). The Minto Pyramid Principle: Logic in Writing, Thinking and Problem Solving.  Minto Books International Limited (www.barbaraminto.com). [Note: this edition supersedes the earlier edition, published by Pearson.]  Barbara Minto was a McKinsey consultant and editor; this book is now the “Bible” in this area for major consulting firms. Some profound truths about good thinking and communication, cast in a way which makes sense for folks in the business community.

Myers, D. G. (2002). Intuition: Its Powers and Perils. New Haven: Yale University Press. “Europe in ten days” tour of the ways intuitive thinking can go wrong, according to serious psychologists. Pretty exhaustive coverage, but most of it will just wash over you.

Paul, R. W., & Elder, L. (2002). Critical Thinking: Tools for Taking Charge of Your Professional and Personal Life. Upper Saddle River, New Jersey: Financial Times Prentice Hall. Paul and Elder are prominent critical thinking instructors. This book packages their insights as practical tools for personal and professional life. Stresses psychological and ethical issues, though often becomes a bit too “pop psychology”.

Piatelli-Palmarini, M. (1994). Inevitable Illusions: How Mistakes of Reason Rule our Minds. New York: Wiley. Very readable introduction to some of the most famous cognitive biases and blindspots. More diagnosis than therapy.

Salmon, M. (1989). Introduction to Logic and Critical Thinking (2nd ed.). San Diego: Harcourt Brace Jovanovich. The best of the standard undergraduate textbooks. A bit dull, but very sound.

Spence, G. (1995). How to Argue and Win Every Time. New York: St. Martin’s Press. Written by a criminal attorney who (according to the dust jacket) never lost a case. If you can look beyond the very “American” style, there is much wisdom here. It is a treatise in the art of rhetoric, but it is principled rhetoric rather than mendacious sophistry.

Whyte, J. (2004) Crimes Against Logic. McGraw-Hill. A short introduction to “fallacies,” i.e., common patterns of bad reasoning. Whyte runs through about a dozen, but there are dozens of others. Witty, fast-moving and brief.

Yep, that’s what we do…

On the AILACT list, Michael Scriven wrote:

Mark got in a dig about ’speed reasoning’ my most popular course; perhaps I should mention that the first thing I say in the first session is, there’s no royal road to speed reasoning, you just have to become good at plain old slow reasoning first, and then do it a thousand more times, and you’ve mastered speed reasoning. BUT WE CAN HELP with the first part, by giving you a nice bunch of tools, beginning with argt structuring, plus a number of templates for patterns to spot as problematic, plus some neat ways to counter those, and now let’s see how that works in ten subject matter fields, and then we’ll test you on five other ones to see if you’ve ‘got the point’ See… it’s easy to improve!

As it happens, that’s a pretty good description of the pedagogical approach in our “Critical Thinking: The Art of Reasoning” subject at the University of Melbourne. Though I’d emphasize that we don’t just begin with argument structuring, we use argument structuring in diagrammatic form (i.e., argument mapping) throughout the subject.

Talk at Victorian Skeptics

Anyone likely to be in Melbourne on Feb 19 is welcome to join the Victorian Skeptics for an informal talk:

The abstract is

Academic philosophers, like most professionals, think they’re pretty good at what they do. I’ll present some general reasons for scepticism on this score. Then I’ll focus on one particular respect in which philosophers think they’re pretty good – teaching critical thinking. I’ll show detailed empirical evidence on critical thinking skills gains, which suggests that if you want students to get better at critical thinking, you should teach them critical thinking (not philosophy) and if you want them to get even better, you should teach them using argument mapping.

The talk is a blending of two things. First, a talk I gave about five years ago to various philosophy departments in Australia, in which I challenged the audience to come up with positive reasons to think that they are, in their core professional activities, any better than investment professionals such as stockbrokers, fund managers and the ilk, which have been shown by mountains of evidence to be useless at choosing superior investments, even if they are quite good at skimming vast sums of money from the savings of others. In response, philosophers generally came up with, at best, the kind of lame arguments they’d instantly ridicule others for making; the main outcome of all this, as far as I could tell, was resentment towards me for even raising the topic, which may partly explain why I haven’t been invited to talk at any philosophy department ever since.

The second thing is the work of a Masters student at the University of Melbourne, Claudia Alvarez, who has written on whether studying philosophy is, as philosophers claim, especially effective in developing critical thinking skills. Claudia did (or at least, carried through to completion) a meta-analysis which gives us the best available fix on whether this claim is true. In fact, if you make reasonable comparisons, it is hard to make a strong case that philosophy is especially effective, and it is markedly less effective than certain other strategies, such as… teaching critical thinking. The thesis will be completed and available very soon. (I’m happy to give a talk on this material at philosophy departments, but I don’t expect to be swamped with offers.)

It should be fun…

Year in Ideas tidbits

Last week the New York Times published its “6th Annual Year in Ideas” in its Magazine.  81 bite-sized presentations of new ideas or trends; worth a read.  Trouble is, not much of what we read is remembered for very long.  Below I’ve listed the ones I’d like to be able to recall in a week or even a year’s time. 

• Empty-Stomach Intelligence.  Mice think better when they’re hungry.  Maybe we do too.  “Ghrelin” is the hormone at work.
• Eyes of Honesty. People are more virtuous when they’re being watched.  Even by obviously-fake eyes, such as on a poster.
• Hidden-Fee Economy.  People can be sorted into “sophisticates” who seek low up-front prices and avoid hidden fees (such as exorbitant mini-bar prices in hotels), and “myopes” (or “suckers”) who are constantly shelling out.  
• Homophily. Social websites are exploring ways to overcome “our inexorable tendency to link up with one another in ways that confirm rather than test our core beliefs.”
• Hyperopia - an excess of far-sightedness.  In the long term, ants tend to have more regrets than grasshoppers about how they spent their time.  This quirk of human psychology doesn’t mean you should just relax and indulge now though.   
• Low Starting Prices.  On Ebay, lower starting prices result in higher final prices – a kind of reverse anchoring effect.  This is because the anchoring effect is overcome by other effects.  A low start price results in many more bidders, enhancing the apparent value of the item, and exploiting people’s sense of commitment to create greater competition.
• Publication Probity.  The Journal of Spurious Correlations is devoted to publishing only negative results, aiming to counteract pervasive “publication bias” whereby positive results are published much more readily than negative, distorting our understanding of the world.
• Voting Booth Feng Shui.  How we vote is affected by where we vote.  People voting in schools are more likely to vote for state support for education.

Critical thinking in the secondary classroom

An inspiring account from Kylie Sturgess of the Methodist Ladies College in Perth, Western Australia.  Kylie was runner-up in the 2006 Australian Skeptics‘ Prize for Critical Thinking.

Lots means Lots

How do you help your students to achieve really worthwhile gains in critical thinking skills?

We worked on this problem for about five years at the University of Melbourne. We wanted a method for improving critical thinking skills which demonstrably achieves substantial results.

I’ll add now that we wanted a method which reliably acheives these results, i.e, gets them year after year, and in a variety of different contexts.

We think that we succeeded in this. The Reason method, as we called it, achieved gains of about 0.8 standard deviations semester after semester with University of Melbourne students.

We’ll soon be releasing a comprehensive review of the empirical literature (a “meta-analysis”) which compares our results with results found in other studies. Looking at the charts of the data, there certainly appears to be something special about the Reason method.

Increasingly, teachers and researchers around the world are doing their own studies to see if they can obtain the kind of good results showing up in our studies.

That’s great. It is crucial that the results be independently tested and (we hope) verified.

However, any independent attempt at replication should attempt to recreate the essential ingredients of the method being tested.

If you do a study which drops some of the essential ingredients, it doesn’t tell us much about the method.

Unfortunately that’s what we’re seeing. Again and again, attempts by third parties to find out if the Melbourne argument-mapping-based method “really works” don’t really test that method. They drop out a crucial ingredient – and then, usually and predictably, they find that they don’t achieve the same good results.

There are two crucial ingredients to the method we devised.

The first and most important is practice. Lots of practice. The Reason method, in a nutshell, is an application of the Ericsson theory that high levels of skill in any field come from lots of “deliberate practice.” Our idea (hardly very original or brilliant) is that the same will be true of reasoning or critical thinking skills.

The second ingredient comes out of that notion “deliberate.” Basically, deliberate means “good quality”. The challenge we confronted was – how do you get students to do LOTS of “good quality” practice?

Our insight was that you could improve the quality of practice students are doing by putting that practice into a good environment. In particular, we came to believe that argument mapping is a far better context for practice of reasoning skills than typical “prose”-based contexts (such as the typical university lecture, discussion section, book etc. which makes little or no use of argument diagrams).

So the second crucial ingredient in our method is using argument mapping. Specifically, doing one’s practice using argument mapping software.

Let me repeat that for emphasis. The method consists of

1. LOTS of practice
2. using argument mapping software

We used to call the method, in more technical language, “DPAM” – which stood for “deliberate practice using argument mapping.” Not very catchy.

Yanna Rider, one of the Austhink team, came up with the much better acronym LAMP. The method is Lots of Argument Mapping Practice.

Now, whether or not you use argument mapping, doing LOTS of practice is going to demand of a lot of institutional resources. Crudely put, it is going to take a lot of time and effort from staff, or time and effort from a lot of staff.

That’s a problem. Educational institutions are usually stretched pretty thin already, and putting MORE resources into some teaching exercise is a “big ask.”

What they are more often looking for is some way to get results with LESS resources.

So, what we usually find in independent attempts to “replicate” our results is that, when you look closely at how the method is being implemented, the focus is on argument mapping. the LOTS of PRACTICE has been downplayed or ignored.

Our prediction of course is that, to the extent that you don’t do the practice – the extent that you do AM rather than the full LAMP – you won’t get results as strong as ours.

The good news in all of this is that it is possible to achieve substantial gains. But you have to be prepared to do what it takes.

If you find out about an argument mapping study whose results were less impressive than ours, ask – were they really doing enough? Or does it look like they were hoping that argument mapping is some kind of magic bullet?

We believe that an argument-mapping based method is more efficient than other methods, because it offers a better quality of practice. So, for the same amount of resources or practice, you’ll get better results. But if the amount of practice your students are doing is negligible, the results will also be negligible.

One of our goals is to help educational institutions have their students do lots of argument-mapping-based practice without imposing significant extra resource requirements on those institutions. So a teacher can use the LAMP method without creating a lot of extra work for herself.