Metacognition in higher Ed Chapter 6

LaVaque-Manty and Evans

The Domain

Psychology and Political Science writing
Try to think like a domain specialist which is already a bit metacognitive

The Intervention

• planning: reflect on known and to be learned items
• monitoring: reflecting on writing choices
• evaluation: what works/what doesn't

The Results

Key questions:

-- Is this metacognition?

• Made a 'pre-judgment' -- judged own knowledge -- so yes
• Reflective awareness of one's own process -- yes
• Evaluation was more reflective and less metacognitive?

-- Is this success?

-- Can this be improved?

Metacognition in Higher Ed Ch 3

Using Reflection and Metacognition to Improve Student Learning
Kaplan Et Al

"Improving Critical Thinking Skills in Introductory Biology Through Quality Practice and Metacognition"
Chapter 3 -- Lemons et al
Theme: Critical Thinking in Biology
Strategy Pursued: problem solving practice, essay questions with samples -- including use of a rubric
Critical thinking as a habit of mind -- study strategies that enhance learning, retention, domain transfer.
paired metacognition and reflection in order to enhance overall critical thinking.

The Intervention
• determine overall goals
• design activities
  • considered Bloom's taxonomy
  • aligned assessments to taxonomy
  • created separate module called 'critical thinking in biology' to run in parallel with reflective content
    • module contained questions -- similar to exam
    • rubrics with domain knowledge and crit thinking components
    • guiding questions to encourage self-reflection
  • articulated learning goals to students, articulated what constitutes a complete answer
  • pre-lab problem solving
  • repeated emphasis to students that they expect both content knowledge AND critical-thinking skills
• problem-solving
• essay-judging
• essay-writing
• thoughtful design, shared with the students -- why these questions are being asked, in this way. It's not arbitrary!

The Results

This seems particularly useful because they worked from multiple angles, both training thinking skills explicitly AND requiring those thinking skills in the execution of content-area/domain work Interesting comments about how students think questions are intentionally tricky or the motivation is opaque. Intentional trickery can be present in multi-choice questions, riddles, insight problems. Helpful to be metacognitive about whether you are suspecting trickery and a dual-path, "if a trick, then..." and "if sincere, then..."? Some sloppy teaching, some opaque though thoughtful teaching. Opportunity for instructors to be more open about their process?? It is striking how overt the instructors chose to be with their expectations and intentions. Often not the case!

Metacognition in Higher Ed Ch 2

Metacognition in Higher Ed Ch 1

Reflective Pedagogies and the Metacognitive Turn in College Teaching

Naomi Silver

Ch1 of Metacognition in Higher Ed book

Why MetaCog & Reflection, and Why Now

• student metacognition found to be most important variable to improve learning outccomes
• also an alterable variable -- it's something teachers can do something about
• metacognition improves engagement --> engagement improves learning and is a purpose of education

Importance of Engagement

• outcomes depend on student's engagement in educationally purposeful activity
• engagement considered a predictor of learning/personal development
• adds to skills/dispositions for post-college life

I learned this as a buzzword, and essentially it translates to "clickers"

Definition: engagement is involved in/responsible for learning; making decisions about learning
Definition: metacognition is what allows students to make decisions about how they learn best by helping them become aware of what they are doing when they are learning

General principles -- embed metacognitive instruction in the content matter to ensure connectivity, inform learners about the usefulness of metacognitive activities to make them exert the initial extra effort, and incorporate prolonged training to guarantee the smooth and maintained application of metacognitive activity.

More on History/Definition of Metacognition and Reflection

Reflection: Dewey's definition -- more sustained that a general stock-taking, closer to critical thinking. Dewey also emphasizes a genuine problem ("a forked-road situation") -- ambiguity, dilemma. You consider the possibilities then arrive at a provisional understanding. Boud expands on Dewey by using a range of disciplines for context, with a general framework of "reflection is a response to experience" and "returning to the experience, attending to feelings, re-evaluating the experience".

Schon also uses a range of contexts, and expands to discuss a reflective practicum as part of professional training. Emphasizes problem finding -- "name the things to which we will attend, and frame the context in which we will attend to them". Makes a distinction between reflection-in-action and reflection-on-action -- two-part temporal process. Others add a "pre-action" phase (which may be implicit in Schon?).

Metacognition

Flavell on metacognition in the 70s and 80s; "thinking about one's thinking" -- but Flavell incorporates metacognitive knowledge, metacognitive monitoring, and metacognitive control.

Week 12 -- Reflection

This week, we looked at metacognition in education, and the development of metacognition in children. The Winne-Hadwin model describes a series of steps and strategies to use as part of planning a cognitive activity in order to make the most effective use of metacognitive abilities. The model includes 4 phases (task definition, planning, execution, and revision) and encourages individuals to follow a cycle inside each phase using the mnemonic COPES -- conditions, operations, products, evaluate, standards. I found this model to be highly useful, and it seems like the sort of thing that ought to be on the wall of every classroom. This framework allows a teacher or a learner to lay out their work according to this structure and map activities on to it. By having a structure and mnemonic, the learner can avoid missing components, proceed effectively, and gain metacognitive insight alongside the task they are completing. A structure of this kind also seems useful for ability transfer -- since the model is theoretical and general, it exists external of a single domain, and can be applied to a range of topics.

One area I am not sure about is why the class discussion was so critical of the idea that core content should be learned -- not simply "learning how to learn". The textbook does not support this view; for example the text describes an approach to math like this:

• domain knowledge - facts & rules of math
• general strategies for problem solving
• knowledge about one's own cognitive functioning
• self-regulatory skills

This is essentially the same structure I was proposing as the ultimate aim of education, but this view proved to be unpopular during class discussion. It may be that the emphasis on content in schools has led the teachers in the class in particular to be eager to defend the teaching of thinking skills.

One element missing from the discussions in the text concerns the role of parents. Certainly I am somewhat biased in wanting to incorporate this element of a child's education -- being a parent myself, I assess our readings with an eye to being a teacher and researcher as well as with an eye to parental behavior. Metacognitive parenting does not seem so different than metacognitive teaching, with the exception that the relationship is more 1-on-1 than most teachers have the opportunity to do. I think it would be beneficial for researchers on this topic to consider models for how teachers and parents can work together in the education of children. Parents generally have more contact hours with their children than any particular teacher does (even with a 7-hour schoolday, and the same teacher for 2 or 3 years, the weekends, summers, vacations, etc. boost the parents' hours)

One

D&M Concept Review Ch 10

1 - What mental abilities are measured by theory-of-mind tasks?
Theory-of-mind tasks measure whether or not someone understands that someone's state of mind or knowledge can differ from reality, and that another individual's understanding may differ from their own. It includes understanding that there is such a thing as separation between their mind and someone else's, and recognizing people can be deceived -- and that they themselves are capable of that deception. Theory of mind is a metacognitive monitoring capability because it is an indication of someone's awareness of the existence of their own thinking. It is also a metacognitive control capability, because it requires that someone suppress false signals (such as the fact that you know something that the character in a story does not know) in order to arrive at the correct conclusion.

2 - Metacognitive theory explains that young children have difficulties passing theory of mind tasks because this part of their mind has not sufficiently developed yet. There seems to be an inflection point, a moment where these pieces come together. Some metacognitive abilities surface earlier and grow gradually. That part of the brain may not have finished developing.

3 - what accounts for developmental improvement? Monitoring of learning, monitoring of retrieval, or the use of strategies? Strategy use/effectiveness seems to increase in tandem with metacognitive activity. Children show monitoring of learning and retrieval even when very young.

4 - Wishful-thinking hypothesis
This states that children may give poorly-calibrated answers to JOL tasks because they are describing what they want to achieve rather than what is in their brain. This does not fully explain overconfidence, however, since JOL of others was also poorly calibrated. Overconfidence has an adaptive potential function by challenging children to attempt tasks that are beyond their reach, which promotes learning and growth.

D&M Ch 10 Discussion Questions

1 - Metacognitive task with two balls, a ball, and play-doh jars.

• theory of mind -- hide the ball, then move it, when one of the dolls isn't looking
• Ask the kid to remember the items, first the names, then the names and looking at them, then the names and playing with them
• mold the play-doh into shapes. What is this? (bird) Is this really a bird? (no)

Since Tristan is 5, I would expect him to pass the theory of mind and appearance-reality tests. If not, it may be that he has not yet developed the idea that other minds are separate from his own.

2 - Monitoring -- contexts when children are good at judging their learning and/or retrieval.

Younger children did not seem to choose strategies on memory tests, and had overconfidence, even after many trials. Rudimentary metacognitive capabilities seem to exist but have not fully developed.

D&M Ch 10 - Childhood Development

Ch 10 - Childhood Development

Development of Theory of Mind

Theory of Mind (ToM) is the ability to "attribute mental states to ourselves and to other people".

Includes:

• understanding mental states and beliefs (false beliefs, emotions, desires)
• understanding others may have different beliefs
• understanding what it means "to know" and "to forget"

These abilities let us predict the behaviors of others. Animals seem able to predict behavior...ToM or pattern recognition??

How to Measure the Progress of ToM

Key factor: ability to demonstrate an understanding that someone can have a false belief that does not match reality. This reflects understanding that an understanding can be different from actual state of the world.

False belief experiment: Max and his truck, relocated by his mother. Results were consistent across multiple beliefs, countries, etc. Tried this on my kids -- Mira as a 4 year old, no problem. Stella as a 6 year old, no problem either.

Another example -- if you know your competitor will always choose the prize you want the most, you might choose to mislead him/her. General notion: "at 3 years of age, children do not realize that other people can hold a false belief, whereas at around the age of 4, children begin to understand that people's beliefs do not necessarily reflect reality"

Key factor: appearance versus reality - candle shaped like a bird, is it a candle or a bird?

Theory of mind seems to develop coherently and as a system -- once they do well on one ToM task, they tend to do well on many others.

Fantasy play -- telling reality from imagination. Children seem able to tell the difference, but to still feel afraid of things they know are imaginary. Some children even thought that perhaps something they imagined might be able to become real. Area of future research.

Theories of ToM Development

Children seem to develop in spurts -- to go from no theory of mind to strong theory of mind in six months. Pop! Theories about development focus on one of two approaches -- a special mechanism or type of development ("modular"), or an example of more general development.

If modular theory -- a specific neurocognitive system is responsible. Innate structure matures from birth through childhood. But if innate, why can't children pass it before they are 3.5 yrs old? Some arguments are that we just can't measure it well before then, and that even infants show some awareness of ToM.

If general theory -- neuro systems support many abilities, for example overall executive functioning skills (inhibit irrelevant thoughts, work with multiple thoughts at once) might be responsible for ToM tasks. Notion is that children must consciously inhibit interfering memories so as to not be deceived. Seem to correlate ToM with executive control -- but causality, if any, is not clear. May be functional interdependence.

Development of Metamemory

We know that memory monitoring and feeling of knowing is a real thing in humans -- what about in non-humans? And when do children begin to develop what we can clearly measure in adults?

Metacognition in Nonhumans

How about chimps? Commonly thought of as having "mental age of a 3-year-old". Lack of verbalization is a challenge. Study design is key.

Josep Call -- if an animal knows it does not know, it would seek information. So information seeking is used as a metacognitive indicator. Ex: hide a treat in one of two spots. Chimp will look to see which spot the treat is in. Dogs will just go for the treat.

Smith/Shields/Washburn have shown that primates and dolphins have an 'uncertainty' response. Offered an 'opt out' choice when a task is too difficult (no reward, and no punishment either).

Counter: does complex conditioning or innate tendency explain this instead? Hampton studied opt-out/uncertainty in monkeys, and showed that monkeys chose among difficult tasks/high reward and easier/low reward effectively -- they knew when they knew. Can't just be conditioning because his design relied on memory.

Overall conclusion: animals have some metacognition abilities.

Development of memory monitoring in children

Global judgments
Asked to make a a memory judgment, children tend to be overconfident. Practice didn't eliminate their overconfidence either. Children may not be good at monitoring their performance? No, this was ruled out. Wishful thinking? Inability to differentiate what you wish for and what you expect. But this is not proven, because overconfidence often persists even when predicting the performance of others.

Overconfidence may be adaptive -- encourages ambitious trials and learning experiences.

Item by Item Judgments of Learning
Less research on this topic. We believe that adults use inference to make JOLs -- ease of learning, study trials. Children were tested to see if they are sensitive to these cues. Children seemed to respond to the ease of learning and study trials heuristics, just like adults.

How about delayed JOLs? Younger kids given fewer. Children still had better than chance accuracy and higher accuracy on delayed JOLs than for immediate JOLs, and younger children did as well as older children in delayed JOLs.

Conclusion: Children have remarkably adult-like metacognition. Strategic use of study time does improve with age.

FOK in Children
Study -- give definition of words, until they failed to define 35 of them. Afterward, they made an FOK for the words and a multiple choice for each. FOK accuracy improved with age in one study but did not improve in many others.

Strategy Use

Learning a list of words -- most successful if you use a strategy, like a mnemonic. Researchers have investigated if children develop in their use of strategy.

Knowledge about strategies -- do children know about effective ways to promote their learning? Kreutzer et al did a series of interviews to gauge knowledge of strategy, memory, and learning. Children at all ages had some intuitions about these things, although older children articulated better, and tended to offer better strategies.

Effectiveness of Strategy Use

Experiment: list of words, rehearsed. Tendency for early words and most recent words to be best-remembered. Younger children may not spontaneously use the best strategies in these cases. But if instructed to use effective strategies, they are capable of doing so and benefit from them -- if they have a production deficiency. Some children have a utilization deficiency, in which they are trained to use a strategy but don't execute it well.

Borkowski et al: "When a child possesses a number of strategies together with knowledge about their various uses, he or she is able to make an informed judgment about strategy deployment".

Metacognitive model of strategy discovery: application of heuristics.
Associative model of strategy discovery: discoveries through doing tasks, actions, assessing outcomes

Hybrid of metacognitive and associative strategy discovery seems to lead to best results.

Theme: "mutual influence between metacognitive and cognitive processes that is responsible for how we think and behave when faced with many different tasks"

relationship between ToM and Metamemory

ToM and Metamemory do not develop in lockstep. Each is multidimensional and the relationship is complex. ToM investigate knowledge about mental states, metamemory researchers investigate knowledge and processes relevant to task performance. ToM researchers tend to investigate younger children, metamemory researchers focus on 6 and up.

Lockl/Schneider - study at 3, 4, and 5, combining both ToM and Metamemory experiments. Kids' performance correlated throughout the study -- early success and late success. Language is an essential tool -- but even with corrections for language acquisition, the correlations continued. Early theory-of-mind is a precursor of metamemory. Representation seems to be a crucial step.

D&M Ch 9 Concept Review

1 - Why might students not notice inconsistencies between what they know and what they read?
The student might be only absorbing the material, not synthesizing it into an overall understanding or building an internal model of the text.

2 - Self-explanation.
Self-explanation is the process of telling yourself what you know or understand about a topic. It might improve learning by enabling students to more readily see the gaps or inconsistencies in their understanding. Explanation is less passive, and activates different parts of the brain and different kinds of thinking than just reading.

3 - Self-efficacy.
Self-efficacy is the belief in one's own ability to do things. Students who believe they can learn are more likely to choose significant goals, helpful strategies, to modify their plans as they go (if it's not working, the problem can be fixed by a new strategy, rather than being caused by some failure on their part) and to carry out their plans more effectively than students who are focused on their own limitations.

4 - Knowledge telling versus knowledge transforming
Knowledge telling is a style of writing that emphasizing delineating everything that a person knows about a topic, regardless of order or significance. Knowledge transforming is a style of writing that connects knowledge together toward an argument, narrative, or model, converting raw facts into a synthesis on the topic. Writing monitoring for a knowledge teller is likely to be oriented around form -- am I writing this correctly, do I have all the facts. Writing monitoring for a knowledge transformer is likely to be oriented around content -- does this make sense, am I making a strong argument here, are there angles I am not considering. Revision for a knowledge teller might look more like copyediting -- revision for a knowledge transformer might look more like a rewrite or structural edit.

5 - Metacognition in math
Math problem solving includes developing a model of a problem, the application of rules, the application of strategies, and developing one's insight into the problem. Student metacognition supports successful math problem-solving by helping them to ask questions about their process and potentially change that process along the way: whether they are developing a useful model, whether they are applying the right rules correctly, whether they've chosen the right strategy, and whether there might be some element of the problem they are missing. Since we know that organizing assumptions are both necessary for a problem to be solved and also a blocking factor to successful solutions, strong metacognition allows students to both prevent an unhelpful organizing assumption from taking hold, and to release a useless assumption in favor of a more useful one.

D&M 9 Discussion Questions

1 - Intro Psych Exam
According to the Winne-Hadwin model, the four main stages for students to study for the exam would be: task definition, goal setting and planning, enactment, adaptation. Inside these stages, the model suggests the acronym 'COPES'.
Task Definition
C - Conditions
The student should learn as much as they can about the exam. When, where, and how long? What materials will it cover? How will the exam be structured? Is there a penalty for guessing? How are items weighted? Is the exam oriented around memorizing facts or understanding concepts? Will it be computational or essay-oriented? Are notes allowed? Do they need to have broad or deep familiarity?
O - Operations
The student should choose sources (both written and human) and ask questions to understand.
P - Products
The end goal of the task definition phase is a full understanding of what the exam will be.
E - Evaluation
Evaluation should include comparing one's understanding to the syllabus and to statements from the professor about what will be included.
S - Standards
At the end of this task, the student should be able to describe their expectations for the exam, in sufficient detail to inform the development of a study plan.
Goal Setting and Planning C - Conditions
The student should think about when and where they should study. How much time will they need? What kind of time do they have available? What locations are optimum, and when are those locations available? Do they need silence? A solid block of time?
O - Operations
The student should think about what methods work best for studying this topic and for preparing for the kind of exam they are expecting. Will it help to rehearse essay answers, create flash cards, read through all of the readings, attend a review session, look through class notes, meet with classmates to discuss, or meet with the professor? Are there problem sets, practice exams, or intermediate assignments which might show weak spots?
P - Products
The end product of this phase should be a study plan. When, where, and how will the content be studied?
E - Evaluation
The plan will be evaluated in two phases -- one, before the exam, the student should review their plan and compare it against their past choices and results. Secondarily, the student should commit to reflecting on their study plan after the exam to improve their understanding of how to plan for exams appropriately.
S - Standards
The overall standard for the study plan should be that it is both reasonable (the student can indeed accomplish it, and it seems fit for purpose) and based on strategy rather than whim or preference -- and after the exam, the additional evaluation criteria is whether or not it was successful.
Enactment
C - Conditions
The student should set up their environment according to their plan.
O - Operations
The student should sit down and actually carry out the study plan in the way they described.
P - Products
The product of enacting this study activity should be an increased understanding of the material.
E - Evaluation
The student should be able to reflect on their studying to determine if they are learning the material.
S - Standards
The primary standard for studying is whether or not the materials are learned to the level described in their understanding of the exam and in their study plan.
Adaptation C - Conditions
If the conditions change, the student should either adapt their plan or change their conditions -- if the normally quiet room is noisy, move to another room or wear headphones, for example.
O - Operations
While studying, the student should continue to monitor their learning and make adjustments. Is their strategy working? Are there topics or issues they did not expect to have difficulty with and yet they have indications that they are not meeting with success?
P - Products
The studying activity should produce increased understanding despite any errors made in the previous phases.
E - Evaluation
Evaluation of adaptation phase should be two-part -- before the exam, the student should determine whether they are effectively adapting. After the exam, the student should consider whether their expectations for the exam were correct, whether their study plan was the right one, and whether they successfully executed on their plan.
S - Standards
The primary standard for the success of adaptation in a study activity is whether they responded appropriately to the feedback they received from their own mind during the process. A secondary standard is to consider this same question after the fact.

According to the model, students might believe they are ready when they're not due if they fail to successfully evaluate their work in each phase (the 'E' in COPES).

2A - Metacognition in Note Taking
Note taking is a writing process that accompanies reading, and serves three functions -- it promotes and supports understanding by helping the reader to be reflective, synthesize materials, and remember them; it allows self-evaluation after a reading assignment by allowing students to ask whether what they have written makes sense and reveals understanding of a text; it supports later work to study or review a topic, since the notes can be reviewed directly rather than the text re-read.
Metacognition might be revealed by studying whether effective note-takers use computers or hand-write, whether they tend to quote, synthesize, or extract key points, whether they use mind maps, diagrams, or paragraph forms. When students don't understand a topic, how do their notes differ? Do they change their note-taking strategy based on in-class experience, exam experience, peer learning, or some other factor? What do students say about their approach to note-taking -- can they describe a strategy that suits the coursework, their own learning styles, etc.?

It may be possible to study note-taking by analyzing the notes the students take, versus their performance on exams -- how much is written, what kinds of things are written, and whether there is a correlation between what is in the notes and what they are successfully tested on (did they write the correct key items down? did they write key items down correctly? did they use their notes later?)

2B - Metacognition in Class Discussions
Participating in class discussions is an opportunity for students to process materials as a group, to test their understanding, to ask questions, and to learn from the thinking of others. Metacognition in class discussions might be revealed by how someone participates (actively, passively, leading, contributing, distracting, synthesizing, arguing) as well as how often (once per class, six times per class), as well as how they respond to the discussion. Do they gain understanding? Change their mind? Persuade others? Or walk out no different than when they entered? This may be tested by doing quizzes before and after a discussion as well as by observing classes and coding behaviors in class. Another angle on class discussion is to assess whether students change their approach over the course of the semester. Do they become progressively prepared and engaged, do they seem to learn from the classroom dynamic and improve the depth of their participation and contributions over time?

D&M Chapter 9

Theme: studies of metacognition and student education

General Models of Student Self-Regulated Learning

Non-metacognitive influences: motivation, goal setting, goal orientation
Metacognitive influences: self-reflection, control of learning

Winne & Hadwin's Model

• emphasizes monitoring-and-control processes; self-regulatory behavior
• Four stages
  • task definition
  • goal setting and planning
  • enactment
  • adaptation
• Inside each stage: COPES
  • Conditions (in which studying occurs)
  • Operations (using strategies)
  • Products (making something)
  • Evaluate (what you produced)
  • Standards (compare to your product)

Studies related to metacognition
Cognitive research -- tends to be laboratory based and focus on minute analysis of underlying mechanisms

Educational research -- tends to be conducted in classrooms using representative materials; investigates relationships between metacognition and student achievement.

Self-Efficacy

Definition: people's beliefs about their ability to successfully complete a given task.

• People who believe they can succeed set higher goals
• select strategies based on effectiveness -- not difficulty or time

Study: Chemers, Hu, Garcia.
Student questionnaire on self-efficacy. Higher self-belief correlates with higher performance (GPA).

Overall findings suggest improving self-efficacy boosts performance by improving self-regulation in learning. People who believe they can achieve will set more specific and more demanding goals, and use more effective strategies.

Quiz
• I know how to schedule my time
• I know how to take notes
• I know how to study to perform well on tests
• I am good at research and writing papers
• I am a very good student
• I usually do well in school and at academic tasks
• I find my university academic work interesting and absorbing
• I am very capable of succeeding at the university

Metacognitive Monitoring

"Monitoring and control are central to effective self-regulated learning and performance."
Overconfidence --> understudying, use of ineffective strategies, underachievement
Calibration of monitoring is related to effective self-regulated learning.
Observation of 'unskilled but unaware' -- tendency for students to be fairly confident, but in some that's unwarranted. Good performers got more accurate over the course of the semester, but poor performers did not recalibrate their judgments. Providing incentives for performance and judgment does seem to improve calibration.

Training accurate judgments --

• rated how well they understood
• described concept they had particular difficulty with
• answered practice questions
• judged their confidence

Results were compared against exams. People with calibration training performed the same on first exam as those who did not receive training. As the class progressed, judgment improved. Improvements in calibration were predictive of higher scores on the final test. More research needed into training methods, understand duration of impact, and transfer

Research - changing answers during a test.
Scores are mostly a result of preparation, intelligence, and motivation. Changing your mind is often beneficial -- trust your confidence on this. If you were guessing, go with the hunch/gut/initial reaction...but if you have a reason, go for it and make the change.

Metacognitive Control

Purpose: to allow people to control their cognitive processing. Ex: after an exam, using results to improve performance on next exam. Increase efforts, decrease efforts, etc.

Study -- Rocky's Boots. Problem-solving program with boolean logic. Question: will training students to monitor their progress and success during practice phases help them to better regulate their problem solving. Structure: some students get no training, one group gets problem-solving training, one group gets problem-solving training and cognitive monitoring prompts. Monitoring group outperformed the other two on difficult problems, and used less time. Thinking about thinking may have encouraged them to use more strategies -- monitoring improves control, which improves performance.

Domain Metacognition

Reading
Literacy is arguably the foundation of education. Major omissions in instructions experiment -- first graders didn't notice, third graders did. Comprehension monitoring -- internally enacting the instructions lets you evaluate whether your understanding allows you to obtain the goal of winning the game.

Comprehension monitoring includes detecting inconsistencies and omissions, evaluating understanding, and comparing to previous knowledge. Control activities might be re-study or using strategies.

Reading Model:

• to create an internal representation of the text, including the words/sentences and the meaning.
• to meet an evaluation standard (understand all, memorize crucial concepts, etc.).
• to use strategies to achieve these goals
  • prepare to read -- set goals, skim, etc.
  • construct meaning from the text -- identify main ideas, make inferences, interpret, monitor
  • reflect on reading -- ask questions, summarize

These phases are recursive and have no fixed order. Each may include monitoring and control activities.

Call this constructively responsive reading.

Ann Brown's work on Metacognition/Education
Gifted/slow was originally thought to be a matter of memory capacity. Brown argued instead that the issue is metacognition (knowing when, where, and how to remember).

One experiment had students read texts and then try to identify which parts were important versus what could be deleted without compromising the main ideas. They asked participants to judge this by first removing the least important ideas, then the next least, and so on. Youngest participants did not distinguish, older students did.

Another experiment had children learn to monitor their comprehension by summarizing, developing questions, predicting what will happen next, and take turns leading a discussion with these activities. Students improved in response to this, and transferred the skills readily.

Strategic skills for reading develop over time (re-reading, summarizing, slowing down, relating portions of texts)

Self-Explanation? Improving your performance by talking to yourself in the "right way". Good students tended to be those who did some "thinking out loud" about a problem, exploring a worked-out example and making inferences, examining trouble spots, etc. Explain your understanding as you proceed for a cognitive boost. Asking students "why do you think that?" is another prompt -- and, even better, ask them "how did I figure this out?" (explain a correct answer). Similar results shown in math, analogy, and science. This approach is thought to promote a more reflective strategy.

Writing

Evaluation and revision is similar to the comprehension & comprehension monitoring in reading. Initial writing models were informed by think-aloud protocols.

Hayes & Flower Model

• Planning
• Translating
• Reviewing

Bereiter & Scardamalia Finding Two differences between novice and expert:

• Novices list everything they know - knowledge telling
• Experts turn knowledge into a complete story - knowledge transforming

Math

Math students spend very little time on the problem, and focus more on the solution, even if it's a dead end -- mathematicians work mostly on the problem itself. Mathematicians also monitored their progress -- asking themselves if they are moving in a useful direction, and changing direction if not. this section was very strange -- geometry students characterized as knowing more about geometry than the mathematicians??? who are these people they did the study on??

De Corte et al recommend that students develop:

• domain knowledge - facts & rules of math
• general strategies for problem solving
• knowledge about one's own cognitive functioning
• self-regulatory skills

Fuchs et al intervention

• Large-scale self-regulation learning program
• Problem solving, monitoring, goal-setting, math self-efficacy, self-monitoring
• Compared to transfer training and no-training control group
• Self-regulation group outperformed others

Achievement is possible via high IQ and/or strong metacognition

Concept Questions, D&M Ch 8

1 - DNA has shown that even witnesses with strong confidence and victims of trauma can be completely wrong about a perpetrator.

2 - Hindsight bias is also called 'knew it all along' -- the idea being that sometimes people say that they were certain of something before they knew the outcome. But evidence shows us that their votes are changed and their certainty is substantially increased by knowing the result. It's explained by a combination of factors -- a need to be right, a confusion of the past with the present, or simply an inability to actually put information aside when making recollections about the past.

3 - Lie detection is typically not very accurate. I think it is probably more difficult when you do not know the subject well, or do not have any way to verify conflicting information. Most of the behaviors that indicate lying tend to be undetected by the human eye. Most of the so-called indicators of lying tend to be characteristics of multiple situations, including stress. There are a few heuristics that might be used, but these can be defeated. Generally people cannot detect lying.

4 - We know that jurors are influenced by eyewitness testimony, especially when the witness is confident, because we have verified through experiments that this factor swings votes substantially.

Discussion Questions, D&M Ch 8

1 - memory is highly suggestible. I think that there are limits to what you can make people believe about the past. The more illogical the story, or inconsistent it is, the more likely people are to disagree or reject the attempt to implant a memory. If the person has some specific memory that is not consistent with the tale, it may stick out at them and give them a thread to pull on until the suggested nature of the memory is unraveled. I also think that people with particularly strong memories or skeptical minds would be more resistant to implanted memories.

2 - Failing to remember an interrogator under a high stress situation was actually not a surprising result to me -- under stress, what you see and perceive changes a great deal. You may not be focused in on recalling a broad general face image or able to easily make a mental reproduction of what that same face might look like on the street or in a mugshot instead of screaming at you. After a trauma, I think the brain is also highly susceptible to manipulation -- you want to help the police, you are seeking reassurance, your flashes of memory may be so full of gaps that anything familiar seems like a thing to hold on to. Carefully focusing on the details needed to make a later identification, in context, and then generating them spontaneously in response to the very first question you get on the topic, seems like the best way to be confident of your recollection. One experimental design that might support this would be to generate stress through sports or games -- something that people would be comfortable and would pass ethics/IRB. For example, bet $100 on a football game, then ask questions about the participants, versus $5 on the game.

Reflection -- Week 6

Ch 8 in D&M and Communications MindTools
This week's readings focused particularly on metacognitive failures in people otherwise considered healthy and normal.

What we know about how people's minds work tells us that lying politicians and news accounts and the like are particularly damaging and dangerous. It is very difficult to reverse the impact of widespread lies. Even after people know the truth, studies show their perspective is still impacted. In addition, I have also seen people respond to fabrications with extreme hindsight bias, saying, that they know something's not true, but it's something that person would say, so it doesn't matter (like that Twitter fabrication described here)

One observation I had about the material was the notion that living mindfully, with your mind present in the moment, seems like a technique that would improve both memory accuracy and your ability to protect your recollections against influence.

The discussions of hindsight bias in the case of jurors has significant implications for the criminal justice system, and particularly the importance of proper forensic evidence processing and interrogation techniques. The backlog of DNA evidence is a disgrace. I hope that the legal system is able to absorb what psychology is teaching us about the dangers of simply asking a jury to disregard information.

I myself have found that when I have a 'knew it all along' response to a situation. I am not sure the model that was presented is a direct corollary to this. To me, that 'knew it all along' feeling seems like a result of a feeling of "fit" -- the result is consistent with my thinking up to that point, and it all plugs together in a familiar/recognized way.

For example, with the Trayvon Martin case, I found myself thinking both ways -- that the evidence as it was reported to me surely pointed toward conviction, and that the prosecutorial conduct of the case pointed toward acquittal. But once the verdict came out, I distinctly felt that I knew they were not going to convict Zimmermann, or at least that I should have known, which feels like much the same thing.

Once the verdict came out, the story came together. In theory, I knew that I was not in the jury room, and I was sure that there are many cases which look clear in the media but which are conducted in a way that the jury decides that the case has not been made effectively. But I was aware of a visceral reaction that the whole situation had been unfair, that I'd known or should've known it was going to turn out this way, that I should have known better than to hope that a seemingly-guilty person (as reported by the media) would also be guilty in court.

The distinction is a subtlety of the system that can be overlooked, but a careful juror is trying to make a decision based not on "what really happened" but rather "what the evidence proves". You hope that the evidence tells you what happened, but people lie, evidence gets overlooked or is simply lacking. So a vote to convict and a belief about reality are really different things. It does seem that people's ability to draw these types of distinction is susceptible to bias -- evidence may be weighed differently depending on what people suspect to be true based on race, gender, age, etc. A tighter and more vigorous case may be needed in situations where people are biased toward empathy for the accused.

The Communications MindTools talk about the virtue of planning a first impression or a communication activity, and what we know about hindsight bias emphasizes the value of this -- if you mess it up, it's very hard to fix later. Even if everyone agrees that something was awry and wants to put it aside in a fair way, psychology tells us that the first impression is persistent. There were some parallels between the active listening and active questioning section and the cognitive interview style that is described in the text -- in both cases, you are working to support the speaker's ability to deliver the message they want to deliver, without influencing it or cutting it short.

D&M Ch 8 - Law and Eyewitness Accuracy

Issue: eyewitnesses are supposed to share what they saw, everything they saw, and nothing else. But we've seen how inaccurate memory can be and how difficult it is to not fill in gaps -- the witnesses are essentially swearing to something impossible. But people tend to believe it is possible, and juries in particular have a preference for strong certainty. Witnesses can be wrong. People fail to detect lies. Also, hindsight bias -- the "I knew it all along" effect -- inadmissible testimony still taints their judgement.

Ex: show video of a crash, people are likely to say the speed was faster if asked how fast the cars 'smashed' into each other than if asked how fast they 'hit' each other. Repeating false information once or even multiple times can cause people to remember incorrectly. This is what makes lying politicians and news accounts so disturbing. Or people respond to fabrications with extreme hindsight bias, saying, that they know something's not true, but it's something that person would say, so it doesn't matter (like that Twitter fabrication described here)

Ex: implanted memory of being lost in a shopping mall. So easy to get people to recollect things in detail that did not happen, just by telling them it did with certainty.

Confidence usually means truth, but we know from the lab that people are frequently overconfident. How do people have such high confidence about something that didn't happen? How does confidence normally work as a reliability index? How can confidence be manipulated? Can confidence ever be infallible?

Many examples of high-confidence memory being incorrect.

Where does confidence come from?

• Frequency of exposure increases confidence.
• High confidence errors lead to a particular sort of surprise.
• Confidence increases people's level of belief in the statement.
• Usually confidence is a good marker, but confidence can be manipulated until it does not discriminate between true and false memories.
• Asking for memory source increases ability to identify actual memories from suggested/manipulated memories.
• Actively engaging metamemory processes improves memory identification. Living mindfully improves reflection

The cognitive interview: A good witness interview guides you through memory-enhancing procedures -- think about the context, including surroundings and your reactions; reporting everything that comes to mind, even if fragmentary or inconsequential; remembering events in different orders; recalling events from different perspectives. Allow to proceed interrupted. Do not suggest content.

Ex: faux patient bio, given to clinicians. Asked to make inferences based on background, given more background over time. Their confidence increased, but the accuracy was no better than chance.

Is Confidence Ever Infallible?

What if the events were traumatic? Extreme trauma and stress seems to produce less reliable memory.

Witness Confidence in a Trial

Experimental trial - witness confidence was the most powerful predictor of conviction. However, if caught in a mistake, a more-confident witness might appear less credible than a less-confident one.

What about lying?

deception cues -- signs of lying
leakage cues -- signs of the truth despite the lie

Lying is revealed more often in microexpressions -- usually only visible on slow-motion replay of a video. Sometimes lying is more cognitively demanding, so results in more pausing, errors, brow-furrowing, speech errors. But basic fidgeting and gaze averting is not a good sign. Extensive studies show that people are not good at detecting lies, even with training and expertise, with small exceptions.

Repeating a lie makes people more likely to believe it.

Hindsight Bias

AKA the 'knew it all along' effect
Given an unknown, then told the answer, people do a bad job of estimating retrospectively or to act as if they did not know the outcome.

OJ Simpson case -- similar -- people made stronger judgments looking back, saying that they knew that's how it would go.

Why can't people look back with accuracy on their views about an unknown?

• option: personal needs view. People like to be right -- need varies in how strong it is. Some people have low tolerance for ambiguity (used the 'Rokeach Dogmatism Scale'). Children and older people may have lower tolerance for ambiguity. But this isn't sufficient to explain the prevalence, some people who score fairly low on the personal needs ranking still display hindsight bias.
• option: memory view. Purely cognitive. Structure of hindsight bias is similar to mis-remembering things. Actual memory blends with what is suggested or learned later. Hard to retrieve the earlier information.
• option: anchoring view. Outcome pulls your perception toward it -- this overall effect has been shown widely.

I myself have found that when I have a 'knew it all along' response to a situation, it seems like a result of a feeling of "fit" -- the result is consistent with my thinking up to that point, and it all plugs together in a familiar/recognized way. For example, with the Trayvon Martin case, I found myself thinking both ways -- that the evidence as it was reported to me surely pointed toward conviction, and that the prosecutorial conduct of the case pointed toward acquittal. Once the verdict came out, the story came together. I also know that I was not in the jury room, and I am sure that there are many cases which look clear in the media but which are conducted in a way that the jury decides that the case has not been made effectively. This is a subtlety of the system that can be overlooked, but a careful juror is trying to make a decision based not on "what really happened" but rather "what the evidence proves". You hope that the evidence tells you what happened, but people lie, evidence gets overlooked or is simply lacking. So a vote to convict and a belief about reality are really different things. It does seem that people's ability to draw these types of distinction is susceptible to bias -- evidence may be weighed differently depending on what people suspect to be true based on race, gender, age, etc. A tighter and more vigorous case may be needed in situations where people are biased toward empathy for the accused.

Inadmissible evidence is very difficult for juries to ignore, for example a coerced confession when coupled with circumstantial evidence. In studies, even obviously coerced confessions led to higher conviction rates. Malpractice results look similar -- hindsight is 20/20 (the "retrospectoscope"!). Looking back, it seems obvious that everyone should've known there were issues. And yet in the moment, people may not.

Effect is pervasive -- a brain-teaser is obvious after the fact. A new discovery is something people 'knew all along'.

Week 11 -- Reflection

This week, we had no class and no readings, but the mindtools journal was due. Thinking back over the mindtools activities, I definitely found them useful. I tended to assign myself scenarios of my own interest as ways to try out the tools, and I found that to be an effective way to learn the material and techniques. I find myself thinking back over the different templates and planning to make use of them in the future. I also used the gap in the course schedule to re-read some sections of the text, and to begin planning for the course final.

Ch 7 Concept Review

1 - Source memory: remembering how you know something, why you made particular note of it, who said it, what was going on at the time. Item memory: The memory itself.

2 - In source monitoring framework, people use heuristics to judge their source. If they choose a bad heuristic, may remember source incorrectly ("so vivid" or made an image -- could be a dream or real).

3 - Brain areas for source judgments? Prefrontal cortex for the memory of the thing and source, and the medial temporal lobe to bind the pieces together.

4 - Forward model -- schizophrenics may confuse inner voice with outer voice because they fail to recognize their own voice or own thought. Feedback from their actions may be garbled or misunderstood by the body.

5 - Mirror sign -- failure to recognize own face in the mirror, don't receive the familiarity signal you are expecting.

Ch 7 Discussion Questions - D&M

1 - Ouija board. What makes people convinced that it's moving somewhere? It may be feeling the other person move it.
2 - Politicians might use false familiarity to convince someone of their opponents bad character, advertisers might use powerful imagery and positive language to make their product seem better. Politicians can be evaluated through research, and both can be assessed via blind trials ('istandwith.com' and the pepsi challenge).

D&M Ch 7

D&M Metacognition Ch 7

Source Judgments

All judgments about memories or cognitions are metacognitions -- judgments of recency, frequency, source.

Judgment of source

Judging the source of a memory or the context. Did you read it or did someone tell you? Did you do it or only think about it?

"Source monitoring is the process in making attributions about the origins of thoughts and memories"

People can accuse the wrong person of a crime -- they saw them near the time it occurred, just not doing the crime.

Tend to base our source judgment on heuristics. If two potential sources are highly similar, more likely to make an error.

Important to distinguish source memory versus item memory when trying to study memory.

Study: ability to identify where pictures were located. Upsetting pictures were the hardest to recall location. Depressed people did particularly poorly.

Vivid memories or people who make vivid memories also remember sources poorly.

MEM model -- multiple-entry modular memory system framework

• people don't label memories with source
• source is judged by experiences at retrieval -- perception, context, semantics, emotion information
• source is also judged by memory of mental operations when encoding

Example: Fame Test. Reading a list of names, told the people are not famous -- but a day later, people started to think maybe they were famous.

People seem to make better recollections if asked specific details and asked how they know the person, but if just asked if they're famous (yes/no), may use only familiarity as a guide. Aging seems to weaken source recall more strongly than it weakens overall recollection.

Current debate: is source judgment a one-step process? If you are asked, did X or Y tell you Z, you are more likely to choose whatever seems more familiar, not necessarily what is true or actually recalled. Prevailing wisdom is that the process is two-part -- recognition AND familiarity. But one-step (familiarity only) explains results in some studies.

Cryptomnesia -- unconscious plagiarism.

Confabulation -- originally thought to be gap-filling, but this is only the case with Korsakoff amnesia. Some indication it is caused by faulty reality monitoring.

Schizophrenia -- hallucination, a difficulty distinguishing events that are internally generated from those that are externally produced.

One theory focuses around reality monitoring. Internal stimulation seems to have an external source -- it's your thought, but it seems like a voice. Likely to make rapid/incorrrect judgments about source of perception. Even when given clues about the unreality of the voices -- knowledge that others could not hear them, that they came in dreams, a belief that they predicted the future -- schizophrenics emphasized this as evidence that the voices were real (bad heuristics!).

Another theory focuses on forward motor model. Before a normal person acts, they make a plan and receive feedback to match their actions so they can correct them. You predict sensory feedback and receive it. But in a schizophrenic person, the internal feedback from one's actions gets distorted. The match between intended action and actual action isn't enough -- the lack of physical feedback or lack of fine-grained monitoring of feedback. Example: you can't actually tickle yourself...unless you are schizophrenic.

Mirror Sign
Self-monitoring deficit -- we look in the mirror and don't recognize ourselves. Or we look in the mirror and see someone younger than oneself.

Reality monitoring deficits in normal individuals
Watch someone's hands under a smock in the mirror -- can seem like your own hands. Moving a mouse or controlling a game. Having a plan in your head can make you think you've done it yourself!

Current area of research: how source and item are bound together in the brain. Model suggests prefrontal cortex activation and medial temporal lobes to bind the pieces together (judgment + source).

Livingston Summary on Metacognition

Taken from Web Article - Jennifer A. Livingston. "Metacognition: An Overview" http://www.gse.buffalo.edu/fas/shuell/CEP564/Metacog.htm

The term metacognition is most commonly associated with John Flavell (1979).

Metacognition (Flavel)

• Metacognitive Knowledge
• Metacognitive Control
  • person variables -- how people learn & process
  • task variables -- nature of the task & the demands it will place on you
  • strategy variables -- knowledge of meta/cognitive strategies and when they are appropriate

Distinction to draw between the two:

"Knowledge is considered to be metacognitive if it is actively used in a strategic manner to ensure that a goal is met. For example, a student may use knowledge in planning how to approach a math exam: "I know that I (person variable) have difficulty with word problems (task variable), so I will answer the computational problems first and save the word problems for last (strategy variable)." Simply possessing knowledge about one's cognitive strengths or weaknesses and the nature of the task without actively utilizing this information to oversee learning is not metacognitive."

week 10 notes from lecture

Study of beliefs about why things happen

Is behavior fixed or malleable? How do people tend to explain things? Personal awesomeness or outside forces?

Two types of attitudes:

• Situational: external cause
  • caused by context
  • often used to explain own failures
• Dispositional: internal cause
  • caused by who we are
  • often used to explain failures of others

Actor-observer effect: we tend to attribute our behavior to the situation and others' behavior to traits

Fixed mindset and growth mindset -- an attitude. Fixed: the way people are is the way they are; people can grow, but there's a baseline capacity. More likely to quit/not persevere in the face of failure. Growth: believe their capabilities are not fixed. Have the capacity to exceed any particular limits.

Hard work pays off with both mindsets. General perspectives. Not cognitions.

NOT THE SAME AS LIMITED/UNLIMITED?????

Related to locus of control -- where do we think control is.

Critical reasoning in ethics (short guide)

From http://books.google.com/books?id=FPvU8_fmwPoC&printsec=frontcover#v=onepage&q&f=false
Traits of fairmindedness:

• entertain all moral viewpoints sympathetically
• assess without reference to your own feelings on the topic
• be self-critical -- assess your own views as strictly as you assess the views of others
• willingness to adhere to moral standards even if they disadvantage you

Rawls: try to assess a situation as if you do not know your own position in society, talents, and abilities.

Hume -- emotional sense of morals, reason is subsidiary
Kant -- reason about morals (without emotion)

Fairmindedness comes with age

From http://www.swissinfo.ch/eng/fair-mindedness-comes-with-age/6885916
Between the age of three and eight, children develop a sense of fairness. Children without siblings seemed to be more sharing. Some proportion of nature and nurture involved.

Practicing Fairmindedness

From http://faculty.spokanefalls.edu/InetShare/AutoWebs/jimp/Library/Practicing%20Fairmindedness.htm

Definition: "We refer to the ability to seek out and understand alternative points of view as fairmindedness. Critical thinkers work to develop a healthy sense of fairmindedness. "

Heyman & Dweck - childrens attitudes

Children’s Thinking about Traits: Implications for Judgments of the Self and Others
Gail D. Heyman and Carol S. Dweck
Child Development, April 1998, Volume 64, Number 2, Pages 391-403

The relation between the way in which children interpret human behavior and their beliefs about the stability of human traits is investigated. Results suggest that beliefs about the stability of traits may serve an important role in think- ing about and functioning within the academic and sociomoral domains.

Overall, fixed-entity views tend to increase perception of helplessness. Children made predictions about future behavior based on current behavior. Children who saw opportunity for people to change with time tended to allow for other kinds of change with time.

Results: "We suggest that when important traits are thought to be stable, attention is focused on categorizing people based on their abilities and personality attributes rather than on understanding the relevant social and motivational processes."

Bottom line: even children develop beliefs about mutability and attributes, and at very young ages.

Hong et al - theories about mutability of traits in Hong Kong university students

Hong et al
Implicit Theories, Attributions, and Coping: A Meaning System Approach
Journal of Personality and Social Psychology
1999, Vol. 77, No. 3, 588-599

"Attributions have been widely recognized as mediators of adaptive and maladaptive behavior patterns in the face of setbacks." I.e. what you believe to be true about the way your brain works impacts whether your response to a situation is helpful. (what you believe is what you achieve! Sorta.)

Structure: sort people by whether they are inclined to incremental (intelligence can be learned) vs entity (intelligence is largely fixed) theory, and whether they made effort (try harder next time) or ability (guess you're not really smart after all) attributions, compare their response to difference stimuli: puzzles or offers of remedial help in their English language skills.

Findings: Your beliefs about human potential tend to impact your ultimate achievement levels.

Job et al - theories about willpower in everyday life

Implicit Theories About Willpower Predict Self-Regulation and Grades in Everyday Life
Veronika Job1, Gregory M. Walton2, Katharina Bernecker1, Carol S. Dweck2
1 University of Zurich 
2 Stanford University
Journal of Personality and Social Psychology, in press.

Laboratory studies have shown that when people believe willpower is abundant, they exhibit better self-control after a demanding task -- this article attempts to assess the phenomenon in everyday life. Basic finding is that people with non-limited theory do indeed have more success and procrastinate less in everyday life.

Structure: students self-reported their schedules, activities, and whether they were focused on pro-academic behaviors or gave in to impulses. At the end of the quarter, where consent was obtained, their grades were compared against the results as well. Thinking about your abilities in a limited way seems to lead you to give up earlier than necessary. (Essentially, your belief structure becomes a cop-out.)

Findings: "Most pertinent to the present research, Job, Dweck, and Walton (2010) found that people’s implicit theories about willpower determined whether people showed ego depletion at all." This assertion appeared to be validated by the study.

Week 10 - Reflection

This week's mindtool was about stress. The text took the position that stress is negative, although in our class we seem to be incorporating both "eustress" and "distress" concepts -- the notion that stress can enhance or detract from our performance, happiness, etc. I found the 'Wheel of Life' exercise to be useful; it's a tool similar to things I've used before, but I can see it being beneficial if done multiple times. I'm putting it into my calendar as a monthly recurring item; it'll be interesting to review the collection as it evolves over time -- maybe after 6 months or so, I'll look back over the set. Overall, I was pleased to find that my life right now is not very stressful. I would have answered many of the questions much differently six months ago, but the changes I've made to my path seem to be working; I'm healing nicely and moving forward. It feels very good! The list of "think on your feet" strategies was a nice summary of what seems like "common sense" -- although that may be my hindsight bias talking! At any rate, I appreciated having them all collected for reference.

One area I'd like to improve on, and learn more about, is relaxation techniques. I definitely do not have a relaxation practice of any sort, and it would probably help me a great deal. The list in the text was friendly and not challenging; I also appreciated that it's not pseudoscience or new-age-y.

The readings this week were journal articles which explored people's beliefs about themselves and about human nature, and how these beliefs seem to impact performance -- including intelligence, ability to resist temptation, and so on. What I keep returning is the fact that these beliefs are likely at least somewhat correct, but they're not particularly useful. This is a very interesting notion to me, the idea that knowing the truth, or at least focusing your outlook on that truth, can actually be detrimental. I suppose focusing on the positive side of things is what helps humans keep going at any rate -- if we thought about all the things that can kill or hurt us in the world, we might never get out of bed (which would give us obesity and blood clots and depression, and also kill us). I think my latter framing of the seeming contradiction -- that focusing your outlook on things you can't change is not useful -- is the key takeaway. I suppose it's something like eating fruit from the tree of knowledge and becoming enlightened as to the nature of good and evil; once you see it, you can't un-see it, even if it harms you.

What I wonder about is whether knowing this would ever show up properly on experimental studies. Given the prediliction psychologists have for studying college students -- and even college psychology students -- it seems like people's performance on these tests would be skewed over time, or skewed by prevailing psychology theories. It seems like a more precise instrument is needed, one which distinguishes between "facts I think are true" and "attitudes that shape how I live my life day to day". Believing that intelligence is not infinitely malleable does not stop me from working hard to learn, and to teach every child in front of me -- confronted with a question as to whether intelligence is infinitely malleable, which way should I go? With the attitude that reality seems to predict, or with the value/guiding principle that shapes my actions? The other question to investigate further is whether training had an impact -- i.e. are we looking at innate or at least semi-fixed attitudinal traits, or will people readily adopt a different view after some training? CPS training was shown to be highly effective; what about willpower training?

I'd like to see the other side of the research -- in terms of limitations in willpower, how past behavior predicts future behavior, or the "fixed" nature of intelligence -- since right now we have only read the arguments that knock down these ideas. I know that there are a great many economic studies that show how one's birth circumstances tend to predict the path of one's life. I have no doubt that people can get smarter or be more creative, but I'm not sure I can give up all idea of limits, innate tendencies, and so on.

I did find it disappointing that the researchers factored in "unhealthy foods" as being sweets or salty snacks, since these items are reasonable in moderation. I think it would have been reasonable to ask about "overeating" of these items (like 'overspending').

It would be interesting to know if the studies have corrected for the possibility that the reason some people see capacities as unlimited is because they themselves have higher/less limited capacities, while those who see limits in part see them because they actually have them, or experience them.

I've seen studies that showed how a good teacher could make a letter-grade worth of difference to a child -- which is great. But the other 80% of their outcome was dependent on what they walked into school with, what they did themselves while there, and what they went home to. Children of poverty tend to be poor when they grow up; the "brightsiding" argument might tell us that they ought to pull themselves up by their bootstraps, stick out their chin, and go succeed -- and if they don't, it's their own darn fault, and we ought not subsidize them in their laziness.

Severe poverty which makes everything a struggle, or even the poverty of the working poor in the United States, seems like a huge burden on one's willpower -- by contrast, having a really big problem set due (the "severe stress" in the stress study) doesn't even show up when graphed against stresses that are at the bottom of the Maslow needs hierarchy.

Even for the same task -- the stress and willpower required to get a kid out the door in clean clothing every day -- the cost in time and effort seems much greater for people of lower means. For an upper-class family, perhaps someone comes in and helps with the laundry and the clean-up, or a transportation company comes to the door. For a middle class family, someone at home has to do the laundry, but the machine is probably right there in the house, or down in a laundry room, and either the mini-van is parked out front or the school bus drives past the house. For a poor or working-class family, that laundry might mean a 4-hour commitment on a Sunday (when the other families might be out playing, or shopping, or resting), and getting to school on time might mean a long walk or a prolonged trip on public transit.

The other element I didn't see incorporated is how the cycle of willpower expenditure and revival functions in the face of setbacks. I suspect that having to work very hard for very low results over time would sap one's overall ability to recover.

The study about children judging certain traits as fixed is an indicator that good information about human potential should be taught at all ages. Children need to know that they can change and improve themselves, and that hard work will have a reward. That said, it seems to me that beliefs about sociomoral stability are very reasonable and evolutionarily advantageous in the short term -- the kid who bullied you for three days straight is heading your way, you probably don't want to stick around to see if he's softened up overnight, the berries that were poison yesterday are probably still poisonous today. It seems like it would take substantial education to help people to categorize traits as mutable. Children in particular do not have the experience of a long life in order to take things into perspective or understand how substantial change can happen over time. Asking children to assess whether someone will change over time may not be particularly fair, unless you have a method to gauge someone's sense of protracted time.

Mindtools on Stress

Strategies reviewed:

• Keeping a stress diary
  • include how well you responded to the situation
  • Provides perspective on the sources and frequencies of stresses
  • a few weeks is most helpful, after that the benefits reduce
• Job Analysis
  • decrease job stress by understanding your situation better
  • focus on areas you can improve (prioritization, delegation, communication)
  • make sure your behavior, skills, and your work environment are aligned by understanding each
• Performance Planning
  • Think through your performance in advance
• Imagery
  • Calm yourself by imagining a calm place
  • Focus yourself by imagining excellent performance
• Physical Relaxation techniques
  • Deep Breathing
  • PMR - progressive muscle relaxation. Tense, then relax, then really relax.
  • develop a 'Relaxation Response' -- a series of meditation-like steps to get yourself to a relaxed place
• Thinking on your feet (techniques)
  • Try to relax
  • Listen closely
  • Ask for the question to be repeated
  • Use stall tactics
  • Use silence
  • Stick to one point and one supporting piece of evidence
  • Prepare some 'what ifs'
  • Deliver clearly (not the same as loudly)
  • Summarize and STOP
• Rational Positive Thinking
  • Observe your own thoughts and feelings. Are you feeling inadequate? Judging yourself more harshly than you would judge others? Being perfectionist? Worrying about things outside your control? Worrying about an upcoming performance?
  • Spend time replacing your negative thoughts with positive ones, the kind a friend would give you as reassurance.
• Positive/Negative Thinking

  1 - 3
  2 - 5
  3 - 3
  4 - 2
  5 - 3
  6 - 4 
  7 - 5
  8 - 4
  9 - 5
  10 - 5
  11 - 3
  12 - 3
  13 - 4
  14 - 5
  Total: 54
  Result: Great job! You have a generally positive and optimistic outlook on life. You don’t take things personally and you are able to see that setbacks won’t ruin the rest of your life. Even then, if you are experiencing negative thoughts, do some work to iron them out.
  Permanence: 13/20
  Pervasiveness: 21/25
  Personalization: 20/25
  

• Anger Management
• Burnout
• Self Confidence
• Boosting Your Self Esteem
• Locus of Control
• Wheel of Life

D&M Ch 6 Concept Questions

1 - Why are RC sometimes overconfident?

• Hindsight bias -- tend to only think of reasons that reinforce their own thinking
• Hard/easy effect -- if we recognize a test as 'easy', we may not see our own errors

2 - What are techniques to improve accuracy?

• Practice tests
• Consider your own past results
• Recognize own mental state
• Think about probabilities/mathematical approaches, not just your own experiences
• Don't work too hard on 'outsmarting' the test or experimenters

3 - Why do groups contribute to overconfidence?

• Make it less likely that you consider alternatives
• Susceptible to frequency bias -- as you repeat your answer to one another, you may become more confident
• Socialization; desire to get along with others and assert own role in group may cause you to argue more strongly for your point of view

4 - What's the purpose of retrospective confidence judgements?

• Deciding whether to answer a question
• Deciding how strongly to argue a given position

D&M Metacognition Ch 6 Discussion Questions

1 - using metacognitive judgments while studying for a test

Judgments of learning:

• function: as I'm studying, making an assessment as to whether I'd studied an item enough
• might impair my studies by causing me to skip over items that seem too hard or too easy to be worth the time
• might foster my performance by helping me to optimize my time
• Useful heuristic: if I make a flash card on the topic, do I know the answer to the question
• Would be useful if... I have made a flash card that represents what will be on the test
• Would not be useful if... I have low retention beyond the study session

Feelings of Knowing:

• function: how strongly I feel confident about the item
• might impair my studies by being distorted by my emotional state -- am I stressed, depressed, etc.
• might foster my performance by encouraging me to re-study even items I know
• Useful heuristic: When I look at the question on my flashcard, do I know if I know the answer, or am I guessing
• Would be useful if... I can assess the accuracy of my answers
• Would not be useful if... the answers are open-ended or not provided

Retrospective Confidence Judgments:

• function: after answering, checking back over to see if I'm right
• might impair my studies if I allow myself hindsight bias (doing the exercise in my head)
• might foster my performance by take a practice test, assess my own performance, and then repeat it
• Useful heuristic: how well do I usually do on tests of this type?
• Would be useful if... I've taken similar tests and accurately recall my results
• Would not be useful if... the other tests were not comparable

D&M Metacognition - Ch 6 notes

Chapter 6 -- Confidence Judgments

Retrospective Confidence Judgments -- AKA RC
This is when you generate an answer as well as judging our confidence in whether the answer is correct. A type of metamemory judgment. Useful in memory regulation. Confidence is central in our decisions to withhold or share an answer, as well as whether others will believe us. Calibration -- how well your level of judgment matches your level of performance.

Factors Influencing Retrospective Confidence

The Overconfidence Effect and The Hard-Easy Effect
People tend to be overconfident when challenging tests, and underconfident with easy ones. May be a systematic cognitive bias, but another theory is that experimental design is to blame; psychologists sometimes trick people.

Debiasing Techniques in Retrospective Confidence Judgments

Some techniques are process-oriented -- how the person judging will process and represent the relevant information. Other techniques are response-oriented -- informing people of the level to which others are overconfident. Response-oriented approaches may not be advisable since the judge is not pushed to understand the structure of the problem, source of the bias, etc. Process-oriented Technique: Give a reason for your answer.
Hypothesis: due to personal bias, participants are likely to be even more overconfident as they confirm their own answer.
Result: generating reasons did not impact overconfidence. But generating reasons they might be wrong did reduce overconfidence. But other studies have contradicted this finding.
Bottom line: giving reasons or counter reasons doesn't seem to be enough to consistently help people to do better.

Response-oriented Technique: Give participants feedback about their overconfidence.
Performance feedback -- overall accuracy feedback on a series **study of this showed almost immediate impact, taking overconfidence down to almost zero. Maybe this is the research basis on the benefit of the 'practice exam'?

Depressive Realism Hypothesis
Some studies show less overconfidence in people who are depressed. Certainly depression lowers overall performance. The depressive realism hypothsis posits that when people are depressed, they're more realistic; non-depressives are overconfident and unrealistic. Seems to be true that many people are overconfident. A competing hypothesis is that they're simply more negative, not more realistic -- they're selectively processing negative information.

Non-depressed and depressed people seem to have similar performance on memory tasks, but still judged selves poorly, seeming to validate the selective processing theory.

Confidence of Groups We know that RC when playing trivia alone can reveal substantial overconfidence. Are "two heads better than one" in this case? Groups perform better, although perhaps only slightly better. Confidence increases with groups, whether correct or incorrect -- even when students are trained in metamemory, confidence, and calibration. Ultimate conclusion: groupthink may improve performance, as well as inspiring undue confidence.

Theories about Retrospective Confidence

Why so overconfident? Pessimist: people are just bad at judging their internal state (Tversky and Kahneman 1974 - people use poor heuristics). Optimist: experiments are set up in ways that make people look bad (Gigerenzer - probabalistic mental model; experimenters use tricky questions).

Pessimists People don't always use probability correctly (ex: judging 'Linda' with two traits instead of one, when likelihood of two traits is likelihood of X * likelihood of Y) T & K found this 'conjunction fallacy' in numerous domains, and even experts make this error in their own domain. T & K say that people are led astray by a 'representative heuristic' -- characterizing results based on a population or pattern rather than 'playing the numbers'. Another faulty heuristic is the availability heuristic. This is when you judge something more probable because you can retrieve instances of it. Often accurate, but can produce bias due to the ease of recollecting something, or on your own experiences. Another faulty heuristic is the anchoring-and-adjustment heuristic. This is when you judge likelihood by beginning at an initial value and adjusting -- can be biased by the introduction of some figure (Now how much would you pay??) This is present even when the anchoring value is truly random, not just fixed at some manipulative point! People judge that the anchor they're being given must not be outrageous; surely the experimenter isn't trying to make something too easy or too difficult.

Optimists Judging probability -- ex: you're judging on a 100-point scale, but the question is a yes/no. People may be more sensitive to frequency of past events than absolute performance. Optimists claim that this is part of evolution -- people tended to have access primarily to their experiences, not the average experiences of all people, so tend to focus on that information. Optimists who re-ran the "Linda" experiment above but used frequency language instead of probability language, and fewer people were tripped up by the fallacy. But attempts to apply this finding more broadly have been inconclusive, so this is still an unknown.

Intuitively, the probabalistic mental model seems to explain low accuracy of RC judgements. Idea is that either you have the knowledge and feel quite confident about it, or you must infer an answer. If you must infer it, you may pick a valid/useful cue, or you might not; the cue you pick might be quite good but just happen to be wrong in this case. Example good heuristics for relative city size ("does the city have a basketball team") -- might be no good for "tricky pairs" as chosen by the experimenter. If the pairs were random, your heuristic would be good. Seems like this should be testable via real-life field conditions rather than in an experiment, if a real-life situation can be found that cleanly tests for this....

Key hypothesis: representative questions should show excellent calibration -- and this seems to be true.Oh good, optimists and pessimists are both right! Let's synthesize!

A hybrid approach

Humans judge their accuracy well if the questions reflect their real-world experiences, but are sometimes biased. Dougherty's model of 2001 integrates these approaches, combining both optimistic and pessimistic factors as well as incorporating a formal model of memory.

Relationship between RC and people's knowledge about metacognition

From Brewer et al - theory is that when making an RC judgment, people use both their knowledge and their metamemory beliefs. Hypothesis to test -- if people recall the episode in which they learned something, they're more confident in the item. Example study -- ask people to recall if a sentence is identical to one they studied. Experimenters used a "deceptive lure" where the meaning of the sentence was unchanged but a single word was changed out for a synonym. Result: recalling the study episode increased confidence even if you were deceived -- more confidence, but not more accuracy.

Convergence and Divergence in Thinking About Confidence

Debate continues as to whether it's more accurate to use heuristic-based or ecological approaches to assess human judgement. Ecological approach assumes some heuristic use. However, debate continues as to whether the issue is indeed with people using bad heuristics or bad experimental design.

Function of Retrospective Judgements

Ex: test with short-answer and multiple-choice questions. How do you decide on your answer, and whether to answer, and whether to change an answer? RC helps with this. In studies, people often are asked to recall all they can, which weights volume -- but in a jury trial, witnesses are told to tell the whole truth and nothing but the truth -- both volume and accuracy are weighted equally.

Mind Tools -- Decisionmaking

There are six steps to making an effective decision: 1. Create a constructive environment. 2. Generate good alternatives. 3. Explore these alternatives. 4. Choose the best alternative. 5. Check your decision. 6. Communicate your decision, and take action.

Pareto analysis

- brainstorm options - group and score them (ex: resolve complaints by addressing the one complained about most frequently) - use the '80-20' rule

Paired Comparison Analysis

Compare each item in a list to every other item, 1 to 1.

Grid Analysis

List the options and the relative quality of the various features. Then weight the features.

PMI -- Plus, Minus, Interesting

Plus factors, minus factors, and outcomes, implications, etc.

Force Field Analysis

Factors for and against a decision.

Six Thinking Hats

• White Hat -- what data is available? Look at trends, extrapolate.
• Red Hat -- try using emotions, intuition, gut reaction (self and others)
• Black Hat -- look pessimistically, cautiously, defensively
• Yellow Hat -- think positively, look at the benefits
• Green Hat -- creativity, freewheeling thinking
• Blue Hat -- process control. Switch between the hats.

Starbursting

Understand new ideas by brainstorming questions.

• Who, what, Why, When, Where, How
• Just work on questions

Stepladder Technique

Encourage entire group to participate. Basic idea -- start with two people, who come up with some ideas. Add a third person who presents their initial ideas first. Then lay out options. Then add fourth person, who lays out their ideas first, then lay out all options. Reach a final decision only after all members have discussed. This sounds really interesting, but I'm not sure how well it would function in a workplace context. Successive half-hour meetings?

Delphi approach keeps ideas anonymous with an objective facilitator; takes longer.

Cost/Benefit Analysis

Measure costs, measure benefits. Assess payback time.

Cash Flow Forecasting

Spreadsheet showing incoming and outgoing funds.

Decision Tree Analysis

Start with a decision. Draw out lines for each possible solution. Consider results of each, and any decisions or uncertainty. Build out tiers from each decision. Assess likelihood and costs of final branches, then work backward.

Reflection -- Week 5

Chapter 6 in Dunlosky & Metcalfe discusses the concept of Retrospective Confidence, building up our understanding of the last metacognitive monitoring processes presented by the basic model of metacognition -- and at this point we've talked about all the angles of how/whether we know what we know or think we know. Retrospective confidence is a post-performance measure, while Judgment of Learning is a pre-performance measure and Feeling of Knowing is a during-performance measure. One of the findings that I found most intriguing is that absent conclusive evidence about one's performance on a given task, it's quite difficult to assess that performance. This may seem obvious, but society in general does not have many immediate/natural sources of feedback to help us guide and improve our performance, especially once we become adults. Performance reviews from bosses are rarely given more than once or twice a year if at all, asking a spouse or friend for feedback on one's social performance can be seen as fishing or simply awkward, and traditional learning approaches more often include questions without answer keys or rigorous processes for coaching people to understand their results. Day to day activities do not typically include "coaching" once some level of mastery is attained (driving, time management, cooking). Other types of feedback loops are more subtle or episodic (parenting, work performance, creative writing).

One topic I don't fully understand is how to more effectively counter the impact of poor retrospective confidence as an individual in real-life situations; it seems like reflection offers some insight, but it seems difficult to do this correctly without outside feedback.

I appreciated reading the efforts to understand the attempts to assess how psychological testing techniques might bias results; in particular it seems like real-life performance examples might be useful in making headway on this question. One example of a real-life situation is longitudinal studies of performance on tests like the SAT or ACT; another might be something relative to resistance to financial fraud or deceptive advertising.

In Chapter 7, we dug more deeply into source monitoring, and how this function can go awry. Under stress, or with age, the ability to tie contextual information to a particular memory can diminish. A crime victim might blame a tv star; we might think we thought of an idea when really we read it somewhere; a name might seem famous when really the name was just on a list you saw. Vivid dreams or readily-made images can also lead us to mistake reality with things we only imagined.

One area I still have questions about is how we might be able to recover, or restore, our sense of reality or a weakening source judgement. I wonder if there are types of therapy that are useful, something like sensory integration or other mind/body activities that help people to recognize or rebuild the connections between action and reality, or distinguish inside versus outside.

This week we also completed the Decisionmaking mindtools section. This section proposes and describes several methodologies to aid in making decisions. Using an instrument like the ones in the book seems useful in several ways: to declare and articulate a process; to encourage decisionmakers to consider angles more fully; to make explicit and document a decisionmaking process; to provide a mechanism for communicating a process. Establishing a decisionmaking process seems like a great way to break through analysis paralysis in project management, too.

In the working world I have seen many instances where a past decision was known but the thinking process used to arrive at that decision was unknown -- perhaps the decision was made by people were no longer there, or the thinking could not be recalled by those who were there. Those who were present when the decision was made are left in the unfortunate position of trying to speak to the decision and give a full accounting of it, which unfortunately is not completely reliable due to faulty recall. Oral historians can end up blamed for what happened. A more Machiavellian type might use the opportunity to manipulate the situation. Creating a decisionmaking document seems like an excellent way to prevent many of the unfortunate consequences of undocumented decisions -- repetition of the mistakes of the past, wasted work on re-discovery, paralysis because "it's always been that way" or "there must have been a good reason, and it'll bite us if we go back and change it now".

Mind Tools - Practical Creativity

Improving a product or service – Reversal and SCAMPER

Reversal
Use: As a brainstorm prompt to make something better How-To: think about how to make it worse

SCAMPER Use: Method of design thinking - prompts to ask yourself in coming up with something new How-To:

* S – Substitute – components, materials, people. • C – Combine – mix, combine with other assemblies or services, integrate. • A – Adapt – alter, change function, use part of another element. • M – Modify – increase or reduce in scale, change shape, modify attributes (e.g. color). • P – Put to another use. • E – Eliminate – remove elements, simplify, reduce to core functionality. • R – Reverse – turn inside out or upside down, also use of Reversal.

Creating or improving products, services and strategies: Attribute Listing, Morphological Analysis & Matrix Analysis.

List out the attributes, think about how they might be different.

Morphological Analysis

Generating many radical ideas – Brainstorming and Reverse Brainstorming.

Use: trying to come up with as many ideas as possible, without criticizing/shut down
How-To: as individuals, as a group, with a random seed to prompt

Widening the search for solutions – Concept Fan.

Use: seeking solutions outside the obvious How-To: write the problem, then fan out options to solve. Then ask, is our problem actually a symptom of another problem, and analyze that problem. Continue as long as it's useful.

Looking at problems from different perspectives – Reframing Matrix.

Use: trying to think about the problem in new ways How-To: Put the question in the middle of a grid, and then the 4Ps -- a box for Product Perspective, Planning Perspective, Potential Perspective, and People Perspective. Or you could look at the problem from the perspective of multiple personas (doctor, technican, patient, insurance company).

Making creative leaps – Random Input.

Add a random word to your brainstorm

Carrying out thought experiments – Provocation.

Use: make deliberately stupid statements How-To: think about the consequences, benefits, how it might actually be sensible, how it might work, and what might happen. Shortcut from de Bono: Po (for Provocative operation)

A simple process for creativity – DO IT.

D – Define problem
O – Open mind and apply creative techniques
I – Identify best solution
T – Transform

A powerful integrated problem solving process – Simplex.

Use: 8 stages in a ring -- Problem finding, fact finding, problem definition, idea finding, selection, planning, sell idea, action.

A powerful approach to creative problem solving – TRIZ

Use: Look for a generalized solution from another domain How-To: catalog of solution patterns, lateral thinking

Reflection -- Week 9

This week, I worked with the notions introduced in MindTools and continued to mull over the experience of the Week 8 lecture. MindTools explored two topics -- problem solving, and practical creativity. The problem solving section introduced a range of tools and structures that can be used to tackle a problem. I did find I wished I had a case study to work from, but eventually found my own examples. I had seen some of the techniques before, but others were new -- and the obvious thought occurred to me that these are all techniques used by consultants. A business brings in consultants to give them perspective on their problems, and the consultant uses high-level problem solving techniques to analyze that problem and provide recommendations back to senior management. But these tools are not the exclusive domain of consultants -- they can be used by managers and leaders as well as individuals.

I am a particular fan of various matrix-oriented analyses -- the tools seem to communicate effectively and . The 'Fishbone' approach (which very specifically put me in mind of a consulting firm once hired by a past employer) seems useful for brainstorming and problem exploration.

One topic I don't fully understand is the McKinsey 7S approach. I understand each of the 'S' areas fairly well, but when it comes time to express alignment between each S, and to represent that as a 'before' and 'after', I get confused about the purpose of the analysis, and it seems like pieces are missing. There's no problem statement, goal statement, etc. For example, if our preliminary analysis indicates that our structure is "highly decentralized" and our strategy is "distributed control", what can we say if the change being made is to centralize operations (changing structure and changed our strategy) -- we've stayed 'aligned', but there's no articulation of the object of the change (e.g. cost reduction), and the change in strategy is not represented. If we sketch out the 'after' state in advance, it seems like there will be a tendency toward optimism -- which makes the 'after' state uninteresting. If investigate the 'after' state later, it's not much different than the exercise of the 'before' activity. It seems like the model needs to indicate what was changed and the intention of the change, not just the alignment of factors.

The many different approaches to developing creative ideas was very inspiring, and I'm eager to try them all out. It makes me think of the idea that problems are 'golden eggs' to Japanese companies. The TRIZ approach particularly resonated with me because there is a similar line of thinking in Computer Science, called Design Patterns. The basic idea here is that there is a set of repeated patterns in software architecture, and that these ideas are durable and useful independent of a given implementation. In fact, one of the leading minds for design pattern thinking is written entirely from a physical builder's perspective -- a traditional architectural view rather than a technology architecture angle. (Timeless Way of Building)

In the week 9 lecture, we discussed dimensions of creative activity -- an approach that reminds me of Costa's Habits of Mind approach to critical thinking. It seems like we can't discuss creativity very far without running into critical thinking, and vice versa. I suspect this theme will continue throughout the degree program. In the live lecture, we had an animated discussion about divergent thinking and creativity, including whether CPS or something similar is inherent in creativity. We defined creativity as being both useful and novel, originating from divergent thinking but ultimately convergent so that the results could be shared with others. The MindTools section drew a distinction between "artistic" creativity and "technical" creativity, and said that CPS activities are intended for the technical type of creativity. This seems wrong to me, but some students in the class felt that their own creative activities did not reflect any kind of CPS. To me this seems more like a lack of awareness or mindfulness in process, rather than the true lack of process.

I have heard many artists talk about their creative process, some in greater detail than others, but it does seem to me that there is a process underlying it all. CPS itself has become so flexibly defined that I think there's a "big tent" definition that can bring all creativity underneath such that awareness of CPS would help with any creative activity. It may become a semantic discussion after a while, in which certain people characterize their work as having process but not CPS, even though they can't define their process. But among those artists who I've heard describe their process in detail, it rather sounds like CPS, even if the words used are somewhat different. After all, CPS might just as well be described like this: "I started out just noodling around with this idea, and then I realized that I should explore A B C in more detail, and then I started doing these things, and now I'm finishing off by...".

It seems like the alternative to CPS is purely divergent activities -- but even the more avant garde types of art have a CPS within them, even if the CPS is entirely in the planning/setup activities while the execution is left to chance. Dadaist poetry strikes me this way, or modern pieces randomized by computer. Another example might be mind-challenging pieces like Gertrude Stein's tender buttons. She seems to string random words together, but they are not random -- there are puns and references and odd repeated refrains to "make the familiar seem strange". Dissecting language, whether with a wrecking ball or a scalpel, still requires that someone first think about how they might accomplish it, try to do so, assess their work, and so on.

Another subject we dug into more deeply in the lecture was the question of whether there are cases where CPS is not appropriate. There is an argument to be made for "just doing" things rather than analyzing them, although this is not the argument on which we focused. Primarily we talked about problem generation as a necessary catalyst for CPS. KT Ulrich in Design Thinking calls problem generation "sensing the gap" -- and he goes on to outline what ultimately is a particularly elegant case study of CPS activity. Perhaps in more "artistic" types of creativity this might be called inspiration, but a more elaborate definition might be "sensing a thing that needs to be expressed" or "finding a subject that one wishes to explore more deeply" -- in other words, the finding of a "problem".