Metacognition Class, Self-Reflection -- Week 2

This week we really dug in to the math and methodology of both assessing people's metacognitive performance as well validating the components of the Nelson model.

This is fascinating stuff to me and really gets my wheels turning for the class project. I have a notion of storyboarding a game for students that would allow them to measure their own FoK, calibration, JoL, relative accuracy, etc. all while working to memorize particular facts that they're interested in -- language learning might be an easy example, but this also makes me think of topics like the bones in the body or the meanings of various suffixes and prefixes in biochemistry. I've heard that medical students in particular tend to be people who are very strong at memorization and focus on large amounts of material, so targeting these knowledge domains might be very productive. The overall paper I submit could include something like a business plan. The program could calculate the key figures and allow people to choose among either a bank of items or enter their own (which could then be socially shared). The results could then give them advice on studying tips, customized to their personal situation.

I found myself wondering about the methodologies and potential datasources in something like the SAT as a 'laboratory' for analyzing FoK and JoL. Students can mark answers for review and go back to them. Surely the data is recorded and probably analyzed -- what does it tell us?

The discussion question concerning a study design for assessing metacognitive growth over time was an interesting one. As a parent, I've seen the 'lightbulb moment' and 'sudden physical leaps' in a child's development -- not only with mastery of a skill like walking, which comes in stages but also has a moment of blossoming within a short span -- but also with certain psychological and intellectual phenomena, like speaking in sentences or suddenly realizing how strings of letters make words. All this makes me think of the deliberate decision we made to take our child to try to take our kids to do psychology tests when they can. The games are ok, the pay is ok, but the payoff from our perspective was helping them to be comfortable working with adults who are assessing them -- maybe in strange ways -- to help them be ready for standardized and IQ tests. The ability to talk to them about science and to see scientists in action (often women) as also significant. Based on these readings about metacognition, it occurs to me that this might also help them to improve their own self-awareness. I always wonder if there is a way to detect and correct for 'bad faith' subjects -- people intentionally getting things wrong -- or 'testing super fans' -- people whose exposure to tests of this type makes their results less applicable for understanding the population as a whole.

One surprising thing I learned through this material was how powerful it can be to combine a fairly simple model with fairly simple and well-designed experiments. Having done a lot of psych experiments at the U of C over time, I always wondered what was 'behind the curtain' and this insight into study design was very interesting. Sometimes you read about psychological models and you wonder how much "real science" is behind the model. I suppose we can once again thank the behaviorists for calling the field into account.

One topic I have questions about is retention -- both on a personal level and overall. What do we know about retention? The text seems to gloss over this, although the MindTools discuss a cycle of information rehearsal. I'd like to dig into this topic more deeply and understand how we think it works. It seems like in general direct memory declines more quickly than feeling of knowing. What's the best technique revive faded memory, prevent memory from fading, or restore memory after it has faded beyond even the feeling of knowing, but really was once known.

One thing I don't understand is this discussion of 'gamma'. It may not be important in the longer term, but I'd like to get my hands on some more information about the math and calculations used for this type of work. I'd like to be able to do some problem sets or otherwise directly apply the formulas to data from experiments. Also, in class, I asked the question as to whether memory is all there is -- is the model we have really comprehensive. The answer, which I didn't fully understand, was essentially 'yes'. I am still struggling with this, however, because it seems like there is such a thing as instinct and other inborn cognitions/strategy/pre-wiring. And the distinction between conscious and unconscious thought is not really included in this model either. There are plenty of things that I'm thinking about, even right now, that don't have to do with pulling up memories. Fuzzy matching and pattern recognition might be another example -- things that human brains at least today seem to be better at doing then computers. The question to build on these things would then be, do we have a way to measure, enhance, model, understand, etc. those parts of the brain. There are parts of my behavior and my thinking processes that are not particularly learned, as far as I know. Hallucination is another potential example -- you're not remembering or perceiving, but rather inserting notions from somewhere else into your perception as it comes in, and not consciously so. I may not have framed the question correctly, because I am not very satisfied with the answer. So I'm still struggling with this topic.

This week's mind tools activity was about effectiveness through time management, goal-setting, and control of distractions. I found the material to be mostly review, but also a very helpful reminder. I'm aware of these techniques for getting the most out of your time, but I do fall away from time-management habits, especially when my routine shifts.

For example, when I was regularly going to work on an early schedule, 7 - 3:30 or so, the first thing I did each Monday was to create a weekly plan with goals and to-dos and so on. I felt very calm and in control of my work. But when I was no longer following this schedule, my planning time became erratic, and I started to lose some of the benefits of the approach I was taking. When I left my job, my schedule became very disrupted -- and due to a combination of personal and professional factors, I would find myself stepping in and out of routines. My life is starting to settle into a little more of a routine now, but I anticipate that it will change substantially this week, and again in a few more weeks, and even more dramatically a few weeks after that, again due to changes in work and changes at home.

Time-tracking is the same way -- when I track my time, I have more control over how I use that time. It's actually really rewarding to feel that slight resistance, when I'm just-about-to-get-distracted, and my brain says "no! focus! if you change tasks, you'll have to put the change on your timesheet!". It's very helpful as long as I stick with it. For now, I'm using the 'Mind Tools' document since it's assigned but my favorite tool for this is the toggl.com app I used at work for a similar "I want to enhance my personal effectiveness" purpose.

My overall self-assessment is that at the macro level, I'm making effective use of my time and moving in all the right directions, but at the micro level there is a lot of running back and forth without as much focus. I make the right moves, but it takes me longer to figure out that move than it should, and as a consequence I waste time. I'm just too reactive right now. I feel like actually changing my behavior is lagging about 2 weeks behind the circumstances that necessitate that change.

To address this, I need to reflect on how to carve out a little more consistency for myself despite all the changes I'm surrounded by and participating in, because the disruption and constant changes to routine will probably not end for several years.

One of the positives about doing this review is that I found that some of the 'advanced' recommendations in MindTools -- the action programs -- I'm already on top of and have been keeping up to date. I use multi-tab Google docs to keep programmatic track of each of several large identities I'm balancing right now (being a student, running two businesses, coaching soccer). Funny how my feeling about my life right now -- macro in good control, micro not so much -- is mirrored by the health of my systems, and the 'macro level' organization (big tracking spreadsheets with long-term goals and projections) is fairly healthy but the micro level is not (to do lists, weekly goals). It's encouraging to know that I'm reasonably self-aware and aligned, and it makes the micro-overwhelm seem so much more tractable. I'm not sure it will make me less reactive, although it won't hurt. And more reflection time may fill in the rest of the gap I'm experiencing. Here's hoping.

Class #3 Notes

Questions: what about pattern recognition? more information on retention? Just rehearsal?

Metacognition D&M Ch3 Concept Review

Concept Review

What aspect of methodology distinguishes between the various metacognitive judgments -- JoL versus FoK versus Ease-Of-Learning?
This part of the Nelson model is called Metamemory Monitoring

Why are various metacognitive judgments linked more closely to different aspects of control? Why JoL --> self-paced study, FoK -->retrieval?
Judgment of Learning is linked to self-paced study because, given flexible time to study an item, the assumption is that people will typically study an item until their JoL is high. Feeling of Knowledge is linked to retrieval (rather than recall) because it is a measure of whether you think you would be able to recognize the answer. If you have a high FoK and then do indeed recognize it, your feeling was more accurate.

Compare Relative Accuracy and Calibration
The distinction between relative accuracy and calibration is that relative accuracy is a measure of your personal self-consistency -- this is important because in responding to questions like FoK and JoL, we are applying a mathematical scale to something we probably don't or maybe even can't internally measure mathematically. If someone is highly consistent, you can correct their numbers to smooth out the impact of personality/psychology due to their overconfidence/underconfidence. Measuring calibration is a way to see just how well attuned someone is to their actual condition internally -- are they overconfident, under confident, etc. Relative accuracy might be measured on a per-item basis (Yes, I think that one was right, or No, I'd like to come back to it later, etc.), while calibration based on an overall average at the end ("I think I got an A"). Might be really useful to look at SAT data where students can mark certain answers as ones to be reviewed later?

If JoL is measured on a 6 point scale, can you interpret relative accuracy? How about calibration?
To measure relative accuracy, you can use a 6-point scale, and assess their level of consistency that way (how sure you are on this item, versus how sure you are on that item, and how does that compare to your actual performance.. But calibration deals in direct deltas of accuracy between your judgment and your performance (percentage versus percentage), so you need to use the same numerical basis for the score.

How would calibration curve look for accurate, under confident, over confident?
Accurate is a 45 degree diagonal, under confident is over the line, over confident is under that line. This is because a typical graph places JoL on the X-axis and actual performance on the Y-Axis. Presumably they could be written the other way around, but the convention is to place the independent variable on the x-axis and the dependent variable on the y axis. So your recall is the facts-of-the-matter (did you get it right or not, presuming you'll do your best on the test of course) but your JoL is going to vary based on your skill at self-assessing.

Metacognition - D&M Ch 3 - Discussion Questions

Question 1 - Study design for measuring the growth of the ability to do metacognitive monitoring.
I think there are two approaches to studying how the brain develops over time -- one approach might be a longitudinal study that tracks performance on a set of activities over time, tracked with a single-event study that brings in populations at various age groups. One idea that might be worth investigating in addition is whether this growth is gradual over time or if it is punctuated by some specific internal milestone. So it would be worthwhile to try out various frequencies within the longitudinal study to see if you can find any evidence for a concentrated period where the child suddenly leaps forward in their abilities. At any rate, to assess metacognitive monitoring and control, I think you'd want to collect data for how different ages seem to fare at the memorization task, including asking them to give you a FoK and JoL assessment as they go. You could watch how an individual's calibration curve changes over time, and compare that against a standard age group curve to see if your intervention in testing them also changed the result. Playing 'memory' at home also seems like a way to increase the accuracy of one's FoK, and maybe JoL. Asking if the child plays memory or has played it frequently might be a way to increase understanding of how children develop these skills as part of the study design.

Critical Thinking, DM Ch 5 - Thinking and Learning in the Workplace

From DM Ch 5, pp 23-28
"Learning and Thinking in the Workplace" by John Edwards

Key findings from research

Computer ate my more-detailed notes!!!

• Personal Practical Knowledge: the most underutilized resource
• Weak models of learning
• Linear management vs. action learning
• Understanding skill acquisition
• The Value of Teaching Thinking Skills
• Confusion and Frustration as a Precursor to Learning
• Listening and Talking in Productive Ways
• Challenge-and-support Structures
• Thinking and Learning: Legitimate Business Activities
• Changing Corporate Culture

Metacognition D&M Ch3 Notes

Chapter theme: metamemory research -- results, and methods beginning with Joseph Hart's 'feeling of knowing' work in the 1960s metamemory experiments were like flash cards -- do you know X, test if they're right, etc. tendency to judge study separately from retrieval

Nelson & Narens framework - shows areas of study of metamemory

Three stages of learning:

• acquisition: study
• retention: (...)
• retrieval: tests

Typical process:

1. study word pair
2. ask to make a "judgment of learning"
3. take a test
4. during test, ask for "retrospective confidence judgement"
5. during test, ask if there is "Tip of the Tongue" situation (soon able to recall, would recognize if they saw it)

	Name	Definition
Metacognitive Judgments
	Ease-of-learning judgments	Judgments of how easy or difficult it will be to learn any given item
	Judgments of learning	judgments of the likelihood of remembering recently studied items Immediate JOL - made immediately after studying an item Delayed JOL - made well after studying an item
	Feeling of knowing judgments	Judgments of the likelihood of recognizing currently unrecallable answers on an upcoming test
	Source-monitoring judgments	Judgments made during a criterion test pertaining to the source of a particular memory
	Confidence in retrieved answers	Judgments of the likelihood that a response on a test is correct (AKA 'retrospective confidence judgments')
Control Processes
	Selection of kind of processing	Selection of strategies to employ when attempting to commit an item to memory
	Item selection	Decision about whether to study an item for an upcoming test
	Termination of study	Decision to stop studying an item currently being studied
	Selection of search strategy	Selecting a particular strategy for producing a response during a test
	Termination of search	Decision to terminate the search for a response in memory

Example techniques:
feeling of knowing --> will you recognize?
source-monitoring judgments --> where did you learn? text, in class, etc.

Key Questions Defined and Explored In Research

• How do people monitor memory?
• How accurate is memory monitoring?
• Can monitoring accuracy be improved?
• How is monitoring used to control?

Metamemory: Collecting, Analyzing, and Interpreting Data

Typically people judge their memory and give them free rein on time allocation for memory task. Basic technique has been adapted and made more sophisticated over time. Example -- cognates vs non-cognates have different indexes, so people must use similarity clues from their memories when making metacognitive judgments of learning.

Basic approach: Compare your judgment with your performance. JoL compared to recall performance, FoK compared to recognition performance.

Two kinds of accuracy -- relative accuracy and calibration.
Relative accuracy or resolution, is the degree to which a person's metacognitive judgments predict the likelihood of correct performance on one item relative to another. something like, are you consistent across the sample set, or does your prediction proximity vary widely? If you were highly confident on ones you got wrong, and had low confidence on ones you got right, your resolution would be pretty poor. But if you were 75% confident on all your right answers, and 25% confident on all your wrong answers, that would be pretty good resolution. Consistency of your self-awareness. You can ask participants to judge this on any scale at all. Often computed using correlation -- range of -1 to 1. 0 means no accuracy, 1 is consistently correct, -1 is consistently incorrect. Must be computed for the individual.

Calibration is a ratio of your prediction to the actual answer So if you said 50% confidence for a bunch of answers that were all correct, you'd have a weaker calibration score. Reasonableness of your self-awareness. You have to use the same scale as the answer itself for this one -- say, 1 to 100 for both. The "signed difference score" -- called a measure of "bias" is often used; compute the magnitude of the judgment and the magnitude of the test performance, then subtract performance from magnitude. Negative values indicate under confidence and positive values indicate overconfidence. Kaylea question -- isn't a 'feeling of knowing' result highly susceptible to error? If all the answers but one are ridiculously wrong, doesn't it throw off the scale? And isn't "ridiculously wrong" rather relative, depending on how the thinker is feeling in their ToT? Example: State Capitals. If you list out cities that are capitals of other states, versus large cities in the same state, versus breakfast foods, versus misspelled versions of the correct answer, you may trigger different recognition levels in different people.

Another method: calibration curve
Plot mean level of test performance as a function of varying level of judgment magnitude. If performance exactly matches judgment, you'd have a 45-degree diagonal. Above the diagonal is under confidence (performance is above the judgment), below the diagonal is overconfidence -- performance is below your judgment).

The 'hard-easy effect'
People are often under confident when making lower judgments but overconfident when making higher judgments.

Frontal lobe damage -- relative FOK accuracy is impaired. Suggests that frontal lobe damage makes patients less able to do memory monitoring, and that frontal lobe is part of memory monitoring.

Study design is significant -- for example, since older adults have lower memory performance than younger, you must compensate for performance if you want to assess ability to do memory monitoring. Impact from number of alternatives, correct guessing.

How Is Monitoring Used To Control
Various experiments devised in order to test the overall model of memory. JoL during study --> determine whether to re-study an item. FoK after a test --> how long they persist at trying to remember an answer.

Critical Thinking - DM Section 1 Ch 3. "Thinking in Context: Teaching for Open-Mindedness and Critical Understanding"

Critical Thinking class, Developing Minds Readings. Section 1, Chapter 3. "Thinking in context: teaching for open-mindedness and critical understanding" By Sheldon Berman

Social Problems curriculum as a method for teaching students about thinking/allowing them to sharpen their thinking skills; build up their level of engagement as citizens at the same time.

"An Empowering Methodology"

• Strategy 1: Create a Safe Environment -- safe for risk-taking, safe for emotions
• Strategy 2: Follow Students' Thinking -- four basic questions:
  • What do you know about this?
  • What do you think you know but are not sure about?
  • Where did you get your information?
  • What questions do you have?
  Also respecting and understanding the students' thinking.
• Strategy 3: Encourage Collaborative Thinking -- submit ideas for group consideration, build on each other's ideas, come to consensus, then process these conversations with students. Important just like teaching clear and independent thinking.
• Strategy 4: Teach the Questions rather than the Answers. Pay attention to student questions, model questioning ourselves.
• Strategy 5: Teach about Interconnectedness -- systems thinking
• Strategy 6: Present and have students enter multiple perspectives -- debate, see how advocates of each position view the problem
• Strategy 7: Build on Sensibilities. Pay attention to intuition, to feelings, to ethical considerations.
• Strategy 8: Help students set standards and work from a positive vision of the future. Think about future impacts, broader impacts
• Strategy 9: Provide students with opportunities for acting on their thinking. Present projects, teach other students, do service learning. Local action or at least something where impact can be seen.

Qualities of Mind: Critique and Synthesis. Critique is familiar, but synthesis is harder to teach and do. Critical thinking emphasizes logic, dissection, organization; synthesis emphasizes integrative, global, intuitive, and creative thinking. Crit: Skepticism. Synth: Openness to new/different ideas. Need to nurture both.

Putting into practive -- professional development with ESRNational.orG!!

Hidden curriculum -- are student contributions welcomed and valued? do students participate in the problems and decisions of the school community? does the curriculum of the school place the school in an ongoing relationship with the local community and the larger world? Is the faculty encouraged to think together and work together? Is the thinking of teachers valued and do teachers have an influence of school decisionmaking? --------- Kaylea's thoughts: I appreciate the focus on informed citizenship rather than 'meeting the demands of the economy' -- more apt. We are already making judgements about the kinds of activities that are worthwhile, so pointing to the economy is not enough. The economy has a rather large demand for things that make people miserable or at least don't move society forward, such as human trafficking, exploitation, violence, pornography, and drugs, and yet no one suggests that we ought to better educate children to effectively participate in this activity.

---

Take 2
Notes from "Thinking in Context: Teaching for Open-Mindedness and Critical Understanding"
DM Ch 3 pp 11 - 17
By Sheldon Berman

Theme -- teaching critical thinking in an integrated/inline way -- part of the curriculum. "Nurtured holistically in the context of meaningful problems". Kaylea's counter-thought -- yes, except a 'clinic' with time to focus on the topic, alongside the integrated curriculum, might be more effective?? Might some types of thinkers appreciate having reference to a framework, then applying it?

Whole-language approach to thinking -- "a love of thinking, an ability to reflect on one's thinking and be open to new ideas, an interest in contributing one's thinking to help others and to improve society, and the courage to think through the most difficult and complex problems."

Not only good for education but also citizenship -- fosters a sense of being empowered and accountable to address issues rather than overwhelmed and cynical.

Strategy 1: Create a Safe Environment
Not risk-free, still challenging, but able to make mistakes and express opinions.

Strategy 2: Follow Students' Thinking
Ask 4 simple questions:

• What do you know about this?
• What do you think you know but are not sure about?
• Where did you get your information?
• What questions do you have?

Encourage students to reflect, write down their thoughts, and connect what they're learning to their lives.

Strategy 3: Encourage Collaborative Thinking
Brainstorm, discuss, cooperate -- need to submit ideas to the group, build on ideas, and come to consensus on the ideas that seem most productive, coherent, attractive, etc.

Strategy 4: Teach the questions rather than the answers.

Strategy 5: Teach about Interconnectedness
Network or systems thinking, streams of cause/effect loops, holistic thinking -- appreciate the system you are a part of and the consequences of actions on others.

Strategy 6: Present and Have Students Enter Multiple Perspectives
If we agree with a speaker, we are likely listening for the cogent argument they present. If we disagree, we listen for logical flaws, misinformation, things left unsaid.
To counter this, ask students to try to understand a contrary point of view and see some truth in it, if possible.

Strategy 7: Build on Sensibilities
Include feeling and intuition -- they are not separate from thinking.

Strategy 8: Help Students Set Standards and Work from a Positive Vision of the Future
What are the criteria for a good outcome? How will people by affected 100 years from now? What will the impact be on you/your school years later? **evaluate your own thinking, but also create a positive vision of the future**

Strategy 9: Provide students with opportunities for Acting on their Thinking
Teach other students, act on the problem directly

Critical Thinking -- DM Section 1, Chapter 2. "Thinking Skills for the Information Age"

Developing Minds, Section 1, Chapter 2. pp. 7-11. "Thinking Skills for the Information Age" LeRoy Hay Main points: -- Industrial-age education versus Information-age education: focus on rote learning versus problem solving -- Deep thinkers not needed on the assembly line, but they are needed to do data analysis and use technology tools. -- Rethink memorization -- Integrate technology as a problem-solving tool -- focus on information literacy -- analysis, synthesis, and evaluation should not be the exclusive province of a small segment of the population. -- sense that we are not producing enough knowledge workers Kaylea's reflection: -- do we really and truly have an information economy? a lot of data points to a service economy, which suggests that the need to use those higher-order thinking skills may still be fairly narrow. We put enough kids through college that now we might require it for a lower-level job, but that doesn't mean a college education is required to do that job -- a whole lot of code and other "information" tasks are pretty rote once you get under the covers -- what about quality of life -- appreciation of art, personal fulfillment, creativity, joy, etc. -- who gave the employer the right to access all the firing power of my brain? there's a social value to having capacity to think about something other than work when I get home; Einstein worked in a patent office... -- what about focus, diligence, process, organization? -- what about the neuroscientific evidence for "real" experiences versus "simulated" ones? -- my basic sense that kids don't need computers everywhere, they need experiences, activities, and problems, in a world which includes technology but doesn't revolve around it -- recent evidence for memorization. . Making certain pathways "easy" frees up resources for higher-order stuff to be "hard" . Open book tests might keep you from having to memorize constants, but you still need to be very familiar with the fact that there's a constant to be used. Exact dates can be looked up, but a basic sense of sequence is needed for writing a history essay. Memorization may be a proxy for these more abstract notions -- or one may be a by-product of the other (by memorizing, we know sequence; by learning sequence, we may end up memorizing). How to tease all this apart? -- abstract reasoning **about technology** as a key skill for IT workers

Critical Thinking - DM Reading Section 1, Chapter 1. "Making America Smarter: The Real Goal of School Reform"

Developing Minds, Section 1, Chapter 1. "Making America Smarter: The Real Goal of School Reform" Lauren B. Resnick Definitions & attitudes about intelligence -- innate or learned? Habits of mind seem to characterize the intelligent -- and habits can be learned, even if tendency toward them is inborn. Effort-based Education and Learnable Intelligence. Is not: - grading on a curve - assessing students according to a percentile system Characteristics: - Organize for effort -- high standards, rigorous curriculum - Clear expectations -- descriptive criteria, models of work that meet the standards - Recognition of accomplishment -- celebrate meeting standards or progress toward them - Fair and credible evaluations -- able to prepare for, with exam aligned to curriculum - Academic Rigor in a thinking curriculum. Problem solving as a 'new basic'. . commitment to a knowledge core . high thinking demand . active use of knowledge - Accountable Talk -- reference to evidence and reasoning, not just authoritarian assertion - Socializing Intelligence -- belief in one's own intelligence. - Self-Management of Learning -- metacognitive skills - Learning as Apprenticeship -- model and analyze complex thinking, mentoring and coaching in project work **** Kaylea's reflection: - appreciate the outline of common core, agreement there - sidesteps any assertions about high intelligence and what appropriate education for "gifted" or high-intellect individuals looks like. Even if we decide it's all about habits, some kids have those habits early. Yes, doing great instruction, problem-based learning, etc. for all kids makes sense. But given the differences in kids when they walk in the door of the school, my assertion would be that the answer is not to 'eliminate gifted ed because all kids are equally gifted'. Absolutely give all kids good teaching that uses their brains, that's a no-brainer. But kids who are ready to take on higher-order tasks very soon still need that differentiated instruction or they are bored, at-risk, and can't maximize their potential. If they can leap to the end quickly, how do they learn to work hard, how do they learn to use their strengths if they can succeed without really trying, how do they learn their weaknesses if they never fail, etc.

Metacognition Class Meeting #2

Pre-class questions: How do practitioners use this information about research methods in metacognition? -- measurement indexes like FoK and JoL What do the methodologies themselves tell us about the nature of metacognition? Specifically, how are they giving an indicator of how to quantify the seemingly unquantifiable? --people make predictions about their behavior, behave, and then we measure if they were right How do the readings about time-management make use of our metacognitive strategies? How would we design an experiment to assess whether and how metacognition enables us to engage in time management? --to-do lists free us up from rehearsal and short-term memory "Objective analysis about subjective phenomena"

Critical Thinking Week 2 Readings - Notes

Videos QualiaSoup:
https://www.youtube.com/watch?v=6OLPL5p0fMg

Very Good quality

"refers to a diverse range of intellectual skills and activities, concerned with evaluating information, as well as our own thought, in a disciplined way. When we're willing and able to examine our own capability as thinkers, acknowledging problems and weaknesses, this can help us refine our thought processes, so that we learn to think and assess information in a more comprehensive way that increases our ability to identify and reject false ideas and ideologies. Critical thinking isn't just thinking a lot. A person may spend a great deal of intellectual energy defending a flawed position or pursuing a question that actually needs reformulating before progress can begin."

• analyzing
• conceptualizing
• defining
• examining
• inferring
• listening
• questioning
• reasoning
• synthesizing

From CriticalThinking.org:
Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness.

It entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-issue; assumptions; concepts; empirical grounding; reasoning leading to conclusions; implications and consequences; objections from alternative viewpoints; and frame of reference. Critical thinking — in being responsive to variable subject matter, issues, and purposes — is incorporated in a family of interwoven modes of thinking, among them: scientific thinking, mathematical thinking, historical thinking, anthropological thinking, economic thinking, moral thinking, and philosophical thinking.

Critical thinking can be seen as having two components: 1) a set of information and belief generating and processing skills, and 2) the habit, based on intellectual commitment, of using those skills to guide behavior. It is thus to be contrasted with: 1) the mere acquisition and retention of information alone, because it involves a particular way in which information is sought and treated; 2) the mere possession of a set of skills, because it involves the continual use of them; and 3) the mere use of those skills ("as an exercise") without acceptance of their results.

Critical thinking varies according to the motivation underlying it. When grounded in selfish motives, it is often manifested in the skillful manipulation of ideas in service of one’’s own, or one's groups’’, vested interest. As such it is typically intellectually flawed, however pragmatically successful it might be. When grounded in fairmindedness and intellectual integrity, it is typically of a higher order intellectually, though subject to the charge of "idealism" by those habituated to its selfish use.

Critical thinking of any kind is never universal in any individual; everyone is subject to episodes of undisciplined or irrational thought. Its quality is therefore typically a matter of degree and dependent on , among other things, the quality and depth of experience in a given domain of thinking or with respect to a particular class of questions. No one is a critical thinker through-and-through, but only to such-and-such a degree, with such-and-such insights and blind spots, subject to such-and-such tendencies towards self-delusion. For this reason, the development of critical thinking skills and dispositions is a life-long endeavor.

...

Critical thinking is, in short, self-directed, self-disciplined, self-monitored, and self-corrective thinking. It presupposes assent to rigorous standards of excellence and mindful command of their use. It entails effective communication and problem solving abilities and a commitment to overcome our native egocentrism and sociocentrism.

From Aaron Dewald:

https://www.youtube.com/watch?v=fRmhB3MW6GE
Not criticism or judgment -- critical is "a way of thinking", not a result.
Quotes Moore and Parker: "The careful application of reason in the determination of whether a claim is true."
Quotes Tittle: "Judicious reasoning about what is true, and therefore, what to do."

• Careful, intentional thinking ("judiciousness"). Purposeful, deliberate, thorough. Fully evaluate ALL parts of a claim. Take in complexity, breadth, depth. Goal is not only voting 'yes or no'.
• The use of reason or logic.
• Judgement about beliefs.
• Application to real-world problems

Bloom's Taxonomy:
Remembering --> Understanding --> Applying --> Analyzing --> Evaluating --> Creating

Ennis, as quoted on http://criticalthinking.net
A SUPER-STREAMLINED CONCEPTION OF CRITICAL THINKING
Developed (last revised 11/26/10) by Robert H. Ennis, rhennis@illinois.edu.

A critical thinker:
1. Is open-minded and mindful of alternatives
2. Desires to be, and is, well-informed
3. Judges well the credibility of sources
4. Identifies reasons, assumptions, and conclusions
5. Asks appropriate clarifying questions
6. Judges well the quality of an argument, including its reasons, assumptions, evidence, and their degree of support for the conclusion
7. Can well develop and defend a reasonable position regarding a belief or an action, doing justice to challenges
8. Formulates plausible hypotheses
9. Plans and conducts experiments well
10. Defines terms in a way appropriate for the context
11. Draws conclusions when warranted – but with caution
12. Integrates all of the above aspects of critical thinking
Although the word 'critical' is sometimes used in a negative sense, this conception of critical thinking is not negative. Also, it does not treat critical thought as persuasion, but critical thought will, we hope, often be persuasive. The future of democracy depends on it.

Kaylea's definition:
Critical thinking is the careful and skeptical application of reasoning to assess the validity of claims, evidence, and argument, in the service of some significant action, decision, or conclusion.

Metacognition Week 2 - Mind Tools - Time Mgmt.

Quiz on time management: 1 - 5 2 - 2 3 - 5 4 - 4 5 - 2 6 - 5 7 - 4 8 - 5 9 - 5 10 - 3 11 - 3 12 - 5 13 - 4 14 - 5 15 - 2 Total: 54 Goal setting: 17/20 Prioritization: 30/35 Managing Interruptions: 15/20 Procrastination: 10/15 Scheduling: 12/15

Metacognition Week 2 -- side notes and thoughts

All this information that we know how to measure is so exciting. So why don't we teach it to kids? "Study skills" sounds so dull, and "hack your brain" is something pitched at 20-somethings, while we might target older people with "revitalization". Why don't I know more about how I learn, about the techniques that help me the most, about where I can work to improve -- why isn't this part of standard curriculum, teaching people to learn how their minds work, at a level appropriate to where they are in maturity, so that they can get the most out of their school experiences? A thread of metacognitive knowledge should be woven through our school curriculum!

Metacognition - Week 1 - Self-Reflection

This week, we studied the basic definitions of metacognition and its key components -- metacognitive control, metacognitive monitoring, and metacognitive knowledge. We also walked through the history of metacognition, from early days of introspectionism, to the challenge posed by behaviorism, to the rise of metacognitive theories.

One observation I have about this week's materials is how central a role language plays in the evolution of metacognitive theories, at least in this subject area. In some ways, it stands to reason that if you can't talk about something, it's difficult to make progress on it. But metacognitive psychology in particular seems closely tied to the existence of models -- and, thanks to the behaviorists -- key experimental techniques tied to these models. By contrast, biology and physical sciences seem to make progress devoting substantial portions of their effort to observation, and description, as well as the development of language and theory alongside these techniques. Modern neurobiology seems to offer a great deal to metacognition, hence cognitive neuroscience.

The history of metacognition was full of surprises and new information for me -- the rather 'fuzzy' initial experimental approaches taken by true introspectionists, the shock treatment administered by the behaviorists, the careful reclamation of a theory of consciousness and reasoning, or at least a theory of something beyond stimulus. It occurs to me that introspectionists and behaviorists have something in common -- they all seemed to model the mind with a focus on a two-step process: memory -> recall; thought -> action; stimulus -> response. Whether the mind was sufficiently divisible to observe these processes or not, they seem to have very few steps or variables involved. Neither school seemed to specifically address or model what we call reasoning. More modern theories of mind seem to recognize that human thinking is more like a stew than a broth -- composed of a range of impulses, analytical skills, knowledge, memories, and so on.

One way that we learn about our minds is through our children -- our own, or through larger experiments involving children. I enjoyed the experiments that revealed that people don't always know when they have something memorized, or they choose the thing on the right even though it's identical to the others. We watch them grow, and learn about them,, human development, our own selves at the same time. My older daughter at the age of 5 came home from school with a metacognitive theory of her own -- she talks about her "true-teller", which seems to be a more focused and thinking part of her brain. Sometimes if we ask how she knows something, or why she thinks something, she'll tell us that her true-teller told her. My younger daughter at 4 taught me a significant lesson about how language works, when I asked her "Why did you let your sister cut your hair?" and she told me, "Mom, she letted her own self!". I rejoiced in the milestone she achieved when, after hearing me describe the wonders of the rainforest, she asked 'How do we know that?'. The world of children is a continual source of amazement.

I found it a little bit funny that something so basic -- the 'tip of the tongue' experience -- offered so much fodder to scientists over the centuries. I suppose the universal nature of the experience is helpful here. Another surely-universal experience that might be explored is when we only understand something retrospectively, or when we change our minds. Retrospective understanding might be defined as the piecing together of clues and information that only make sense now that we have some distance from them. Changing one's mind also seems like a rich source of theories and analysis -- we know it is difficult, and that new information is not enough. I wonder what the behaviorists had to say about the data here. Something I still have questions about is how these ideas fit together with other parts of psychology and neurobiology -- the parts of mind in Freud (which seem useful even if many of his specific assessments are discounted now) and other psychology frameworks.

Ch 2 Concept Review

Introspectionism for Wundt versus modern introspectionism (Hart and others)

Classic introspection techniques took the approach that what people say about their minds is the best and most reliable source of information. Individuals were asked to give their responses to certain words, or to describe the workings of their minds, and their reports were catalogued. This approach had many gaps and flaws -- without any independently verifiable experimental methods, the results were eventually discarded as too subjective and lacking in evidence. The behaviorist attack was largely effective in sweeping away cobwebby thinking and pushing the psychological sciences to work harder to validate their approaches. Joseph Hart developed a series of experiments that allowed these introspective phenomena to be measured independently -- by demonstrating that in some cases, people could accurately report on the workings of their own minds, in ways that were then provable. The example described in the text is his RJR model -- which utilizes the much-studied 'Tip of the Tongue' phenomenon to demonstrate that even when people do not know the answer to a question, they can judge with reasonable accuracy whether they would recognize the answer if they saw it. These reports are verifiable by how accurate their answers are, when coupled with how well they predict their own recognition. Hart found that people tend to whether they will be able recognize the correct answer to a question; they know if they "know" it, even if they can't recall it. This suggests that there is more than memory or response to patterns/stimuli going on in the brain.

Imageless thoughts

The classic introspectionism experiment is image-association, in which participants were asked to say what image they saw before a word came into their heads. Experimentalists found that a significant portion of these words did not generate any images in their various study participants -- which suggested that there were portions of the mind's workings that could not be reported verbally.

In the Ericsson and Simon model of introspectionism and memory, the verbal reports from a subject are most accurate when the person has been asked to monitor their thinking in a particular way and are immediately asked for the results of this monitoring. Lower-accuracy data can be obtained by asking people about their thinking at a later point. If the individual did not pay attention to a particular element of their thinking, they may not be able to do much more than guess. In the situation described where items are preferentially selected based only on their position, people may simply not know why they made a particular choice, and potentially fabricate a reason. The choice might be made based on some instinctual or subconscious level, such as proximity to the hand, or location in the area of greatest focal clarity for the human eye.

Discussion Question

Studying for an exam on the history of metacognition -- some multiple-choice, some essay questions. How to study?

I would use the following approaches to studying for the exam:

1. Read the chapter multiple times [COGNITIVE]
2. Be able to re-tell the story in my own words (find someone to listen to me talk) [METACOGNITIVE]
3. Memorize each scientist and their contribution [COGNITIVE] with flashcards [METACOGNITIVE]
4. Write out a timeline with names and dates; years are hard for me but writing may help [METACOGNITIVE]
5. Think about a few potential essay topics -- compare and contrast, agree and disagree [COGNITIVE], rehearse potential answers [METACOGNITIVE]

A TOTE analysis would assert that approach #2 enacts the Test of whether the goal for #1 has been reached. Feedback from my listener (or the act of writing out a re-telling and then reading back over it later) would act as the actual test/exit condition. Approach 3ii is the test on approach 3i. Approach 4 is also a test on approach 3i. Approach #4 circles back to approach #1 as a further mechanism to assess whether the above topics have been learned.

Chapter 2 Notes

History of Metacognition

The tale of Simonides the poet, as told by Cicero -- said to have invented the 'method of loci', which instructs you to connect the things you want to memorize with specific locations.

Middle Ages -- the Abbey Memory System, a medieval version of the same approach. See visual of abbey locations.

Comte's Paradox
French philosopher Auguste Comte, argued that one cannot observe the workings of one's own mind, because a single mind cannot be divided into two parts. "The observing and observed organ being identical, how could observation take place?"
Trained Introspection
Attempted to teach people to observe their conscious actions. Advocated by Wilhelm Wundt, and later his student E.B. Titchener. Simply ignored the notion of paradox by positing that the "observing organ" could indeed be divided. Modern neuroscience seems to validate this notion of divisibility, because self-reflection and memory retrieval are located in different systems of the brain. Retrospective Introspection
Franz Brentano argued that concurrent observation can't occur -- intense emotions overwhelm, and introspecting can be misleading because observing inner processes may change them. Retrospective introspection is observing a mental process by recalling the events stored in memory that arose from that process.

	Trained Introspection	Retrospective Introspection
Definition	Observe mind while the process is occurring	Go back and think about the process
Comte?	Disputes tenets; says mind can divide	Makes irrelevant; uses memory rather than simultenaeity

Flaws in Introspectionism

• People tried to use it to understand the structure and function of the mind
  • Believed to produce a "picture of the mind"
  • called infallible, Brentano asserts "does not admit of doubt"
• Does not address processes that do not produce mental images or sensations

Oswald Kulpe

• Student of Wundt
• focused on introspective methods
• tried to resolve higher-order mental processes
• kept running into "imageless thoughts"
• ended up categorizing the misfit data instead of questioning the underlying method

Enter Behaviorism

• Defined by John B Watson
  • did animal experiments and taught introspectionism
  • Came to believe that psychology should focus more on behavior than with the mind
  • disdain for notion of consciousness
  • emphasis on what could be observed, proved, measured, etc.
• popularity grows through 1920s, becomes dominant
• shift toward metacognitive begins in 1960s
• produced a wealth of data and theory about how humans/animals behave

Resurgence of Cognitive Approaches

Why shift away from behaviorism?

• did not fully explain animal behavior
  • more than "stimulus - response" was observed.
  • Example: monkeys "expecting" a banana may reject something they want (lettuce).
  • Expectation is an unobservable phenomenon, which behaviorism did not accept.
  • E C Tolman explained that behavior is influenced by motivational factors - drive, incentive
  • B.F. Skinner VS Noam Chomsky
    • Skinner had simple behaviorist model for many significant human behaviors. Then he touched language and Chomsky pounced! Mine, cried Chomsky! All mine!!
    • Skinner argued that language was a collection of stimulus/response reflexes
    • Chomsky's counter:
      • reading and understanding an unfamiliar sentence is more than associating/reflex responses to single words
      • psycholinguists begin to postulate mental rules and syntax as the basis for language acquisition, generation, comprehension
• emerging theories about mental processes
• not enough to have 'unanswered questions' -- must have an alternative means to explore those questions
  • computer science -- the computer as a model for the mind.
    • Rules for symbol manipulation
    • control and supervision
    • provides cognitive psychologists with a language for discussing the mind language an interesting key here; need a model and a language for something if we are going to talk about it

three Alternatives to Behaviorism

The TOTE Loop

Test
Operate
Test
Exit
Miller, Galanter, and Pribram -- describe mind in terms of relationships and plans. Imagine future, make plans, use of working memory. (AKA "Miller et al.")
"A feedback loop in which the outcome of a test informs: whether the state being tested for is present ('congruity') -- if so, exit while (congruity() != true) { doStuff() } congruity() { // assess whether desired state is achieved } doStuff () { // whatever activity we think will lead to congruity. Use memory, reasoning, etc. }

Human Memory

Atkinson and Shiffrin -- human memory is composed of a series of stores.

• sensory store
• short-term store (limited capacity)
  • once in short-term store, can be operated on by control processes
  • can rehearse it, elaborate on it
• presumably long-term store comes next but chapter does not go past sensory -> short term

Problem Solving

• Newell and Simon -- "a collection of information processes that combine a series of means to attain an end"
• means include:
  • choosing a goal
  • select a method to generate a solution
  • evaluate the results of that method
• recursive in nature -- may choose a new method, or even a new goal

Initial response in the scientific community is to explore these components -- the short-term store, describing how people represent problems as they solve it -- metacognitive largely ignored (how processes are monitored & controlled). Shellshock from the behaviorists (lack of solid methods to study these processes).

Return of Introspection/Rise of Metacognitive School
Lieberman in 1979 argues "for a limited return to introspection in the analysis of human thought and action." Introspection has limits but has value; some examples that this is true -- those who report rehearsing something remember it better, those who use the loci technique do better than those who don't rehearse.
Nisbett and Wilson in 1977 argue that introspections are invalid; in those occasions where people do report their processes accurately, it's due to educated guesses/inferences. Example: people tend to choose an object on their right.
Ericsson and Simon in 1980 try to reconcile: a theory of introspective reports that explains when they will be valid, and when they will not. They refer back to Atkinson & Shiffrin's 'stages' model of memory. If the information is in the short-term store, introspective will be valid (ex: asked to keep track of it as you go). If not currently in short-term store, may be valid, or may not have access to the information (ex: not asked to track, may have forgotten later). If the info never resided in short-term memory, then largely invalid (ex: what part of your cortex lights up when I....). Might make a correct guess.
Conclusion: people's introspective reports can be valid under well-specified conditions.

Jose T Hart - TOT phenomenon. Studied by testing people on trivia, then on whether they say they'd know the answer if they saw it. He calls this the 'Recall-Judge-Recognition' model (RJR).

• Step 1 - Recall. Try to pull answer from memory
• Step 2 - Judge. Can you recognize the correct answer?
• Step 3 - Recognize. Do you see the answer listed here?

The Metacognitive School
John Flavell -- coined the term 'metamemory'. Many metacognitive analyses. Example: ask children to study until they say they're ready to be tested -- they may say they're ready, when they're not. Significant paper defining metacognition in 1979.
Ellen Markman did a similar experiment with elementary school children and incomplete instructions
Ann Brown: "distinction between knowledge and the understanding of that knowledge is a valid and important distinction with great heuristic power"

Flavell further subdivides metacognitive knowledge

about how oneself processes information

about a specific category of task "Math is hard"

what strategies are effective

Kaylea's Musings

"reasoning" vs "memory" -- introspective methods seem to miss this distinction Parts of our past can only be understood later; memory alone not enough Stella's idea of her 'true-teller' Wonder how these ideas fit together with the parts of mind in Freud and other psychology frameworks cycle of theories, gaps, generation of language to discuss/describe

Ch 1 Concept Review

What is metacognition? Metacognition is 'cognitions about cognitions' -- knowledge, understanding, learning, or just plain thinking about what it is that one knows, understands, is learning, is thinking about, etc. Metacognitive knowledge is a specific category of metacognition focused on what we know about knowing -- the facts or understandings we have about metacognition, as well as our awareness of our own understanding. Metacognitive monitoring is the process of feeding information from the acting cognitive brain into the metacognitive stores -- it's how we tell ourselves whether we know something, or are making progress in figuring something out. An example of monitoring is the feeling of "just not getting this" -- when we realize that what we're doing to learn something is not effective. Metacognitive control is the process of changing how we are approaching a topic in order to think about it more effectively -- by control of our environment, by putting other thoughts out of our minds, by ordering ourselves to focus.

Ch 1 Discussion Questions - Metacognition Text 1 - The 'cell phone' analogy. The listener has control over the input because they can mute the volume, put the call on hold, turn on speakerphone, or make it louder. The speaker might be able to hear some of these alterations, but they might not. The listener can always interrupt, walk around, view distractions, make noise, or fail to murmur during pauses such that the speaker does not feel listened to. The listener could also go somewhere that has poor reception or is so noisy that the conversation is interrupted. This metaphor is incomplete because by creating a conversation between two distinct people, we introduce a separate and independent entity with his or her own priorities and contexts. The conversation is made external, and subject to social conventions and the limitations of spoken language. The use of technology as an intermediate layer is problematic because it has more delays and everyday opportunities for failure than the "internal conversation" that metacognition describes. However, a cell phone conversation does have some points of comparison with metacognitive monitoring, just as the text describes. A phone listener is listening both to understand and to communicate that understanding to the person speaking. While talking to a friend, you might also be trying very hard to think about what is being said and what you should say in reply, as well as trying to think a few steps ahead about what your friend might say in response. Meanwhile you want to make sure that you give all possible signs to your friend that you are listening to her, and if you begin to get off-track or say something you don't intend in response, your close monitoring of the conversation might allow you to correct before you offend or confuse her. One even more modern example worth considering is text messaging. When texting with someone, there is the content of the message, the timing of the reply, and whether the person spent much time composing it. There's also the possibility of autocorrect interfering in what you're trying to say but possibly communicating more about the words you use frequently. Some examples: 2. Some examples of cognitive processes that I am unable to monitor might be one common resolution to the 'tip of the tongue' state -- when the thing you couldn't remember for the life of you suddenly pops into your head while you are doing something else. Another cognitive process that I would find difficult to monitor is a 'lightbulb moment' or sudden flash of insight. It is probably also difficult to monitor the progression of certain kinds of knowledge attained very gradually or when one is very young -- such as asking a native speaker how they learned English, or asking someone how they learned to speak, read, count, assign colors, and so on. They might be able to recall specific events in the learning process, but the multi-year learning process would not lend itself to monitoring. I think that monitoring must specifically require an awareness of the before-state (a "don't know" feeling) and the after-state (a "got it now" feeling), as well as the ability to perceive the process of moving between them.

Metacognition Chapter 1 D&M Notes

Notes from text (quotes) Ch 1.

Metacognition	"refers to thoughts about one's own thoughts and cognitions"	Ex: a 'tip of the tongue' event Deciding to write down a grocery list or directions making decisions about how to study because of what you know about how your brain works
Metacognitive Knowledge	"declarative knowledge about cognition -- facts, beliefs about cognition that you can state verbally". How learning operates how to improve learning	Ex: Cramming doesn't work but it's better than nothing Noise and distractions are bad for studying
Metacognitive Monitoring	"assessing or evaluating the ongoing progress or the current state of a cognitive activity"	Ex: judging whether your are approaching the correct solution assessing how well you understand what you are reading
Metacognitive Control	"regulating some aspect of a cognitive activity including stopping it, deciding to continue it, or changing it midstream"	Ex: deciding to use a new tactic to solve a difficult problem deciding to spend more time trying to remember the answer to a trivia problem

Relationship between Metacognition and Cognition

Meta-level and Object-level
Object level is the actual processes of interest -- attention, learning, language processing, problem solving
Meta level has a model of the understanding, informed by monitoring of progress and metacognitive knowledge overall.
When meta modifies object: Control
When object modifies meta: Monitoring