As I have posted before, the research results from John Hattie (and team) multi-year, multi-study, and multi-student (53,000,000 – yes, 53 Million students) meta-analytic research studies yield our generation’s most profound, prolific, and pronounced guidance as to WHAT WORKS in teaching and learning. My aim in sharing is to keep the conversation alive about how to best learn from research studies and how to best replicate that which works in an effort to improve the learning for all of our children!! Research should always be viewed with two very critical eyes. Studies on this or that can come and go … BUT – Hattie’s research studies, his books, his findings are HUGE. I believe that Hattie is our generation’s Benjamin Bloom (see Bloom’s Taxonomy).
Another topic about which I post is Twitter and its use as a PLN (Professional Learning Network). On Twitter (the world’s largest professional learning network), there are a number of educational leaders, educators, and bloggers who gather, analyze, share, question, etc. “in time” guidance on how to best impact learning. Of course Twitter has other purposes, but my purpose in using Twitter is for news and for educational leadership information and for increasing the leadership ability of those I serve and for increasing the learning of those I serve.
Today I came across an impressive report/synthesis of Hattie’s Research findings from http://www.teachthought.com, Teach Thought’s website has this blurb about their director:
Terry Heick is a former English teacher turned education dreamer who is interested in how learning is changing in a digital and connected world. This includes, among other changes, the rise of self-directed learning.
He is also interested in the power of questions, the role of play in learning, clarifying digital literacy, the flexibility of project-based learning, marrying mobile learning and place-based education (especially through mentoring), the potential of video games and simulations in learning, what it really means to “understand” something, and how all of this produces wisdom and self-knowledge in students.
In addition to his work with TeachThought, he also presents at national conferences, provides professional development for schools, blogs for edutopia, and creates content for companies including learn.ist.
With their permission, I am reprinting their blog post in its entirety:
Hattie’s Index Of Teaching & Learning Strategies: 39 Effect Sizes In Ascending Order
Effect Size Defined
Statistically speaking, the strength of the relationship between two variables. John Hattie, Professor of Education and Director of the Melbourne Education Research Institute at the University of Melbourne, Australia, says ‘effect sizes’ are the best way of answering the question ‘what has the greatest influence on student learning?’
Effect Size Applied
- Reverse effects are self-explanatory, and below 0.0
- Developmental effects are 0.0 to 0.15, and the improvement a child may be expected to show in a year simply through growing up, without any schooling. (These levels are determined with reference to countries with little or no schooling.)
- Teacher effects “Teachers typically can attain d=0.20 to d=0.40 growth per year—and this can be considered average”…but subject to a lot of variation.
- Desired effects are those above d=0.30 (Wiliam, Lee, Harrison, and Black 2004) and d=0.40 (Hattie, 1999) which are attributable to the specific interventions or methods being researched– changes beyond natural maturation or chance.
- Blatantly obvious effects: An effect-size of d=1.0 indicates an increase of one standard deviation… A one standard deviation increase is typically associated with advancing children’s achievement by two to three years*, improving the rate of learning by 50%, or a correlation between some variable (e.g., amount of homework) and achievement of approximately r=0.50. When implementing a new program, an effect-size of 1.0 would mean that, on average, students receiving that treatment would exceed 84% of students not receiving that treatment. Cohen (1988) argued that an effect size of d=1.0 should be regarded as a large, blatantly obvious, and grossly perceptible difference [such as] the difference between a person at 5’3″ (160 cm) and 6’0″ (183 cm)—which would be a difference visible to the naked eye.
Effect Size CAUTION
Reduce temptation to oversimplify. This is one more resource in our efforts to problem-solve on behalf of our students. We need to be careful about drawing too definite a conclusion from an effect size without examining the study. For example, homework is shown to have an overall effect size of 0.29, which is low and well below the average of 0.40. But when you look more closely, you find that primary students gain least from homework (d = 0.15) while secondary students have greater gains (d = 0.64).
Editor’s Note
Data is only as useful as its application. As hinted at above, don’t fall into the trap of assuming the teaching and learning strategies and other impacts on student achievement at the top of the list are “bad,” and those at the bottom are “good.” These are not recommendations, but rather a comprehensive synthesis of a huge amount of data. Every study has a story, and every strategy and impacting agent below has a background.
The most helpful part of this chart–and the reason we asked Dana to share her work here–was the column on the right where she adds a short statement or tidbit that helped contextualize the data point. Otherwise, judging purely by the chart, inquiry-based learning. self-directed learning, class size, and teacher content knowledge perform terribly, while skipping a year, reciprocal teaching, and teaching of study skills are through the roof.
Ultimately, to best use this data to inform teaching and planning, every study we need to be looked at on its own. We would need to clarify what the terms were for success. We’d also need to plainly clarify the definition for every word and phrase for every impacting agent and strategy so that we were all speaking the same language. We would then need to identify and analyze other variables in each study–inquiry with or without technology, with or without access to local communities, with students reading at, below, and above grade level, using culturally relevant or irrelevant text, and so on.
Which makes two of his books–Visible Learning and the Science of How We Learn, and Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement–must-buys so that you can do that kind of analysis on your own rather than skimming a blog post and extracting misguided takeaways (which is why we hesitated to publish it to begin with). That said, the results of the synthesizing of the data appear below.
Also, we used a new embedding host for the document. If you have trouble viewing or scrolling, let us know in comments, via email, or on twitter!
What Has The Greatest Influence On Learning? A Synthesis Of Hattie’s Synthesis
| Retention (holding back a year) |
-0.13 |
Repeating a grade. Also negatively correlated with social/emotional adjustment, behavior, and self-concept. |
| Open vs traditional learning spaces |
0.01 |
Open classrooms range widely in features—not correlated to increases in achievement. |
| Student control over learning |
0.04 |
Effect of student choice and control over learning is somewhat higher on motivation outcomes than achievement outcomes, but neither have major consequences on learning and too many choices can be overwhelming. |
| Teacher subject matter knowledge |
0.09 |
Little data to support claim that teacher content knowledge is critical to student achievement. Darling-Hammond claims content knowledge influential up to some level of basic competence but less so thereafter. Since publishing Visible Learning, Hattie has studied this topic more in depth and has shared that the issue is a pedagogical issue—teaching is occurring at a surface level such that deep content knowledge has not presented itself as influential or not. Expert teachers know how to connect their content to other relevant issues and content and how to organize that content. |
| Ability grouping/tracking/streaming |
0.12 |
Refers to whether classes are heterogeneous or homogeneous. Studies consider achievement effects and equity effects. More than 300 studies show tracking has minimal effects on learning outcomes and “profound negative equity effects.”Separate gifted programming is not considered in this set of studies—see ability grouping for gifted. |
| Gender (male compared with female achievement) |
0.12 |
Males and females are more alike than they are different, and differences are minor. 2,926 studies all point to the same conclusion. “The differences between males and females should not be of major concern to educators.” |
| Matching teaching with student learning styles |
0.17 |
Contends different students have differing preferences for particular ways of learning—auditory, visual, tactile, or kinesthetic, for example. No gains in achievement found when teacher matched instruction to preferred modality. Much skepticism surrounding claims around learning preferences. Research does not support correlation between matching learning style and increased achievement. |
| Within-class grouping |
0.18 |
Defined as “teacher’s practice of forming groups of students of similar ability within an individual class.” Such groups that were studied were formed on semi-permanent basis. This is different from grouping for purpose of targeting instruction toward a specific skill area in which a heterogeneous group (achievement-wise) needs support for a short amount of time/intervention. Effects on research re: within class grouping (excluding gifted) was higher when class size was above 35—i.e. students in class sizes over 35 benefitted from small group instruction. Different from small group learning, defined as teacher assigning a task to small group and expecting them to complete. |
| Extra-Curricular |
0.19 |
Not a high correlation between extra-curriculars and achievement—sports is even lower than academic-related activities like speech/drama/music; however, because students enjoy activities, they are engaged and keep attending schools where they “gain the dividend of instruction in more academic subjects.” Effects from activities were found to be more related to identity formation and peer self-esteem, which are especially important to adolescents. |
| Reducing class size
(Reduce from 25-15, effect between 0.10 to 0.20) |
0.21 |
Effects may be higher for working conditions which may or may not translate into effects on learning. For smaller class size to yield higher effects, the type of instruction needs to be re-conceptualized to ensure the needs of all students are met within whatever the class size. Need to focus on strategies that are maximized in smaller or larger groups and apply respectively. |
| Individualized instruction
(Note: NOT special education) |
0.22 |
Based on ideas that each student has unique interests and past learning experiences, and individualized program takes this into account. Allows for student flexibility and individual differences. Small effect, but one study claimed higher effects based upon teacher adapting instruction to needs of students and aligning to capability in addition to finding resources that were fitting. Other whole class/group influences like peer tutoring have higher effects. |
| School finance |
0.23 |
Minimal relationship between educational expenditure and student achievement; more positive correlation between expenses for cost of instruction (e.g. teacher salaries and instructional supplies) and achievement. Not amount of money spent, but how it is spent. |
| Teaching test-taking and coaching |
0.27 |
Many studies around SAT preparation show influence impacted by length of coaching/training. Other studies indicate that familiarizing students with the examination process and examiner can make a difference, more so than test prep. Students in the low SES group performed significantly higher on standardized tests when they were familiar with the examiner. |
| Homework
(Note: Elementary effect size of 0.15, and high school of 0.64) |
0.29 |
Involves “tasks assigned to students by teachers that are meant to be carried out during non-school hours.” Effects twice as large for high as for junior high, and twice as large again for junior high as for elementary. Smallest effects in math. Largest in science and social studies with English in the middle. Effects greater for higher than lower ability students. Homework for some reinforces that they cannot learn by themselves. Can undermine motivation and internalize incorrect routines and strategies. |
| Inquiry-based teaching
(Note: Hattie wondered why effect wasn’t higher and since publishing, has learned that teaching content so students have some background knowledge about which they are inquiring increases effect) |
0.31 |
Art of developing challenging situations—students observe and question phenomena, pose explanations, devise and conduct experiments, collect data, analyze data, draw conclusions, design and build models, or any combination. Open-ended. Greater effects when teaching process rather than content. Shown to produce transferable critical thinking skills. |
| Using simulations and gaming |
0.33 |
Typically involves use of model or game (such as role playing, decision-making) with an aim to engage students in learning. Aims to mimic real-world problems. |
| Decreasing disruptive behavior |
0.34 |
Teachers need skills to ensure no student disrupts his/her own learning or that of others. Argument is NOT that disruptive student should be removed. |
| Computer-assisted instruction
(Note: Web-based learning, interactive video methods, and simulations are analyzed separately) |
0.37 |
25 times out of 100, computer-aided instruction in the form of tutoring, managing, simulation, enrichment, programming, and/or problem-solving will make a positive difference. Majority of studies are about teachers using computers in instruction compared to those who don’t—fewer about students using them in learning in different ways. Use of computers more effective when a diversity of teaching strategies, when teachers receive pre-training in their use, when multiple opportunities for learning, when the student (not teacher) is in control of learning, when peer learning is optimized, and when feedback is optimized. |
| Integrated curricular programs (e.g. global studies class that incorporates both science and social studies or thematic unit– Friendship) |
0.39 |
More effective in elementary and middle school than high school. Greater effect when instruction was organized around a theme (0.46) and process skills were emphasized (0.36). Greater effect for lower achieving compared to middle and higher achieving students and when more experienced teachers implemented. |
|
Effect Size greater than 0.4 effects student achievement |
||
| How to develop high expectations for each teacher
(Note: Hattie contends teachers must stop over-emphasizing ability and start emphasizing progress—steep learning curves are the RIGHT of ALL students regardless of where they start. Be prepared to be surprised!) |
0.43 |
Studies included effects related to the notion of self-fulfilling prophecy—teachers are more likely to have their students reach their expected outcomes regardless of the “veracity” of the outcomes. Studies in this meta-analysis also show students know they are treated differentially in the classroom due to expectations by teachers for certain students to take AP courses, for example, or others to pursue technical fields. |
| Professional development on student achievement |
0.51 |
Research re: PD seems to focus more on changes in teachers rather than impact on student outcomes. PD likely to change teacher learning but has less effect on teacher behavior. PD in science has highest effects on student outcomes (0.94) then writing (0.88). Seven themes re: what works best in PD were advocated as a result of 72 studies. |
| Home environment |
0.52 |
Includes measures of the socio-psychological environment and intellectual stimulation in the home. Most highly correlated factors with achievement were maternal involvement, variety and play materials. |
| Peer influences on achievement |
0.53 |
Studies include a variety of influences: peer tutoring, helping, friendship, and giving feedback. Studies examining what happens when a student moves schools show single greatest predictor of subsequent success is whether student makes friend in first month. |
| Phonics instruction |
0.54 |
Teaching students the alphabetic code. Designed for beginners in early elementary. |
| Providing worked examples |
0.57 |
Typically consist of a problem statement and the appropriate steps to a solution. Three steps: introductory phase, acquisition/training phase, test phase (assess learning). Reduces cognitive load for students such that they concentrate on the processes that lead to the correct answer and not just providing an answer. |
| Cooperative vs individualistic learning |
0.59 |
Most powerful when students have acquired sufficient background knowledge to be involved in discussion and learning w/peers. Most useful when learning concepts, verbal problem-solving, spatial problem-solving, retention and memory. Effects increase with age. |
| Direct instruction |
0.59 |
Not to be confused with didactic teacher-led talking from the front. Refers to 7 major steps:
Speaks to power of stating learning intentions/outcomes and communicating standards for performance and then engaging students in getting there. Effects were found to be similar for regular education and special education—i.e. direct instruction is effective for all. |
| Concept mapping |
0.60 |
Involves development of graphical representations of the conceptual structure of content to be learned. Importance of concept mapping is in its emphasis on summarizing main ideas in what is to be learned. Assists in synthesizing and identifying major ideas, themes, and interrelationships. |
| Comprehension programs
(Interesting note: Hattie did not find a 4th grade reading slump, just no growth or increase during upper elementary years. Several possible reasons for plateau: most curricula does not attend to reading progressions, lack of building upon learning to read once students have learned to read, and possibly perceived “unimportant” reading difficulties appear for the first time in Grade 5 when students encounter information materials and multiple text types requiring more inference and comprehension. |
0.60 |
Comprehension programs with dominant focus on processing strategies (e.g. inferential reasoning, rules for summarizing, and chunking texts) produced higher effect than did text programs (e.g. repetition of concepts and explicitness) and task programs. |
| Teaching learning strategies |
0.62 |
Teaching kids how to learn and developing students’ strategies for learning. Need to provide students with learning strategies in the context of learning, a chance to practice, and assurance that the strategies are effective. Need to understand intention to use, consistency in appropriate use ,and knowing when chosen strategy is effective—learning to learn or self-regulation. |
| Teaching study skills |
0.63 |
To get to deeper levels of understanding and effectiveness, combine study skills instruction with the content. |
| Vocabulary programs |
0.67 |
Students who experienced vocabulary instruction experienced major improvements in reading comprehension and overall reading skills. Most effective vocabulary instruction included providing both definitional and contextual information, involved students in deeper processing, and gave students more than 1 or 2 exposures to the word to be learned. |
| How to accelerate learning (e.g. skipping a year) |
0.68 |
Other forms of acceleration include compacting curriculum, telescoping curriculum, and advanced placement. No negative social effects for accelerated students were supported by the research. Effect size for 2 meta-analyses and 37 studies regarding all forms of acceleration was 0.88. |
| How to better teach meta-cognitive strategies |
0.69 |
Meta-cognitive strategies refer to those “thinking about thinking” strategies: planning how to approach a learning task, evaluating progress, and monitoring comprehension. Self-questioning is another meta-cognitive strategy. |
| Teacher-student relationships |
0.72 |
Interestingly, “when students, parents, teachers and principals were asked about what influences student achievement, all BUT the teachers emphasized the relationships between the teachers and the students.” “Building relationships implies agency, efficacy, respect by the teacher for what the student brings to the class (from home, culture, and peers) and recognition of the life of the student.” |
| Reciprocal teaching |
0.74 |
Teaching cognitive strategies intended to lead to improved learning outcomes. Emphasis on teachers enabling students to learn and use strategies such as summarizing, questioning, clarifying, and predicting. Dialogue between teacher and students around text. Students take turns as teacher and lead dialogue to bring meaning to written word with assistance to learn to monitor their own learning and thinking. |
| How to provide better feedback |
0.75 |
Among most powerful of influences, especially when it is from the student to the teacher. If the teacher is open to feedback regarding what students know and understand, where they make errors, when they have misconceptions, and when they are disengaged, then they can respond accordingly. Feedback is about providing information about the task performance. Effect sizes from these studies show considerable variability, meaning some forms of feedback are more powerful than others. Least effective: programmed instruction, praise, punishment, and extrinsic rewards. Feedback is more effective when it provides information on correct rather than incorrect responses and when it builds on changes from previous trials. |
| Providing formative evaluation to teachers |
0.90 |
Refers to teachers attending to what is happening for each student in their classrooms as a result of their instruction—when teachers ask, “How am I doing?” Highest effects when teachers seek evidence on where students are not doing well. |
| Teacher credibility in the eyes of the students
(Note: This link is to an interesting article on credibility and how to build it: http://bit.ly/WRZ5iA) |
0.90 |
“If a teacher is not perceived as credible, the students just turn off. If a student doesn’t get (the value of education) by the age of 8, they are behind for most of the rest of their school life. Students are very perceptive about knowing which teachers can make a difference to their learning. And teachers who command this credibility are most likely to make the difference.” |
| How to develop high expectations for each student |
1.44 |
Refers to students’ expectations for and beliefs in themselves. Involves students predicting or self-reporting their grades. Implications: teachers need to provide opportunities for students to be involved in predicting their performance. “Making the learning intentions and success criteria transparent, having high, but appropriate, expectations, and providing feedback at the appropriate levels is critical to building confidence in taking on challenging tasks.” |
(See above for effect sizes and context/explanation.)
- Retention (holding back a year)
- Open vs traditional learning spaces
- Student control over learning
- Teacher subject matter knowledge
- Ability grouping/tracking/streaming
- Gender (male compared with female achievement)
- Matching teaching with student learning styles
- Within-class grouping
- Extra-Curricular
- Reducing class size
- Individualized instruction
- School finance
- Teaching test-taking and coaching
- Homework
- Inquiry-based teaching
- Using simulations and gaming
- Decreasing disruptive behavior
- Computer-assisted instruction
- Integrated curricular programs
- How to develop high expectations for each teacher
- Professional development on student achievement
- Home environment
- Peer influences on achievement
- Phonics instruction
- Providing worked examples
- Cooperative vs individualistic learning
- Direct instruction
- Concept mapping
- Comprehension programs
- Teaching learning strategies
- Teaching study skills
- Vocabulary programs
- How to accelerate learning (e.g. skipping a year)
- How to better teach meta-cognitive strategies
- Teacher-student relationships
- Reciprocal teaching
- How to provide better feedback
- Providing formative evaluation to teachers
- Teacher credibility in the eyes of the students
