The Schoenblog

How do I know if my teaching really has an impact on learning?

That’s the question that I grappled with for most of the day on Friday. Do my instructional choices lead to a positive impact on learning? Am I building a culture for meaningful learning--as a teacher, as a coach, and as a part of a school system? What does the research say about high impact teaching? Any why don’t we talk about these questions more often?

On Friday, I attended a conference from my local @PNWBOCES exploring student engagement and the 4 Cs: communication, collaboration, critical thinking, and creativity. Educators from all around the region gathered to learn, share, and connect, and while the sessions were valuable, it was the keynote from Peter DeWitt that really stuck with me.

Peter spoke about high impact, research-based, practical strategies to make positive changes in schools and learning. He discussed the value of climate as the 5th C and a “plate where everything else should rest.” I liked that he provided time for the audience to discuss, reflect, and participate, and loved how seamlessly he integrated research into his ideas. In particular, his discussion of the work of John Hattie in Visible Learning spun through my mind all day.

If research shows certain strategies to have a tangible impact on learning, why isn’t that something that we’re talking more about? Teacher training and professional learning often discusses big ideas, anecdotes, and abstract strategies, but doesn’t always address the practical impact nearly enough. I’d guess that most teachers agree that reciprocal teaching is a good strategy, but for me, learning that it has a .74 effect size makes it all the more important.

Hattie’s work ranks 138 influences on learning by studying their effect size based on a study of 1,200 meta-analyses. Basically, he studies the influences that have represent best practices and lead the highest impact on achievement. The average effect size is 0.40, while 1.0 represents the growth of one year; therefore, an effect larger than 0.40 is above average and above 1.0 is a huge growth in learning. Learn more about Hattie and these influences at VisibleLearning.org.

Within our ability.....#studentengagement16 #edchst #NYEDChat pic.twitter.com/1yvlGXbzTq

— Fred Ende (@FredEnde) December 2, 2016

What Role Should Research Have in Teaching?

Teacher estimates of achievement, collective teacher efficacy, and self-reported grades. These three influences have the highest effect sizes of all, according to Hattie (1.62, 1.57, and 1.33, respectively). I want my student to rise to high expectation. I hope that my teachers feel confident in their abilities and teaching. And I love having students self-assess, report, and reflect. But having a foundation in this research to further develop these ideas provides a whole new language to the conversation.

I don’t believe in silver bullets in education; there’s not one influence or idea that will simply fix our problems and help students achieve, but an understanding of these findings has really inspired me to learn more. It’s probably the end-of-semester doctoral student mode, but I’m fascinated by this type of research and a more thorough understanding of how I really influence student achievement.

Recovered my sketchnotes from this morning's keynote from @PeterMDeWitt #studentengagement16 @Occh2 @LearnerAlmOUFSD @knv80 @psmirra37 pic.twitter.com/RGYCtKHpXv

— Adam Schoenbart (@MrSchoenbart) December 2, 2016

The average teacher makes 1,500 decisions a day. Let’s make sure that they're the best ones. Hattie’s work reminds us that so many influences and strategies have a positive impact, but it’s important that educators are making the best choices to produce the desired results for learning.

As a teacher, I want to keep this in mind and plan to share this research with my students. I want to continue to work on building relationships, promoting student agency, and providing effective feedback for growth. As a coach, I want to use to use it as a springboard for discussion with teachers. I bought Hattie’s books this weekend, Visible Learning and Visible Learning for Teachers, and am eager to dive in for more.

How does research influence your teaching, practice, or work with students? Share in the comments or on Twitter @MrSchoenbart.

One of the most important goals from the #GAFE Impact Report was to determine how teachers were using Google Apps for Education in their classrooms and what they needed to use it more regularly and effectively. The more we understand about teachers’ use, perceptions, and needs, the better they can be addressed to increase teacher comfort and impact and affect student learning.

Most teacher and students saw the positive impact of GAFE on student learning, engagement, and the quality of work, but most teachers’ needs were not met. They needed more access, time, and training to really be able to incorporate technology meaningfully in their classrooms. I had hoped for the data from this study to support what I already knew to be true: we were making great steps forward but not doing enough to support professional learning and development in technology.

In this post, I will share an overview of what the teachers shared about their use and needs. While the data probably isn’t surprising, it is important as it validates what many of us already know: we need recursive, timely, and hands-on professional learning for regular and effective technology implementation in our classrooms.

This post is part 4 in my series #GAFE Impact Report, sharing data and analysis about student and teacher perceptions of the impact of Google Apps for Education. The results come from a 2014 survey I designed, implemented, and analyzed in my school. Be sure to check out Part 1: Overview & Infographic, Part 2: Comparing Surveys & Data, and Part 3: Summary of Findings & Analysis. Though limited to my school, I believe these results to be informative and representative of many schools in the early stages of GAFE integration. Please share your thoughts and feedback on these posts, the survey, or the findings in the comments or on Twitter.

GAFE Use & Impact

65% of teachers agree that GAFE has a positive impact on student learning and 35% agree that it positively impact teacher instruction. Interestingly, the disagree statistics were small (9% and 16%, respectively) and the undecided much higher. This made perfect sense as many teachers did not have regular access to technology to training to use it, so they were still undecided about the impact.

However, only 35% of teachers felt prepared to use GAFE effectively in the classroom and 40% of teachers say the have been able to use GAFE more than sometimes this school year. While the impact and positive perceptions are there, we needed to do more to increase training and access.

When asked which GAFE tools they were using, teachers most often responded with Drive, Communities, Classroom, and Sites.

What Teachers Need

Teacher need more access to technology and more training to use it effectively:




In written responses, almost every teacher wrote about the need for more training. According the survey results, teachers want training in the following areas:

• How to use GAFE with individual training for the different apps
• How to teach students to use GAFE and Chromebooks
• How to implement GAFE in the classroom for best practices
• How to implement GAFE for specific subject area classes or student populations
• Concrete modeling and examples of best practices 

Most commonly, teachers requested small group trainings over a long period of time during contractual hours rather than opt-in trainings outside of the school day.

One Year Later

It’s been over a year since the #GAFE Impact Report and a lot has changed since then. More and more teachers have access to technology and we’ve offered multiple cohorts of opt-in training along with differentiated sessions during professional development days as much as possible.

Next time I write about this survey and analysis, I’d like to address how it’s changed my views on professional learning and the ways we’ve evolved since. One day soon, I’d like to update the survey and try it again to see how the data supports the shifts I’ve seen in my school. As a technology coach, I have the privilege of visiting classrooms and experiencing this change but having the data to support it is always valuable.

Now you, dear reader, have access to it all: the Summary Report, Comparative Data, Infographic, Student Survey, and Teacher Survey. I hope it helps you improve professional learning to impact students through technology.

As always, please leave your comments below or reach out on Twitter. I would love to hear what you think about this study, your own ideas about GAFE’s impact, or ways to improve the study for next time.

This post is part 3 in my series #GAFE Impact Report, sharing data and analysis about student and teacher perceptions of the impact of Google Apps for Education. The results come from a 2014 survey I designed, implemented, and analyzed in my school. Be sure to check out Part 1: Overview & Infographic and Part 2: Comparing Surveys & Data. Though limited to my school, I believe these results to be informative and representative of many schools in the early stages of GAFE integration. Please share your thoughts and feedback on these posts, the survey, or the findings in the comments or on Twitter.

Data Summary

I like data. I could sit for hours building formulas, comparing charts, and analyzing the results of any given survey. And this is pretty easy with a Google form. That was exactly the plan last December when I intended to spend time working through the hundreds of responses for the survey. And then, of course, I got the flu. My analysis and conclusions took a lot longer than I would have liked but I’m happy with the results, both in the findings reached and my analysis, organization, and presentation.

For a nice organized Summary Report in a Google Doc, click here. This is what I shared and presented to my colleagues and administrators. In the future, I’ll write about more detailed results but the summary is plenty to get us started.

Below, I’ll highlight some of the key findings with selected graphics and data to back them up. Remember, the link above has much more detail if you are interested. I’m now eager to go through the process again to see how the findings have evolved as our tech integration and GAFE use has become a more regular part of instruction across the building.

• Students are learning from GAFE at OHS but through a range of different classroom experiences. Students report that only 37% of teachers use the tools regularly and 50% never use GAFE. Additionally, teachers are using many of the most effective apps only half as frequently as students.
• The use of GAFE is growing significantly. 58% of students think teachers are using GAFE more this year and 52% think it’s more important to their learning. Only 10% of students disagree with either of these statements.
• Some students and even more teachers are undecided about GAFE’s impacts. Few disagree with the positive benefits but teachers have an especially large undecided population--an average of 41%. Many teachers do not have enough experience or comfort with the technology and its classroom implementation.
• More students recognize the positive impact of GAFE when their teachers have received training to implement it in the classroom. Depending on the level of experience, positive results grew 5-9 percentage points.
• Only 35% of teachers feel prepared to use GAFE effectively in their classrooms. Both teachers and students report the need for teacher training and opportunities to learn to implement GAFE effectively. Almost all of the teachers say they need more training and access to learn how to use GAFE in classroom teaching and to become comfortable teaching with the technology. 

None of these findings surprised me, but they did validate the work I, and many of my colleagues, were doing. GAFE was growing, seemed impactful, and training paid off. Still, it’s essential to recognize our challenges in training and access.

What Students and Teachers Think About GAFE & Learning

GAFE has a positive impact on students’ learning.

GAFE has a positive impact on teachers’ instruction.

GAFE helps improve the quality of student work (in-school).

GAFE helps improve the quality of student work (homework).

Needs Assessment: What Teachers Want

There are too many OHS teachers undecided about the impact GAFE has on teaching and learning. Many simply don’t have enough experience or practice with the technology.

• 77% Agree: I would use GAFE more if I had more training.
• 72% Agree: I would use GAFE more if I had more time.
• 42% Agree: I would use GAFE more if I had more access to the technology.

For now, please share your thoughts on the data, questions, or your school's implementation of GAFE in the comments or on Twitter.

This post is part 2 in my series #GAFE Impact Report, sharing data and analysis about student and teacher perceptions of the impact of Google Apps for Education. The results come from a 2014 survey I designed, implemented, and analyzed in my school. Be sure to check out Part 1: Overview & Infographic. Though limited to my school, I believe these results to be informative and representative of many schools in the early stages of GAFE integration. Please share your thoughts and feedback on these posts, the survey, or the findings in the comments or on Twitter.

Once the inspiration for this study hit, I knew what I needed to do. Ideas, questions, and complications circled through my mind for a few days. I listed my ideas, tried to put them in order, and to figure out what I was really looking for. Who did I want to survey? How? Where? Why? What was I going to do with the data? I had some answers to these questions and more ideas than I knew what to do with. Little did I know it at the time, but a lot of this process was what led me to my current doctoral studies. With the #GAFE Impact Report, I learned a lot of survey design, methodology, and data analysis, which helped focus my interests back towards further studies.

Luckily, I had some friends at work to bounce ideas off of and quickly made some decisions. I would create Google forms for all teachers and students. Students were easy; I could ask my English teacher colleagues to assign the survey, either in class or for homework. I decided to collect usernames in my domain so I could ensure unique responses from each students and let teachers know when their students had all completed it--although their responses remained private. For the teachers, I e-mailed a different Google form with a plea for their help. I explained my goals and purpose and asked that they complete the form over a two-week window. I then followed up in person with some departments, and after a lot of leg work had overall success. Although my administration okayed my work, it was important that I ask for honest feedback and data as a colleague and not as a forced or mandated survey from a supervisor.

The surveys took a while to design and finalize, and I’m pretty happy with the questions I asked, even now. By Thanksgiving or so, I was ready to roll out my baby.

Survey Design

Thankfully, my friend and mentor Tina has her PhD. She had the knowledge, experience, and insight into survey design, methodology, and motivating busy teachers to respond, which I sorely lacked. I didn’t need a doctoral-level project here, but I wanted to make sure that my study was fair, purposeful, and meaningful. She helped me decide on a mixed-methods approach, including both quantitative and quantitative questions.

The quantitative questions provided the hard data and were required. The qualitative ones asked for supporting responses, and some were optional. I wanted to make the survey easy for responders, with the hopes that this would give me more in depth responses. And it mostly worked. There were some topics, like the value of electronic feedback, that were important to me, but ultimately proved to be too much. I wanted to survey to be clear and focused, so I tried to connect everything back to the impact of Google Apps on Teaching and Learning.

Students

The students survey asked about when, how, and why they used GAFE, and about its impact on teaching, learning, engagement, etc. It took most students about 20 minutes to complete.

Find the student survey here.

Teachers

The teacher survey asked almost all of the same questions for the teacher's perspective but had an extra layer. All students at OHS had used Google to some degree, so all of their responses were valuable. But for teachers who had never really used GAFE in their teaching, some of the questions were designed to fail--or at least collect vague or incomplete data. I can’t expect a clear understanding of the impact of GAFE if a teacher hasn’t used it in his or her teaching.

Instead, I used the page branching feature in Google Forms to differentiate the questions. The first page is the same for all teachers and ends with the question: How often have you been able to use GAFE in your classroom this year? Teachers who answered always, often, or sometimes continued with an evaluation of its impact. Those who replied rarely or never followed with questions about why they hadn’t used it and what they would need to use it more.

Find the teacher survey here.

What impact does #GAFE have on teaching and learning?

Last year, this was the question on my mind. I had just left GAFE Summit NY, one of my first conferences, and was awed at how much everyone was talking about the positive impact of Google Apps for Education (GAFE), Chromebooks, and technology. I attended great sessions, watched my Director of Technology discuss our district’s Chromebook rollout and professional development models, and I even presented about how GAFE has changed my teaching and my students’ learning. But I left wondering about the real impact, which at the time meant numbers for me.

The Plan