Week 4: How and Why You Should Evaluate Technology
Updated: Sep 17, 2018
New EdTech resources are being created and released at an exponential rate. So... How do you figure out if that flashy new tech tool is right for you? Here is how and why you should always evaluate new technology before adopting it for your classroom, school, or district.
EVALUATING EDTECH: THE 'WHY' AND 'HOW'
Picture this: One day you stumble upon a new EdTech resource, advertised as the 'most revolutionary innovation in educational technology in years'. Overwhelmed by the newness and excitement, you decide that your students must have it. So you purchase it without hesitation, only to abandon it just as quickly after it fails to provide the results you anticipated. Sound familiar? (Too familiar?) Don't worry, we've all been there. Even I, a pre-service teacher, am guilty of this. It's far too easy to get caught up in all the hype surrounding new technology, especially in the field of education. This cycle is fairly innocuous when it comes to small tech resources--think applications, or even robots. But neglecting to evaluate technology that will be used wide-scale throughout a school or district (like 1:1 devices or a new Learning Management System) can have horrible consequences on student learning, teacher effectiveness, and the school culture as a whole.
Regardless of the EdTech tool, it is crucial that educators and tech leaders thoroughly and critically evaluate the technology before implementing it in the classroom, school, or district. Changes in technology, when not appropriately tested, evaluated, and planned for, can decrease student learning, disillusion teachers, and impact the entire dynamic of a school. However, when planned for and implemented properly, educational technology can make learning more efficient, diversified, accommodating, and engaging for students (which is why we should not abandon it for fear of 'getting it wrong'). When evaluating technology, teachers and EdTech leaders should consider the following:
Relevance: Is the technology relevant to the lesson, course goals, and/or school mission/objectives? Does it support learning standards? (This should be your primary concern.)
Differentiation/Customization: Does the technology support/encourage differentiation and student customization? Does it make learning easier for a greater number of students? Can students change the settings to fit their individual needs?
Feedback: Does the technology make providing students with feedback easier/more efficient? Is the quality of feedback better?
Thinking Skills: Does the technology encourage students to engage in higher-level critical thinking and/or promote the development of that skill?
Usability/Reliability: Can students easily and independently use the technology? Is the interface user-friendly? Has the technology been positively evaluated and used widely enough by its target audience to ensure its reliability after implementation?
Engagement/Motivation: Are students more engaged when using the technology? Does the technology motivate students to learn and do their best?
Sharing/Collaboration: Does the technology promote student collaboration and support the sharing of student work with other students, teachers, community members, and online audiences?
Don't have the time for a full-scale test run? Reach out to other educators in your professional learning network and ask them what challenges they faced and how they adapted when implementing the same resources. Every classroom is different, of course, but their experiences with the technology should give you a better understanding of how it might fare in your own classroom and what problems to anticipate. Note: This is best for less complicated tech tools like new tablet applications or smaller, student-programmable devices (like Dash and Dot robots or Kano coding kits). Devices implemented on a broader scale (like 1:1 devices and Learning Management Systems) should always be thoroughly evaluated before being adopted and implemented in schools.
EVALUATING EDTECH: EXAMPLE WITH SAMPLE RUBRIC
To illustrate how to evaluate simple educational technology tools, I have created a sample evaluation of an application/website I recently discovered called Newsela. This purpose of this site is to provide students with news stories written at various reading levels to differentiate for comprehension abilities and provide all learners, from elementary to high school, with a way to engage with the news.
To evaluate the application, I used the following rubric, which you can find here. I think this rubric does a nice job of summarizing the main points that teachers should consider before implementing an application into their lesson or daily classroom routine. Note that my points for evaluating technology (located above) are roughly the same, with minor adjustments to account for differentiation, reliability, collaboration, and motivation (which I added from this second evaluation rubric).
While evaluating Newsela as a technology resource, I considered it through the context of my preferred grade level and content area--8th grade English/Language Arts--and the curriculum standards for those students. Here is a breakdown of my scoring of Newsela:
Relevance: 4/4 This is certainly one area in which Newsela excels. The app delivers exactly what it advertises--news stories differentiated for all grade and reading comprehension levels. It also relates directly to several Indiana 8th grade English/Language Arts standards (see 8.RN.2.1-2.3 and 8.RN.3.1-3.3 specifically).
Customization: 4/4 Again, this category appeals directly to Newsela's overarching goal of differentiation and really succeeds. It provides students with the ability to choose from five grade/reading levels--3rd, 5th, 7th, 9th, and 12th--with a simple swipe and automatically adjusts the content/wording of the news stories as appropriate. It couldn't get much easier.
Feedback: 4/4 In addition to providing differentiated news stories, Newsela offers comprehension quizzes by grade/reading level with automatic feedback, as well as open-ended analysis questions scored by the teacher.
Thinking Skills: 4/4 The built-in, open-ended analysis questions encourage students to think critically about the articles and provide textual evidence to support their arguments.
Usability: 4/4 The application is user-friendly and very simple to operate. Students from elementary school to high school could use it on their own with little to no teacher guidance.
Engagement: 2/4 One area in which Newsela is most noticeably lacking is its student engagement. Newsela is not especially engaging or intrinsically motivating compared to other applications, and students will likely still perceive it as 'work'.
Sharing: 3/4 Newsela allows teachers to assign particular articles to students and accumulate data on student progress and reading levels. However, there currently seems to be no way to export this data from within the application itself.
Overall rating: 25/28
There you have it! I hope this example was beneficial and helped make technology evaluation seem less intimidating. I believe that any teacher can and should perform this type of evaluation before attempting to implement new technology resources into his/her classroom. Your students (and your lesson plan) will thank you.
Until next week--Carpe diem!
Miss Hennings
I loved when you brought up the example of the flashy new "revolutionary" technology that you might see shoved in your sights by the internet. It is important to think critically about these tools, especially if they have not been put to the test in many other real world school environments. It is important to not get distracted by the new flashy look of it and to consider all of your options.
I loved when you pointed out the quote below;
"But neglecting to evaluate technology that will be used wide-scale throughout a school or district (like 1:1 devices or a new Learning Management System) can have horrible consequences on student learning, teacher effectiveness, and the school culture as a whole."
I think it is so important that when the technology fails we recognize that it will not only hurt the students and the teachers but it will also effect the school culture. If the school continues to do this people are going to get sick of spending time trying to incorporate technology which will turn the schools culture to have a negative view for technology which is something that we always…