Learning Analytics Amplifies Quality Teaching
The following article was published on E-Learn Magazine on Feb 15, 2018 and is based on an interview with Dr. John Whitmer, at the time Director Learning Analytics and Research for Blackboard Analytics. We are republishing it in its original form. Click here for the Spanish version.
As research and implementation on learning analytics advances, it is possible to catch a glimpse of a future in which data-informed features will become essential to help instructors and students make better decisions.
Learning analytics and student behavioral data insights are a new form of knowledge that was not available before educational technologies were so deeply integrated in the classroom. At least that’s the perception of Dr. John Whitmer, Blackboard’s analytics and research director.
Whitmer manages a team of data scientists distributed across the globe who work on research and analysis. With advanced degrees in statistics, training in machine learning and computer science, as well as experience with e-learning, these scientists think about and provide answers to questions about how institutions, students and instructors use these products in collaboration with Blackboard Product Management and Product Design teams.
“We look at how both faculty and students use our products in order to help make decisions about how to develop new features, refine existing features, as well as develop new dedicated embedded analytics features,” Whitmer explains.
Evolution in Data Collection
Before learning analytics, due to the difficulty accessing behavioral data at scale, the research that campuses conducted about student use of academic technology usually relied on student interviews and surveys, which aren’t always accurate. The response rates are low and the students who do respond are a biased sample, often reflecting power users or people with extreme opinions.
“They give us this new form of knowledge about what happened during a class that allows us to create better learning materials and learning experiences, as well as interact with students while the class is still going on, while there is still time to intervene, maybe even change the success of a student within that same class,” says Whitmer.
Next, learn about the main research findings by Whitmer’s team, as well as innovative features that can make a difference for teachers and students.
Can Learning Analytics Provide a Formula for Student Success?
Sometimes the biggest discovery is that a hypothesis cannot be confirmed. One of the most interesting research findings from Whitmer’s team is that, when looking at scale across all courses and institutions, there isn’t a strong relationship between student use of Blackboard Learn (in particular) and their class grade. In fact, there is a very small relationship between the two.1
“There is often an assumption in learning analytics that if you could get information about what students do and how frequently they use the platform, then you would have a magic wand that would completely reveal students’ success and factors around student learning. But when we look at it on a large scale, we find that this is not true,” he explains.
Instead, they found that it was important to look deeper into course design and the specific uses of Blackboard Learn. “For example, students that spend a larger amount of time looking at their grades tend to do better than students who spend less time doing that,” says Whitmer.
In classes that use assessments or discussion forums frequently, the amount of time students spend on those activities is directly related to their grade. In the end, what matters most is centered around student effort, pedagogical approaches and instructional practices.
How Can Democratizing Analytics Benefit Teachers and Students?
Educational research is often done by a small group of experts, and historically, the results only circulate around administrative and leadership positions within an institution. These leaders can choose to disseminate the information or not.
“But often teachers and students have a lot of questions, and they are very interested in data and information that can help them make decisions,” points out Whitmer. Democratizing analytics means making data and insights accessible to them, as well as to other stakeholders that can benefit from it.
Ethics is also something that needs to be carefully considered, according to Whitmer. “That is an important part of what Blackboard does to ensure that the powerful techniques we can apply are aligned with what we should do, in terms of the ethical obligations that we have to students. And those obligations are different around the globe, given different laws and regulation and cultural practices around student data,” he says.
How Can a Teacher Begin to Use Learning Analytics?
Fundamentally, learning analytics means that the teacher can study students’ actions and how frequently they access learning materials and activities.
“Before learning analytics, we did not know, or at least we did not know at large scale, what happened while students were in class, how often they studied, how they studied, how often they opened the materials, how they interacted with them,” points out Whitmer.
The expert suggests that teachers should start using analytics by identifying the most important questions that they need evidence in order to answer.
“For example, are my most successful students accessing and participating in their learning activities, or are my least successful students not participating in certain types of activities? Looking at those relationships between activity in student learning at a very detailed level, looking at discussion forums, looking at your lecture materials, and reviewing notes, instructors can get a sense of what students are actually doing in their course and how it relates to the success they have,” recommends Whitmer.
A Blackboard initiative can make this process much easier in the future. With embedded analytics, Blackboard is integrating analytics within the workflow of what teachers are already doing, a unique approach in the market.
“Teachers often do not have the time within their busy lives and teaching schedules to go off and think separately about these questions. So, instead of going to a separate reporting area or a menu for analytics, Blackboard is embedding analytics within the workflows the teachers are natively engaging in,” explains Whitmer.
In practice, that means that when instructors log into the LMS, they can receive summary updates about how their students are doing and which ones may need attention. When teachers are grading, they get information about student participation. When instructors are reviewing discussion forums, they get information about the quality of student posts, for example. “We bring the analytics to them,” says Whitmer.
Do Students Notice Notifications?
The possibility of enabling automatic notifications for students based on their performance and activity is an innovative feature in the Blackboard Learn Ultra Experience. These alerts are embedded directly within the course, and when they are clicked, a student receives detailed information as well as a suggested action.2
According to Whitmer, this feature has been discussed for some time by educators, but institutions have been hesitant in providing notifications directly to students because they are concerned about potential negative repercussions. After all, individuals receive so many alerts and notifications daily that distraction and lack of focus are becoming serious problems.3
However, research found that sending rule-based notifications to students is beneficial and recommended. “We have found that students are very interested in receiving these notifications and alerts, and they access them much more frequently than they do other types of e-mails or notifications they get from educational institutions,” says Whitmer. “Students want to know how they are doing, and how they are doing relative to their peers, both positive and negative.”
In that sense, learning analytics can be intended to identify what is most important and put that at the center of the student learning experience, so they are better able to pay attention and to focus.
“Students can have their own ideas about what they need by going through learning analytics and finding successful or unsuccessful patterns in the past. We are able to identify these behaviors and practices and then bring them to students’ attention,” says Whitmer.
What Will Happen Next?
More Sophisticated Solutions, Better Insights: Learning analytics is still at its early stages. As education technologies become more and more integrated, the amount of data available for analytics will increase, as well as new analysis techniques to derive meaning from this data. The solutions for institutions, educators and students tend to become more robust and sophisticated over time.
More Critical Customers: As learning analytics becomes more common, students and faculty will have more experience and become critical consumers of these solutions. They will be able to distinguish, for example, between learning analytics and conventional reporting.
More Effectiveness: Learning analytics will be increasingly used to help instructors, administrators and students to make decisions. Also, students will be more successful in achieving their educational goals. This doesn’t mean that analytics will replace human decision-making. “We don’t see a scenario in which analytics is anything close to replacing human judgment,” says Whitmer. “In a sense, learning analytics is augmented intelligence.”
Sources:
1 Whitmer, J. (2016, March 18). Research in progress: Learning analytics at scale for Blackboard Learn. Retrieved November 22, 2017, from http://blog.blackboard.com/research-in-progress-learning-analytics-at-scale.
2 Whitmer, J., Nasiatka, D., & Harfield, T. (2017, July 26). Do notifications get noticed? New study finds that embedded alerts to students promote action at high rates. Retrieved November 22, 2017, from http://blog.blackboard.com/new-study-notifications-promote-student-action.
3 Rosen, L. (2012, April 09). Attention Alert! Study on Distraction Reveals Some Surprises. Retrieved November 22, 2017, from https://www.psychologytoday.com/blog/rewired-the-psychology-technology/201204/attention-alert-study-distraction-reveals-some.
Photos by: AFP – John Hefti