Improving Outcomes for Learners Through the Use of Analytics


As a former instructional designer, making recommendations to faculty integrating technology into their courses, it was vital for me to know the impact of my recommendations on learner outcomes. Whether the advice was to screencast a PowerPoint lecture from a laptop or create an interactive learning object to engage students with course content, multimedia creation is a time-consuming activity. Is it worth the time investment? What are the benefits of its use? Are my students performing better on assignments and in the course? Do they prefer multimedia resources over text resources? While the traditional course evaluation included a question or two about course materials, technology use, and a field for additional comments, the limitation was in not being able to ask these questions directly. The availability of a variety of data at the course level makes it possible for an individual instructor to take an evidence-based approach to answer questions about instructional practices to improve outcomes for learners.

To explore these questions, two sections of the same course were taught using two different visual designs. One course took advantage of instructor created videos, and other multimedia, while the other course used text-based instruction only.  Several tools were available through the course. Blackboard’s survey tool was used to ask specific Likert scale and open-ended questions about the instructional materials available to each group of students. The attempt statistics report was used to summarize the data from the survey, and the excel download option located in the full grade center made it possible to manipulate the data. The course activity overview report was used to analyze the number of hours spent accessing content. This report also had an excel download option, allowing the number of hours spent by each student to be combined and analyzed with other data. The activity and grade scatter plot analytic report was used to compare the number of times the course content was accessed. With the data readily available through the course, addressing questions about instructional practice is convenient and more likely to be utilized.

Having used these tools, I was able to learn how students experienced the course design, and the impact instructional materials had on learning outcomes. For example, a well-designed course is more likely to positively impact learner outcomes with or without an abundance of multimedia. However, a positive perception of the course is enhanced through multimedia use, and this perception can influence course outcomes. A benefit of multimedia use made apparent through the data is that cognitive load is reduced, and clarity of assignment directions is enhanced. As an instructional designer or online instructor with limited time available for multimedia creation, this information allows for the strategic use of multimedia. While the desired effect of instruction may not always be to reduce cognitive load, it’s helpful to know that multimedia use is a tool by which it can be managed. Using data to answer questions about instructional practice encourages a consistent and confident application of what is learned. Furthermore, grounding instructional practice in data specific to the context for which it will be applied lends itself to improved learner outcomes. Making the data easily accessible from within the course may encourage more course level practitioners to explore an evidence-based approach to improving learner outcomes in the future.

Dr. Torria Davis is the Director of Technology Training for Information Technology Services at California Baptist University in Riverside, CA. She is the author of Visual Design for Online Learning, as well as a researcher, and presenter. She is a 2014 and 2016 winner of the Exemplary Course Program Blackboard Catalyst award.