- The Goals of using Course Level Analytics
- Informing Course design and re-design
- Evidence-Based Student Intervention
- Guidance to improve learning materials and tools
- Observe Social Learning
- Resources
- Support
The Goals of using Course Level Analytics
When accessing course level learning analytics, the goal is ultimately to improve learning outcomes for either specific courses or specific students. At the course level, learning analytics can help inform course design and re-design as well as guidance to improve learning materials and tools. At the student level, using learning analytics can help provide students with more agency over their learning and motivate them to become engaged with the course (Lim et al., 2021). Learning analytics metrics can help identify students who require support, inform the type of support they might need, and allow instructors to support students in the most effective way.
Informing Course design and re-design
Learning Analytics can be used to observe student engagement paths within the LMS, find out how much time students spend viewing material, and observe how they behave within Quizzes and Assessments. All of this data can be very valuable when considering how to Re-Design courses for increased student engagement and improved learning outcomes.
Cindy used Quiz Statistics and Rubric Reports to re-design assessments
Cindy teaches an Intro to Psychology course, and used Rubric reports and Quiz Statistics, to guide her decisions on how to rework her course to make the best possible experience for her students. These metrics informed Cindy’s decision to re-designs aspects of her course, re-teach a key concept and re-design an assessment that some students found confusing.
I redesigned my instructions and my rubrics after my students completed their first assignment. After investigating their grades more granularly via Rubric Reports in Brightspace, it was clear to me that elements of my course required a minor redesign. After viewing my quiz statistics, I could tell that students were not able to identify the correct definition of a correlation. With this information, I decided to re-teach this concept in the next class. Also, I noticed many of my students chose the same wrong answer on a few of my quiz questions. After taking a deeper dive into these questions, I found that some of them were misleading, and I needed to re-write the questions.
Evidence-Based Student Intervention
Data collected by Learning Analytics tools can be very useful in finding students who are at risk of poor outcomes, and who might require individual intervention. For instance, Learning Analytics can be used to find out which students were late submitting assignments or who have a trend of decreasing Quiz or Assignment grades. This type of data provides continuous feedback to the instructor about which students require interventions and how the interventions have affected their learning outcomes.
Mario used Survey Reports and Intelligent Agents to offer additional support to students
Mario teaches an Introduction to Economics course which is a large enrollment course. To better reach out to students that require additional support, Mario has leveraged the power of Intelligent Agents and Survey Reports.
I’d consider myself to be a heavy Survey Tool user in Brightspace. After all, what better way to collect learning analytics than to survey students! For example, I have a survey set up, wherein students can request a phone call from me. I’ve used Brightspace’s Intelligent Agents to send me a notification anytime a student fills out the survey. One way for students to tell me they need help is via my Checkpoint Surveys. Each week, students are prompted to—as part of their Brightspace Checklist—complete a one-question checkpoint survey. The survey asks them something like “On a scale from 1–5, how confident are you feeling in your ability to do well in the course?” I show the aggregate statistics back to students in class so that they know they are not alone. Additionally, I reach out to students (via email) who report they are not feeling confident (i.e., a 1 or 2 rating) and ask them how I can help them feel more confident. Oftentimes, students just need to be reminded about the existing support resources available to them (e.g., Office Hours).
Alex used the Class Progress tool to prepare for one-on-one’s with their students
Alex teaches a large-enrollment Business Analytics course. To create deeper connections with their students, and to help understand individual student’s trajectory through the course, Alex uses the Class Progress tool to get a ‘snapshot’ of the student’s progress thus far.
In large-enrollment courses, students often feel invisible, as though the instructor is not aware of them or their progress in the course. For some students, this can really dampen their motivation and ultimately their success. So, I like to offer 15-minute one-on-one sessions with my students. In the few minutes leading up to the meeting, I use learning analytics to prepare myself for the meeting. Brightspace’s Class Progress tool allows me to view a snapshot of a particular student’s engagement with my course. So, when I’m meeting with the students, I can say things like “I noticed you did much better on Quiz 3 compared to Quiz 1 and 2. You must have been working hard lately.” This way, students know that—even though they are part of a large group—I can see their progress in the class. Oftentimes, this is just enough to keep them motivated throughout the term.
Guidance to improve learning materials and tools
Learning Analytics can track how students use course materials and tools, and this data can be used to identify areas where Learning Materials are insufficient for student needs. Data can also be used to view common points of friction or failure in the way Educational tools are implemented in a course.
Hunter used Content Reports to inform modifications to learning tools
Hunter has been teaching the Social Determinants of Health course for several years and is looking for ways to optimize the use of the Educational Technology tools to give students the best possible outcomes. By using Brightspace’s Core analytics, Hunter was able to optimize their use of Modules and Submodules to improve student’s user experience through the course materials.
Since I teach the same course every year, I’m very interested in how to make changes that might improve students’ learning experience and even their academic success. So, after each iteration of the course, I compare their Final Grades to how they interacted with the course. Specifically, I look at my A range students’ engagement compared to that of my DFW range students. For example, one year I found that A range students viewed the resource submodule I had in the Course Documents Module. So, I turned it into a Resource Module and made it more prominent in Brightspace.
Pat used Survey Reports to change course elements
Pat teaches an Introduction to Statistics for Psychology course. Previously, Pat had used class time to solicit feedback from students on their experiences with the course. Pat is now able to use the built-in Survey tool in Brightspace to perform much more detailed and informative surveys of their students.
I used to ask students to give me feedback during class time. I would pass around cue cards and ask them to write down three things they liked about the course and three things they wish they could change. Now, I have a survey in Brightspace that asks the same thing! There are a few advantages to doing this digitally. First, I don’t have to spend class time on surveying my students and they aren’t restricted in completing the feedback when they might not have had time to fully process their thoughts about the course. Second, Brightspace Surveys allow me to share back the results with the students in an aggregate way. They can see what their peers wrote without seeing who wrote what! This practice is important because it’s often that some students will dislike aspects of the course that other students find useful. Finally, this digital format is more efficient, which gives me more time to actually make informed decisions about what might need changing.
Observe Social Learning
Learning Analytics can be used to observe student social behavior in shared online spaces such as Brightspace Discussion Groups. These student interactions can be a valuable indicator of the level of student engagement with the course, and the degree to which Social Learning is influencing student outcomes.
Terry used Discussions and Discussion Stats to observe and encourage social learning
Terry teaches an Introduction to Ethics course, and they use Discussions and Discussion Statistics to observe students discussions and to encourage their students to share their learning.
Just because we are back to learning in person doesn’t mean that a sense of online community isn’t important. I want students to know that they can—and should—support each other’s learning. Ethics can be a tough subject when taking it alone. So, I encourage them to create study groups. A lot of these groups will meet online. So, one of the first activities available to students in the course page is open a Discussion Forum called “Introduce Yourself”. Then, I respond to each and every post to reinforce participating in class online. I will also create different discussions forums for various topics in the course and use some class time every week to encourage the students to participate in the discussions. Regularly throughout the term, I run discussions statistics to observe which topics are most discussed and which forums receive the highest level of engagement. This helps me understand the aspects of the course that the students find most engaging. But mainly, I share the fact that I have seen their discussions (and statistics) and encourage them to continue to discuss the class and learn from one another, not just me!
For more information on the Analytics Tools discussed in these user stories, please visit our page on Core Analytics in Brightspace.
Resources
- Pedagogical Uses of Learning Analytics.
- Students’ perceptions of, and emotional responses to, personalised learning analytics-based feedback: an exploratory study of four courses. Lim et al., 2021.
Support
If you’d like to discuss how learning analytics can enhance your pedagogy and strengthen course design, connect with Teaching and Learning Services! TLS staff would be happy to provide consultation and/or to guide you toward additional support materials and training.