Designing your courses to gather learning analytics data
Use this use case table to design your course based on the Learning Analytic data you wish to gather from the course.
| If you want to know… | use these tools…. | and this design practice. |
| Which pieces of content students have visited | ||
| How students have performed on quizzes, including their activity, attempt logs and score distributions | ||
| If students have engaged with the course at all |
|
|
| When a new student joins the course |
|
|
| How different cohorts of students have performed relative to one another across iterations of the course |
Tools
These are the tools you can use. This includes Brightspace Core Analytics and built in tools.
Completion Tracking
Completion (Click) Tracking allows instructors to see which pieces of content students have visited in a course.
Class Progress Tool
The Class Progress tool allows instructors to review student performance using any 4 chosen performance indicators from a selection of 9
Quiz Statistics
Quiz Statistics provide information on how students performed on a D2L Brightspace quiz and a detailed analysis of each question. By selecting different types of Quiz Reports, instructors will be able to access class score distributions, quiz attempts logs, individual students’ activity in a quiz, and statistics of each quiz question
Syllabus Quiz
Include a syllabus quiz at the beginning of the course to identify studenrts who have not engaged with any course materials. Here is an Example Syllabus Quiz.
Intelligent Agent
Intelligent Agents (IA’s) are automated tools that can be used to query many aspects of Brightspace courses and build reports based on the queries. Any query that can be made using release conditions can be also done using an IA.
Practices
These are course design practices that can help you design courses that streamline your ability to gather learner data.
Add links to Assessments as Content items in Modules
To have the most accurate data on student performance, assessments must be created/linked to specific content items in modules. If the assessments are located elsewhere within the course, the analytics tracking will not be as accurate.
Use Brightspace tools over other LTI’s
Analytics in Brightspace are only tracked through Brightspace’s tools and functions. Third party or external tools (LTI integrations) are not included in the metrics. To get the full benefit of Core Analytics, use Brightspace’s built in tools.
When using Quicklinks, try to prefer “content”
The quicklink functionality can provide a powerful tool to redirect student to items of interest or importance. When using quicklinks make sure to have them redirect to modules, assessments and other forms of content for the greatest possible accuracy on student access and performance.
Build reports into your course design
If you know that you will be using Quiz Reports and Quiz Statistics in your course, then its a good practice to build these reports and the feedback process into your course design. Create opportunities to use the feedback from Quiz reports in your Quiz designs, and set aside time to review the reports and redesign upcoming assessments if needed.
Include “Checkpoints”
Checkpoints are assignments, tasks and interactions that you can use to observe and understand student behavior. Creating these types of assignments/tasks can give you robust data points at specific points in the term. Intelligent Agents can be very useful for creating these checkpoints. Make sure the data comes from throughout the term and is not front loaded. Having most of your data at one point of the term may skew/bias your analytics.
New Student IA
Having an Intelligent Agent email the instructor every time a new student joins the course can be helpful to provide context for changing student numbers, and in turn the analytics that are based on class counts.
Use IA’s to gauge student progress
IA’s can be used to send emails when students have not completed specific course modules or benchmarks. For example, an Intro to the course quiz with questions about syllabus content can be used to gate the rest of the course content. Intelligent Agents can be set to flag students who do not complete this quiz and email the instructor. Since these students have not accessed the course content at all, they are possibly at risk of poor performance.
Template course structures
The more data you pull from, and the longer the time frame, the more accurate the analytics you receive. Using a course template creates consistency in your data over multiple iterations of your course. This yield more robust analytics that you can use to refine your course. Using templates can also create consistency across multiple courses. Not only will students thank you for a predictable navigation throughout the course, you’ll be able to make direct comparisons and leverage predictive analytics.
Additional Resources
For additional resources on Learning Analytics, how they can be implemented, and the ethics of their use, you can visit the Society for Learning Analytics Research site. SOLAR also provides a Learning Analytics Handbook, a peer reviewed resource intended to provide an overview of the current state of the Learning Analytics field.
Support
For help with Learning Analytics, please submit a request to the Learning Analytics Jira Queue. If you need help with designing and redesigning your courses using Learning Analytics Data, please reach out to Teaching and Learning Services using the Course Consultation Request Form. For any other questions regarding Carleton’s Educational Technology, please reach out via the TLS Support Portal.