- Incorporating Learning Analytics into Course Design
- Using Learning Analytic Tool data to inform course design
- Learning Analytics and Pedagogy
- Additional Resources
- Support
It is important to consider that analytics data collected by educational technology is the product of two factors: the design of the course and its evaluation structure as well as the performance of the learners. Therefore, to maximize the benefits of using Learner Data, instructors are advised to intentionally evaluate their course design with the goal of collecting learning analytics in mind.
Incorporating Learning Analytics into Course Design
These are overall design practices you should consider when designing your course with the intention of obtaining Learning Analytics.
| Recommendation | Example | Resources |
| Template course structures |
|
|
| Include “Checkpoints” |
|
Template course structures
The more data you pull from, and the longer the time frame, the more accurate the analytics you receive. Using a course template creates consistency in your data over multiple iterations of your course. This yield more robust analytics that you can use to refine your course. Using templates can also create consistency across multiple courses. Not only will students thank you for a predictable navigation throughout the course, you’ll be able to make direct comparisons and leverage predictive analytics.
Include “Checkpoints”
Checkpoints are assignments, tasks and interactions that you can use to observe and understand student behavior. Creating these types of assignments/tasks can give you robust data points at specific points in the term. Intelligent Agents can be very useful for creating these checkpoints. Make sure the data comes from throughout the term and is not front loaded. Having most of your data at one point of the term may skew/bias your analytics.
Using Learning Analytic Tool data to inform course design
You can leverage Learning Analytics by collecting data from Carleton’s supported EdTech Tools to help design and redesign your courses to better suit your pedagogical needs. Below you will find a list of Core Analytics and the most effective methods of using them to (re)design your courses.
| Learning Analytic (Data) | Course Design Practice | Resources |
| Completion Tracking and Class Progress |
|
|
| Quiz Statistics & Reports |
|
|
| Intelligent Agents Data Reports |
|
Completion Tracking and Class Progress
Completion (Click) Tracking allows instructors to see which pieces of content students have visited in a course. The Class Progress tool allows instructors to review student performance using any 4 chosen performance indicators from a selection of 9. Effective uses of these analytics are:
- Add links to Assessments as Content items in Modules.
To have the most accurate data on student performance, assessments must be created/linked to specific content items in modules. If the assessments are located elsewhere within the course, the analytics tracking will not be as accurate. - Use Brightspace tools over other LTI’s.
Analytics in Brightspace are only tracked through Brightspace’s tools and functions. Third party or external tools (LTI integrations) are not included in the metrics. To get the full benefit of Core Analytics, use Brightspace’s built in tools. - When using Quicklinks, try to prefer “content”.
The quicklink functionality can provide a powerful tool to redirect student to items of interest or importance. When using quicklinks make sure to have them redirect to modules, assessments and other forms of content for the greatest possible accuracy on student access and performance.
Quiz Statistics & Reports
Quiz Statistics provide information on how students performed on a D2L Brightspace quiz and a detailed analysis of each question. By selecting different types of Quiz Reports, instructors will be able to access class score distributions, quiz attempts logs, individual students’ activity in a quiz, and statistics of each quiz question. Effective ways to inform course design when using Quiz Statistics and Reports are:
- Build reports into your course design.
If you know that you will be using Quiz Reports and Quiz Statistics in your course, then its a good practice to build these reports and the feedback process into your course design. Create opportunities to use the feedback from Quiz reports in your Quiz designs, and set aside time to review the reports and redesign upcoming assessments if needed.
Intelligent Agents Data Reports
Intelligent Agents (IA’s) are automated tools that can be used to query many aspects of Brightspace courses and build reports based on the queries. Any query that can be made using release conditions can be also done using an IA. Here are some ways you can start using IA’s as part of your course design:
- New Student IA.
Having an Intelligent Agent email the instructor every time a new student joins the course can be helpful to provide context for changing student numbers, and in turn the analytics that are based on class counts. - Use IA’s to gauge student progress.
IA’s can be used to send emails when students have not completed specific course modules or benchmarks. For example, an Intro to the course quiz with questions about syllabus content can be used to gate the rest of the course content. Intelligent Agents can be set to flag students who do not complete this quiz and email the instructor. Since these students have not accessed the course content at all, they are possibly at risk of poor performance.
Learning Analytics and Pedagogy
Learning Data is the key, but only effective when pedagogy is considered. More information on the best practices of integrating Learning Analytics into your pedagogy can be found at the Pedagogical uses of Learning Analytics page.
TEI Course Design Resources
Teaching Excellence and Innovation has several resources that can inform your use of Learning Analytics when designing your course:
- Course Design and the Course Design Fundamentals Program
- Brightspace Course Setup resources
- (Re)Designing Your Courses
- Online Course Design Guide
Ethical Considerations for the use of Learning Analytics
Ensure that your Learning Analytics abide by all the principles of Ethical data use at Carleton. Please refer to the Ethics of Data Use page.
Additional Resources
For additional resources on Learning Analytics, how they can be implemented, and the ethics of their use, you can visit the Society for Learning Analytics Research site. SOLAR also provides a Learning Analytics Handbook, a peer reviewed resource intended to provide an overview of the current state of the Learning Analytics field.
Support
For help with Learning Analytics, please submit a request to the Learning Analytics Jira Queue. If you need help with designing and redesigning your courses using Learning Analytics Data, please reach out to Teaching and Learning Services using the Course Consultation Request Form. For any other questions regarding Carleton’s Educational Technology, please reach out via the TLS Support Portal.