Frequently Asked Questions
What is a program-level learning outcome?
Program level-learning outcomes are clear, concise, and program specific statements about the learning goals of the program. They descried the observable ways your students are changed as a result of learning they have achieved through the program as a whole.
Why is “understanding” not a learning outcome?
“Understanding” is broad and largely invisible concept. We all have a certain level of understanding of most things we encounter through our lived experiences. Your students will have some degree of understanding of the discipline(s) they are studying, but this understanding will develop and change (we hope!) over the course of their degree program. The articulation of learning outcomes for the program are one way that change in understanding is made clear.
“Understanding” is also largely invisible. No one can see how much understanding we have by just looking at us. We show others how much (or how little) we understand something through our actions. Being able to provide a definition or a term is one way to show understanding. Being able to draw from previous knowledge to come up with an entirely new idea, or solution to a problem is another. In between there are countless ways in which we use and demonstrate the understanding we have.
Bloom’s taxonomy is a useful tool that can help you decipher what abilities correspond to the level understanding you expect your graduates to have attained by the end of the program.
Why do I have to do this?
The Quality Assurance Framework (QAF) requires that learning outcomes are made clear, aligned to modes of program delivery, and assessed for every academic degree and diploma program within the publicly-funded Ontario universities. Specifically contains the following sections:
4.3.1 Objectives: a)Program is consistent with the institution’s mission and academic plans. b)Program requirements and learning outcomes are clear, appropriate and align with the institution’s statement of the undergraduate and/or graduate Degree Level Expectations.
4.3.2 Admission requirements: Admission requirements are appropriately aligned with the learning outcomes established for completion of the program.
4.3.3 Curriculum: a) The curriculum reflects the current state of the discipline or area of study. b) Evidence of any significant innovation or creativity in the content and/or delivery of the program relative to other such programs. c) Mode(s) of delivery to meet the program’s identified learning outcomes are appropriate and effective.
4.3.4 Teaching and assessment: a) Methods for assessing student achievement of the defined learning outcomes and degree learning expectations are appropriate and effective. b) Appropriateness and effectiveness of the means of assessment, especially in the students’ final year of the program, in clearly demonstrating achievement of the program learning objectives and the institution’s (or the Program’s own) statement of Degree Level Expectations
Is satisfying the QAF that the only reason for doing this?
Certainly not! Although this relatively new criteria set forth by the QAF has stimulated more interest and activity in learning outcomes assessment at the program level, assessment should not be done for this sake alone. The QAF also require that units state the admission criteria admitting prospective students; however, no one creates admission criteria just to satisfy this requirement! In the same way, you should want to assess your program learning outcomes because it provides you with useful information that helps you deliver your programs more effectively.
Do I have to assess every course in our program?
No! That would leave much time for actually teaching those courses, would it?! If you would like to assess actual achievement of program learning outcome, consider selecting one or two key course-embedded activities, such as a capstone project, thesis, or a final project. Remember that program-level assessment is about student achievement over the course of the entire program, and not achievement within individual courses.
Do I have to assess every student in our program?
That depends. Programs with relatively manageable numbers of students may want assess every students’ level of achievement of the program learning outcomes. Programs with large numbers of students may find this to be an impossible task, in which case they may need to select a sample of students to assess. Sampling is ok, as long as efforts are made to ensure the sample is representative of the entire program, and not just the high or low achieving students. Remember that, although it may be student work or student experience you are assessing, the conclusions drawn should be directed toward the program.
Do I have to assess student work at all?
No. Direct, curriculum-embedded assessment is one way to assess the learning outcomes of your program, but it is not the only way.
What other methods can I use?
Surveys or focus groups with students and alumni are another way to assess the learning outcomes of your program. Facilitates faculty discussions are another way. Both of these methods are indirect – meaning they look at perceptions of ability, rather than actual ability. Find out more about assessment methods on our Assessment of Learning Outcomes page.
Do I have to report the findings of our assessment activities?
That’s a hard no. Right now the Cyclical Program Review (CPR) process requires academic units to share the learning outcomes developed for every program they offer, the curriculum maps that show how those learning outcomes are developed throughout the program, and the assessment activities they have undertaken, or will undertake, to assess the achievement of those learning outcomes. Nowhere in the CPR process are units asked to share what they found through their assessment activities.
Will program assessment data be used to determine program funding?
The Ministry of Advanced Education and Skills Development is currently reviewing the funding model for universities. It seems likely that the province will be moving away from the “bums-in-seats” funding model that is based solely on enrollments to one that incorporates other factors, such as outcomes. Some people (us included) are worried that this means the results of program learning outcomes assessment will used to make unit-level funding decisions. But, as faculty are currently responsible for developing the learning outcomes of their programs themselves, there is no way to compare achievement of learning outcomes across programs. Given this highly program-specific nature of program learning outcomes, it seems unlikely they could be used to determine funding allocations.
Where can I get more help?
The Office of the Vice-Provost and Associate Vice-President (Academic) currently has TWO Program Assessment Specialists to help you. Call or email us today, or whenever you need guidance and support.