SCEQ and SREQ Open Ended Responses
- Introduction
- Analysis of qualitative data from student evaluation of teaching surveys
- The Reports
- Faculty Reports
- Subject Specific Reports
- Related Publications
Quality enhancement at the University of Sydney is furthered by the collection, analysis, reporting and action on student feedback.
Data about the student degree experience in the University is collected through the Course Experience Questionnaire (CEQ) (previous year's graduates) and the Student Course Experience Questionnaire (SCEQ) (current students).
Analysis of this data provides a comprehensive picture of trends in the student experience, and the performance of the Faculty in relation to Learning and Teaching Performance Fund Key Performance Indicators (Generic Skills; Good Teaching; Overall Satisfaction), and other related areas (Learning Community, Curriculum, Assessment and Workload, Clear Goals and Standards).
Written observations from students about their experience provide evidence to support the percentage agreement scores, and provide detailed information about key issues in the areas of best practice or areas of concern.
The University of Sydney Course Experience Questionnaire (SCEQ) and the DEEWR/ GCA Course Experience Questionnaire (CEQ) are designed to collect quantitative and qualitative data about students' perceptions of the quality of teaching and learning in their degree courses. In both surveys students are asked to respond to statements linked to items in the S/CEQ Factors, using a 5 point Likert Scale to indicate the extent to which they agree or disagree with each statement. As part of the questionnaires students are also asked to provide comments on the best aspects of their degree experience and those that could be improved. These comments (qualititative data) form the basis of reports on the student experience which are distributed to university senior management; deans and associate deans; and other internal and external stakeholders.
Analysis and evaluation of qualitative data from the SCEQ and CEQ
The manual analysis and evaluation method used by the University to ascertain trends in the student experience is based on an in-house taxonomy which allows for standardisation of reporting across the University and ease of comparison with the S/CEQ quantitative analysis reports. Categories and sub-categories are based on the DEEWR Learning and Teaching Fund Key Performance Indicators, SCEQ Factors/ Scales and additional factors which contributre to the overall satisfaction with the quality of the students' learning experience.
The components of these categories and sub-categories are based on recurring themes in students' comments and have been developed over many years of analysing qualitative data from students' surveys. They represent the range of aspects of the student experience which are considered to have a major influence on the quality of the degree experience.
Key Performance Indicators for Learning and Teaching
The following Key Performance Indicators from the Australian Graduate Survey (AGS) and Course Experience Questionnaire (CEQ) (adminstered by Graduate Careers Australia in association with Australian higher education institutions) are used as part of the National Learning and Teaching Performance Fund:
- Graduate Generic Skills, that is, the level of satisfaction with generic skills acquired;
- Graduate Good Teaching, that is, the level of satisfaction with good teaching; and
- Overall Graduate Satisfaction, that is, the proportion of graduates who were satisfied with the overall quality of their courses.
In addition to the above scales, the University measures student satisfaction in the following areas through the internally administered Student Course Experience Questionnaire (SCEQ):
- Learning Community, that is, student perceptions of the social experience of learning at university. It indicates the students' sense of belonging to a community where learning with other people is a priority;
- Appropriate Assessment, that is, the extent to which assessment measures higher order thinking and understanding rather than simple factual recall;
- Appropriate Workload, that is, student perception of reasonable workloads.
Initiative LT1 in the new University Strategic Plan 2007-2010 states that we "Improve academic program structures to achieve greater coherence; provide choice while maintaining flexibility; and ensuring continuing relevance of our courses to students and employers (1)". Qualitative datea from SCEQ is used to help provide evidence of change in this area of the student experience. It is collected under the category Curriculum in the SCEQ/ CEQ taxonomy for the analysis of qualitative data from student surveys.
Quantitative and qualitative data from the CEQ and SCEQ provide evidence of the success of university and faculty initiatives to improve the overall student experience in general and the student experience of learning and teaching in particular.
Construct of the taxonomy
Part One of the taxonomy consists of eight main categories which are included in all faculty and university-wide reports on trends in the student experience:
- Generic Skills (KPI; S/CEQ Factor)
- Good Teaching (KPI; S/CEQ Factor)
- Overall Satisfaction (KPI; S/CEQ Factor)
- Learning Community (S/CEQ Factor)
- Curriculum
- Assessment (S/CEQ Factor)
- Workload (S/CEQ Factor)
Part Two of the taxonomy consists of additional categories which are used to gain a picture of the students' experiences of learning and teaching, and administration. Whilst they may not all be used in trends analysis reports, it is still important to analyse and collect the data for future reports; for whole of the university (subject specific) reports (e.g. library; research-enhanced learning and teaching; eLearning); and for ascertaining faculty performance against specific recommendations from previous Academic Board reviews. Categories included in this section are:
- Clear Goals and Standards (S/CEQ Factor)
- Research-enhanced learning and teaching
- eLearning
The following information is included for all of the above categories:
- Coverage
- Explanatory notes where necessary
- S/CEQ survey items (categories 1-4; 6-7; 9 only)
- Sub-categories
a. Aspect
b. Answers the question
c. Sample comments to illustrate each sub-category
Since reporting the results of the analysis is an integral part of quality enhancement at the University, a number of different types of report, using the data from student comments, have been designed to meet the needs of faculties, administrators, senior management and other stakeholders.
In 2003, the Federal Government introduced the Learning and Teaching Performance Fund to reward higher education providers that best demonstrate excellence in learning and teaching (1) . Student satisfaciton datda, through the CEQ, is used as part of the formula for determining the level of funding for an institution.
In 2007, for the first time, the University introduced a range of Key Performance Indicators for faculties. These included student satisfaction results from the CEQ.
In response to both these internal and external changes which increased the focus on performance outcomes, and to facilitate improved learning and teaching outcomes and the focus on student feedback, a significant rewiew of the way in which student feedback on learning and teaching is structured and delivered has been undertaken. As a result, the individual faculty reports on key issues in learning and teaching now feature:
- An Executive Summary highlighting key trends in the student experience for the Faculty. This aimed at providing the Dean with a high level overview of key issues to address.
- Arrangement by KPIs (Generic Skills; Good Teaching; Overall Satisfaction); and additional factors that go to make up the student experience of learning and teaching within the Faculty (Learning Community; Curriculum; Assessment; and Workload. Clear Goals and Standards; and Administration and Organisation are included where relevant to the faculty i.e. mentioned by a high percentage of students). This involved revising the taxonomy for the analysis of qualitative data (SCEQ/CEQ) at the University of Sydney by KPIs and Additional Factors. It now includes an analysis for every category, aspect, and component of the student learning experience.
- The inclusion, for the first time, of qualitative and quantitative data from the Course Experience Questionnaire (CEQ)
- Integration of analysis of the outcomes of both quantitative and qualitative data from the SCEQ and CEQ
- Comparative quantitative data from the SCEQ (2001-2007) and the CEQ (2004 - 2007)
- Comparative quantitative data from the SCEQ (2001-2007) and the CEQ (2006 - 2007)
- Key issues within each category from the 2007 surveys, including: main focus of comments; distribution of relevant comments; illustrative sample comments from domestic and international students
- Reports are customised for each faculty, according to the main focus of student comments. e.g. inclusion of data regarding physical facilities; communication skills of lecturers; online delivery of curriculum, etc.
Separate reports on undergraduate and postgraduate coursework experiences of student administration and student support services; and the experiences of postgraduate research students are also provided to faculties. During 2008, these will be redisgned to provide faculties, and administration/ support services, with a greater understanding of the student experience in these areas.
| Student Open Response Reports | |
|---|---|
| SCEQ/CEQ Undergraduate Coursework | |
| SCEQ/CEQ Postgraduate Coursework | |
| SREQ |
Subject-specific reports are developed on demand to provide annual or biennial information on the student experience in cross-University aspects of the student experience.
Current report series include:
Assessment and Student Feedback
- 2003 Student comments on Assessment and Student Feedback
- 2005 Student comments on Assessment and Student Feedback
- Analysis of 2005 Student comments on Assessment and Student Feedback
eLearning
International Students' experiences
- Report on the International Student Experience 2005: Analysis of areas of best practice and suggested improvements
- Report on the International Student Experience 2005: International student life, support and advisory services
- Report on the International Student Experience 2005: Learning and teaching and research training
- International Postgraduate Research Student Experience 2004
- International Student Experience 2003
- International Student Experience of Learning and Teaching and Research Training 2002
- International Student Experience of Student Administration and Student Services 2002
Library Services
- University of Sydney Library Services: the student experience 2005. SCEQ and SREQ Open Response comments: trends and analysis
- University of Sydney Library Services: the research student experience 2004
- University of Sydney Library Services: the student experience 2003
Research-enhanced Learning and Teaching
- Research-enhanced learning and teaching: the coursework student experience 2005
- Research-led Teaching: the coursework student experience 2000-2003
Student Services
- In their own words - finding out what the students think about their University experience
- Getting the most out of the SCEQ data: meeting the needs of multiple stakeholders
- Listening to the student voice at the University of Sydney: closing the loop in the quality enhancement and improvement cycle
- Taxonomy evolution: the changing face of reporting the student experience at the University of Sydney
- Colour coding, KPIs, and the Road to Damascus: shedding new light on reporting the student experience at the University of Sydney
- Analysing, evaluating and reporting on the student experience at the University of Sydney: responding to the changing demands of stakeholders
- The development of an in-house taxonomy used in the analysis of qualitative data from student satisfaction surveys