Responses of individual questions

Based on the transcriptions and notes of the interviews, the 14 interviewees' responses are tabulated below. For ease of comprehension, the category which receives the most responses is presented first.

Question 1 asks: "What role does assessment play in learning? Why do we need assessment in teaching?" Table 1 sums up the responses of the 14 interviewees.

Over 70% (10 out of 14) of the respondents regard assessment as a measurement of the progress of learning and teaching. This is consistent with the fundamental principle of the criterion-referenced model of assessment in which assessment is "designed to assess changes in performance as a result of learning, for the purpose of seeing what, and how well, something has been learned" (Biggs, 1999a, p.144). More specifically, to evaluate how much the learning objectives have been achieved is central to the conceptualization of the SOLO taxonomy (Biggs, 1999a, pp.43-50). As shown in Table 1, the majority of the interviewees share this underpinning philosophy.

On the other hand, nearly half of the interviewees believe that assessment is to "rank" students, which serves the ultimate goal of awarding learners with credentials. This line of thoughts resembles the type of "institutional" mindset which usually stipulates that the distribution of grades and final awards should follow a "normal distribution" (i.e. a bell curve). This "norm-referenced" approach was once the norm within the university and the traces of that norm are evident in many staff members' attitude.¹

Question 2 asks: "When you design an assessment task, what is your major concern?"

Overwhelmingly, the interviewees consider relevance to teaching and learning as the most important element in designing their assessment tasks. This reflects that the alignment between the goals of teaching and learning and the assessment tasks is of top priority among this group of staff members, a phenomenon which is encouraging as far as the implementation of SOLO is concerned.

Question 3 asks: "What types of assessment tasks have you used in your teaching (e.g. exam, term paper)?"

Question 4 asks: "Which type do you think is the most effective?"

Table 3 and Table 4 above demonstrate several characteristics of the assessment tasks used by staff members in the English department. First, a large variety of assessment methods were employed. A total of nine different types were recorded. In terms of effectiveness, no one particular type was regarded as "the most effective". It depended upon the objectives of the assessment.

Second, some assessment tasks involve on-going reflective critical thinking, such as journal writing, and web discussion. Such tasks are deemed to be the more effective ones by some staff.

The third feature is that many of these methods require a combination of high-level skills such as data analysis and logical organization (e.g. projects, term papers, oral presentations, student-led seminars, etc.). These tasks enable the application of the full range of the criteria outlined in the SOLO taxonomy.

Such characteristics show a healthy sign towards the adoption of the criterion-referenced approach and the application of the SOLO taxonomy.

Question 5 asks: "Could you please briefly describe the departmental assessment criteria?" ²

Question 5 was meant to ensure that the interviewees were indeed aware of the implementation of the SOLO taxonomy within the department so that their responses were valid. The result was quite promising. 12 out of the 14 members were able to articulate the correct response. This makes their responses to the follow-up questions, Question 6 through Question 9, valid and meaningful.

Question 6 asks: "To what extent have you followed the departmental assessment criteria in your assessment practice?"

Table 6 indicates that the majority of staff members reported to have adhered to the SOLO taxonomy, though a few complained about it being difficult to follow. Details of the difficulties in applying the SOLO taxonomy will be discussed under Question 8.

Question 7 asks: "In what ways do you think the departmental assessment criteria have helped you in assessing students' work?"

The positive comments about the implementation of the SOLO taxonomy are mainly conceptual in nature. For example, it is found to serve as an overarching philosophy as far as assessment is concerned, from as abstract as clarifying focuses to as practical as helping teachers design assessment tasks.

Question 8 asks: "What difficulties did you encounter in implementing the departmental assessment criteria?"

Table 8 above lists the problems that the interviewees encountered in applying the SOLO taxonomy in the respective subjects that they taught in the academic year 2003/04. It is discovered that most of the problems lie in the logistics of implementation. The major problems are two-fold. On the one hand, how consistency and reliability can be achieved across markers raised some concern. On the other hand, staff members found it difficult to apply the criteria without modifications since the criteria are very comprehensive and broad. Some teachers had to provide more specific descriptions when they applied the SOLO taxonomy to some tasks such as oral presentations. Others found that they had to spend more time on marking because they often had to re-read the assignment several times to ensure that the grades given matched the descriptions in the criteria.

Question 9 asks: "Suggest ways that the departmental assessment criteria can be improved."

The ways to improve the assessment criteria suggested by staff members address the issues raised in the previous question. They deal with the details of implementation. Six interviewees commented that the assessment criteria need to be more refined or more elaborate so that they can be more readily applicable to specific tasks. Others felt the need for parity across teachers and across subjects. They suggested that more specific and detailed procedural guidelines should be provided for staff members to discuss the grades among teachers across subjects.

¹ The University adopted criterion-referenced assessment in 2005/06.

² In one interview, Question 5 was not asked so only 13 responses (not 14) were recorded for this interviewee.

مواضيع ذات صلة

Problems and possible solutions

How useful is the SOLO taxonomy?

Summary of Findings

Results and findings Overall responses

Procedure

ENGL subjects offered in 2003/04

Interview questions Participants

Research methodology

Aim of study

Modified SOLO taxonomy for ENGL

Application of the SOLO taxonomy

Other factors that affect learning: epilepsy, HIV/AIDS, puberty and mental health issues Key issues