Indicators of implementation and evaluation of the service by users
Academic Year 2010/2011

1. Introduction

Following the end of the two academic semesters with regard to 2010/2011, the Internal Evaluation and Quality Assurance Multidisciplinary Team present the indicators of implementation and evaluation of the service marking exams by optical reading.

2. Indicators of implementation

The data refers to 39 requests for the OCR service in semesters 1 and 2 of the academic year 2010/2011, by CUs and Disciplinary areas of the Integrated Master Degree in Medicine.

2.1 No of Curricular Units requesting the service

¬ The total number of CUs or disciplinary areas that requested the OCR service in the 1st stage of assessment requested it again in the 2nd stage of assessment, in both semesters.

2.2 No of answer sheets requested by the curricular units and number of sheets submitted for OCR

¬ In the totality of the two semesters, 14406 prints of answer sheets were requested, of which 10645 were marked by optical reading.

¬ Approximately 26% of the answer sheets were not returned by CUs or disciplinary areas, which accounts for a waste of 3761 paper sheets (more than 7 reams of A4 paper)

2.3 Average delivery time of results to the curricular units (between receipt of the correction key and delivery to the user)

¬ When requesting the service, users were informed that the response time for delivering results would not exceed 48 hours. This goal was largely exceeded in both semesters, and there was a very significant decrease in semester 2 compared to semester 1.

3. Evaluation by users
Data are relative to requests made in semesters 1 and 2 by the CUs of the Integrated Master in Medicine. All 39 users participated in the evaluation of the service they benefitted from.

3.1 Satisfaction with the service

In order to evaluate user satisfaction, a survey by means of a questionnaire was carried out at the end of the examination period. The questionnaire sent to all users (19 in semester 1 and 20 in semester 2) had a response rate of 100%.
The following charts show the distribution of average satisfaction on a scale of 4 categories, against the evaluation indicators used in the OCR service:

Dissemination of the service; Procedures associated to requesting the service; Waiting time for return of results; Accessibility of the information provided (Excel files and images); and Overall satisfaction with the OCR service

Semester 1 (n=19)

Legend:
1,0 – Dissatisfied
2,0 – Little Satisfied
3,0 – Satisfied
4,0 – Very Satisfied
Semester 2 (n=20)

Legend:
1,0 – Dissatisfied
2,0 – Little Satisfied
3,0 – Satisfied
4,0 – Very Satisfied

3.2 Future use of the service

¬ The totality of users (n=39) reported they would use the OCR service again.

¬ The totality of users (n=39) reported they would recommend the OCR service to a colleague.

4. Main conclusions

The OCR service had a major impact on its users, who evaluated it very positively. It is worth noticing that user satisfaction peaked in semester 2 compared to semester 1, which attests the improved management of the process by the Internal Assessment and Quality Assurance Multidisciplinary Team.

There is also evidence that the average time for delivering results to the curricular units has also been significantly reduced between the two semesters.

In general terms, it can be stated that the service exceeded initial expectations!

Thank you.

The Internal Assessment and Quality Assurance Multidisciplinary Tea thank all users of the OCR service for participating in the survey.

Thank you for helping us to improve!

Luís Pereira, Pedro Marçal, António Santos, Rodolfo Martins
Internal Evaluation and Quality Assurance
leituraexames@fm.ul.pt