Dissecting two different assessments:

I will consider two assessments, Computer-based multiple-choice exams (CBMCEs), which I would like to use, and Objective structured clinical examinations (OSCEs), which I already use.

Description of the assessments and how they are used with student

 

Computer-based multiple-choice exams (CBMCEs) Objective structured clinical examinations (OSCEs)
Description Students are asked to answer multiple-choice exams on a regular basis via an online platform. This can be conduct from the learners’ computers or within computer labs inside the institution itself. The later approach can ensure authenticity of the student, but can restrict timing and increase the required resources for the assessments.  The are commonplace in medical education. The learner is asked to move between stations to perform tasks such as interpret a slide under the microscope, carry a test, or talk to a patient and take medical history..etc.

The CBMCEs, which I am not using currently can be placed regularly (weekly or biweekly) and can be redeemable, so they can allow students continuous engagement and incremental leaning throughout the semester. CBMCEs allow students to gauge their level of understanding and add another reason to study regularly. The OSCEs, which I use with the students, are time consuming and demanding on the faculty but very important in medical education. They are conducted usually twice per semester. It is difficult to allow them to be redeemable. Nonetheless, they ensure authenticity of the leaner and the assessment itself. OSCEs make leaners see the relevance of their studies and enhance their motivation to learn. The diversity of the skills that can be assessed add to the depth of OSCEs and these skills can be tested in different ways so that a high level of equitability. The end results is the OSCEs can assess many skills including physical examination skills, interpretation of laboratory results, technical skills and communication abilities.

OSCEs can be improved by providing the most clear instructions possible for each task and defining assessment criteria to increase its reliability. They can be designed as a multistage iterative model allowing for its improvement. That is, initially needs assessment by faculty and regulating agencies can determine the areas that are most important. Next, program development and concepts definition to formulate goals and objectives can follow. Finally, the design and implementation of the OSCE tasks, conducting the assessment and allowing evaluation by the leaners can be carried out. This is to be repeated to improve the outcomes. 

CBMCEs can be improved by writing questions that assess various levels of learning (deep and surface) and skills (recall, analysis, evaluations, calculations and applications of equations). The reliably of CBMCEs, which is good, can be even further enhanced by including a number of questions assessing a single learning objective. Also, the questions can be made to focus on a broad set of course material, to increase the validity of the assessment.

advantages and disadvantages for the students and for learning.

Computer-based multiple-choice exams (CBMCEs) Objective structured clinical examinations (OSCEs)
Validity Can carry significant validity as it can measure most types of knowledge. However, it can fail to capture psychomotor abilities, interpersonal skills, ability to discuss complex issue, and similar outcomes such as the ability to organise ideas or articulate explanations and creativity. High validity
Reliability Very reliable. Depends on the method used in the assessed tasks
Equitability Not very equitable as it can be very standardized and fails to accommodate for many differences in the students. More equitable then CBMCEs
Timeliness Can provide immediate simple feedback with explanation after the test immediately. And the tests can be made in different times an required. Usually require more time for their set-up and performance, and therefore may not be as timely as CBMCEs
Incrementality Can allow incrementality Usually requires more preparations from the faculty and therefore may not allow for as much incrementality as CBMCEs 
Redeemability Can allow redeemability efficiently is the test can be re-taken with a different set of questions for a question bank that contains large number of questions Very difficult because of the time and effort required for the set-up and marking of OSCEs
Ability to challenge Can be appropriately challenging. Can be appropriately challenging
Efficiency Very efficient. Efficient in assessing different skills in a valid and authentic manner.
Authenticity of leaner Subject to authenticity problems (someone else may be answering the computer tests other than the learner). Authentic
Authenticity of the assessment Not very authentic. Authentic
Sufficiency Can be sufficient to assess most abilities except psychomotor abilities, interpersonal skills and the like. very sufficient.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s