Measuring people proficiency is science

It needs to be carefully crafted to see what you want to see by the end of the assessment. In the real world, many of us give our opinion about an assessment at superficial level ad most of the time equate assessment to the capability of an assessment engine or the type of questioning type like MCQ (Multiple Choice Question).

There are many scholarly articles available on how to design an assessment, however, we will be happily ignoring those by saying they are academic just because we don’t know or we don’t want to know.

In this series of blog, we will explore some of the key components of assessment design, assessment type and little on cognitive science behind those components since we are dealing with probing of the brain. When to use which type of assessment and what benefit you will reap.

You might be thinking that who are we to write about designing assessment, in the last 5 years we have evaluated more than fifty thousand assessment results of adult learners for an MNC.

We also offer assessments for different needs (To make a hiring decision or certifying employees) of corporate.

While delivering this we have encountered many scenarios, with that experience and insights we are writing this blog series to provide our perspective and knowledge to the rest of the world.

The stigma of assessment is we always think that it leads to a PASS or FAIL state. We never take assessment results as feedback on our performance. We are conditioned from early childhood that FAIL means stopping the progress.

It’s absolutely needs of the hour that we need a change of perspective to see assessment result as a feedback and it should be used for our progress.

The first and foremost point to be kept in mind that when you design an assessment keep your frame of mind that you are genuinely interested in measuring people's proficiency at what level they know in a given area of expertise.

Another trap that we are in is that, we think performance in terms of MARK. We tend to equate higher the mark means better the performance. This is absolutely a MYTH.

So, what determined the quality of an assessment?

Assessment quality is determined based on whether it’s VALID or how RELIABLE it is?

What determines the validity of an assessment?

Validity means whether the assessment is testing accurately what it intended to test.

How do we ensure the reliability of an assessment?

Reliability is all about how good the assessment yield consistent result when it is administered at a different point in time.

What is the backbone of an assessment? Or the fuel of the assessment OR the factor which decides the fate of an assessment. It’s nothing but,

The QUESTIONS.

Yes, Questions are the critical element of an assessment and through which only you can probe the brain of an individual to respond with responses that indicate what the brain is capable of. It is the key through which you will unlock the brain to see someone's capability or potential performance.

So, you have to learn how to ask a perfect question to access different parts of the brain.

What type of question you should ask when you want to test someone's conceptual understanding?

What type of question you should ask when you want to test someone able to perform?

What type of question you should ask when you want to test someone's remembrance of facts and syntax?

You have to be precise while framing a question and use keywords that will activate a particular area of the brain of the test taker to provide an appropriate response. It’s also important that when you provide options to choose, which also needs to be crafted carefully to kindle the brain of the test taker.

Another common mistake people do when they decide the options for the question. The thought process would be to have one correct answer and rest all incorrect. Always we tempt to focus on ensuring that we are providing a correct option but least bothered about the rest of the options.

“Every option has a purpose or a role to play in the question.”

How good the options are squeezing the test taker's brain to decide the correct response is determined by how good the rest of the options. There are two key concepts you should be aware of and apply when you are writing questions are – Plausible and Distractor.

What is Plausible?

Plausible means the options are possible enough to occur but not for the given question.

What is Distractor?

How good the option distracts the test taker while choosing the correct response. Are you ready to author a question for an assessment?

We will see different types of assessments, what kind of analysis we should perform to ensure validity and reliability of an assessment, how we can we design and when to leverage what type of assessment in our next blog Know Your Assessments