Ashley Chiasson, M.Ed

Some of my first freelance roles were writing assessment items for various training materials. Namely, I wrote new driver training tests for each and every single state. This was a very profitable endeavour, and assessment item writing comes pretty easy to me, but you multiple each state by 100 (or more) and tell me how sore your fingers are after typing out all of those assessment items…that being said, I do still hold a place in my heart for assessment item writing.

What is this ‘assessment item’ you speak of?

Well. An assessment item is any item, able to be measured, which tests what (or whether) your learner has learned the material you have presented to them. Basically, each question in a quiz/test/exam/knowledge check, etc.

Most of us distinctly remember hand-writing detailed responses to long answer essay questions in high school…or filling out bubble sheet upon bubble sheet upon bubble sheet in our undergraduate degrees (at least if you majored in Psychology). Multiple choice, true/false, drag and drop, essays, long answer, short answer, fill-in-the-blanks – these are all types of assessment items (but certainly not the whole gamut).

Assessment items have different bits.

Assessment items consist of several variables: The stem (question) and the answer (either provided or needing to be written out) – in the case of multiple choice questions, there are also distractors .

In writing the new driver tests, I considered an assessment item to be the Stem, the Answer, and three Distractors. The stem would be the question (e.g. You arrive at a flashing red light. What does this mean?), the answer (e.g. you must come to a complete stop before proceeding when it is safe to do so), and three distractors (e.g. you must come to a complete stop and wait for the light to turn green before proceeding, you may proceed – slowly, you may only make a left hand turn).

There is much to consider when writing assessment items.

In the case of the new driver tests, the only real consideration was to be cognizant of the target audience (14-22 year olds) and write in a language for which they would understand, and to develop assessment items based on the individual state driver’s handbook.

However, I also have experience developing assessment items for college-level text books. For these items, I had to consider the content (the relevant chapter of the textbook) and write meaningful assessment items to test the learner, but I also had to consider the learning objectives for that chapter (to ensure I was assessing the learner in order to meet the learning objectives) and I had to consider Bloom levels. I previously wrote about Bloom’s Taxonomy here, and in relation to assessment item writing, there was typically a requirement to test to higher-levels of Bloom’s Taxonomy. No good college-level assessment is comprised entirely of recall (in my honest opinion).

But…how do I know I’m writing high-quality assessment items? Don’t panic – Here are some resources:

20+ Tips for Writing Great Quiz Questions and Response Options

Improving your Quizzes with Immediate Feedback

Constructing a Bloom’s Taxonomy Assessment

Assessment and Bloom’s Taxonomy

Best Practices: 30 Tips for Creating Quiz Questions

The Anatomy of Great e-Learning Quiz Questions

This week’s term was prompted by a thread I read recently about acronyms. Don’t see the connection? I’ll explain it to you in a few – don’t worry!

Target Audience

Your target audience is anyone who will be participating in the e-learning program you are developing. This is a very small detail, usually discussed in a client’s Statement of Work or a Design Document (or sometimes not at all), but it makes a huge impact on the development of your training materials.

For example, the training materials you develop for a group of 18-21 year old students will likely be different than those developed for a group of 50-60 year old women. For the 18-21 year olds, you might use a more colloquial manner of text or you might use a more interactive, game-based interface. For the 50-60 year old women, you would likely recognize their prior learning experiences and explain the content in a more professional manner.

It is essential to understand and consider your target audience in order to create effective learning experiences!

I’m a huge supporter of conducting an instructional analysis, and while not all clients will see the benefits of paying for anything indicative of an ‘analysis’ phase (many clients may feel as though they’ve conducted this portion), it’s still important as an Instructional Designer to ensure you’re conducting a brief instructional analysis – even if it’s just in your head. If the target audience hasn’t been defined by the client, ASK!

Alright. So how does this connect with acronyms?

Regardless of your target audience  you need to avoid use of industry jargon whenever possible. You need to take yourself outside of your industry body and put yourself in the shoes of your target audience. Do they know what TNA means? Because to you, it’s a Training Needs Analysis, but to them it could be an offensive term used to describe several body parts or a popular retail store.

When using acronyms, ensure you are spelling out the first instance of each acronym; this will ensure that your target audience does not become lost in a sea of acronyms, leaving the training more confused than anything. Another suggestion, in acronym or jargon heavy training materials, would be to include a glossary of acronyms or terms; this will provide your target audience with a resource to consult if ever they find themselves confused.

I first came across the concept of adaptive learning when working at my last corporate gig. Someone had a pie-in-the-sky type of idea for moving into adjacent markets, and adaptive learning would give us a HUGE competitive edge. I say pie-in-the-sky because, at the time, the organization could certainly not fund something so new to us. This past weekend, I came across the term again – this time in the context of its increasing popularity. If it’s becoming more popular, I figured we should probably talk about and get to know the term a little bit more.

What is Adaptive Learning?

Adaptive learning is an educational approach which uses technology to create individualized learning experiences. Computer programs, which work diligently to collect data, adapt the material presented based on the user’s learning needs (which are dictated by their responses and interactions with the computer program).

Sounds creepy? Yeah, a little. I sort of liken adaptive learning to the way in which advertisements on websites are adapted based on your browsing history. While it might be creepy, it’s also really cool!

Why is Adaptive Learning Cool?

Adaptive learning is pretty neat, in my opinion, because it allows students to step outside of the traditional face-to-face classroom environments, where teachers are often times over-saturated with the amount of students per class, leading to very generalized teaching (in most cases). Teachers have it tough! And it can be incredibly difficult for them to veer from traditional teaching methods to make learning an individualized experience for each and every student.

Many teacher friends of mine who have taught in the K-12 school system are incredibly frustrated; because of the ‘no child left behind’ concept, kids are being pushed through, regardless of which grade level they’re functioning at. This results in lower-level learners struggling further with the curriculum. Another problem is that it’s getting to the point where teachers are becoming responsible for developing Individualized Education Plans for more than half of their students per class – Doing this is understandable in that the education system wants to cater (as much as possible) to student’s needs, but it also takes a lot of time on the individual teacher’s side of things, which may lead to increased rate of teacher burn-out. Frustrating, no?!

Adaptive learning may be a solution to such a problem, saving teacher resources and frustration!

The only questionable aspect of adaptive learning that I have found is that implementation may be incredibly costly, so institutions really need to recognize the need and the ultimate cost savings on human resources (and ultimately increase in return on investment), before it will ever gain mainstream traction.

Examples, please!

There’s really only one key example that I want to share – Knewton is a technological platform designed to personalize “digital courses so every student is engaged and no one slips through the cracks.” Using analytics, learning gaps are predicted and the presentation of educational materials are adapted to satisfy these learning gaps. HOW COOL IS THAT?!

A more rudimentary example could be a choose your own adventure book; while not necessarily ‘learned’, the outcome is dependent  on (or predicted by) the reader’s choices. Who didn’t enjoy a good choose your own adventure book? Come on!