Support for Multi-Select Questions

Multi-select questions are now live in Problem-Attic. There is support for them in many places, including the editor, Format tab, and scoring app. You can also export multi-select questions to most LMSs, including Schoology and Canvas, and to a Google Quiz.

Please read the following to learn where the questions are in the Problem-Attic database, how they are scored, and certain limitations with questions that have subparts.

Note: multi-select questions require a school subscription if you want to score them automatically or export them to an LMS. You can try most of these options in the Play Area. You do not need a subscription if you just want to include multi-select questions in a document that you’re making or change the labeling of answer choices.

Selection and Formatting

You can find multi-select questions throughout the Problem-Attic database, but most of them are in these three places:

• State Assessments. The best sources are the last few years’ of released tests for Alaska, Louisiana, Missouri, Mississippi and Ohio.

• National Assessments. The two testing consortia, PARCC and SBAC, have released hundreds of multi-select questions. You’ll find more from Illustrative Mathematics (IM) and Achieve the Core (ATC).

• Pre-Alg to Pre-Calc. This is a bank of original questions for middle and high school math. Multi-select is used for many of the process skills, like understanding real numbers and applying algebraic properties.

As of this writing, there are about 3000 “live” multi-select questions in Problem-Attic. We’re quickly adding more, all original, to the Mathematics Meta-Framework and a brand-new database for reading.

You can spot multi-select questions when you’re browsing the database, because answer choices are labeled with blue circles. You select the questions the same as always, by clicking the Add button or double-clicking on the thumbnail.

This screenshot shows two multi-select questions from PARCC:

When making a PDF document, you can change how the answer choices are labeled. The default is to use the same blue circles as on the thumbnails. To make a change, click the Format tab, then expand the Multiple-Choice panel. You’ll see the labeling option for multi-select toward the bottom.

For the label, there are four types of circles and squares that are intended for checkmarks. You may prefer these types of labels if you’re giving students a printed document and you’re not going to do any automated scoring.

The option called “match multiple-choice” may be best if you’re giving students a PDF document (either printed or electronic), but they’re going to mark their answers on a separate bubble sheet, or they’re going to use Problem-Attic’s online answer sheet. Typically the multiple-choice label will include an upper- or lowercase letter, but the exact label style is up to you.

Online tests, export to an LMS

When you deliver multi-select questions to students through an online test, the labeling is done by Problem-Attic. The label style is always a checkbox, regardless of what you choose on the Format tab. Here is an example:

Students click the answer choices for multi-select just as they do for multiple-choice.

If you use Problem-Attic to export a quiz to your LMS, multi-select questions will show up in the other program with checkboxes. Here are screenshots showing an exported question in Schoology and Canvas.

Please note that the scoring of multi-select in another program may be different than how Problem-Attic does it, because it may award partial credit.

How to write a multi-select question

In the Problem-Attic editor, multi-select questions are written in a very similar way as multiple-choice. Here are the usual steps.

1. On the Arrange tab, click Write New Problem.

   

2. In the editor, type the question, and press Enter to start a new line.

3. Type the answer choices, and press Enter after each one.

   

4. Highlight the block of choices.

5. In the editor toolbar, click the Lists icon and choose “Multi-Select Block”. Your choices will now be shown with a light gray background and temporary labels.

   

6. Right-click each of the correct choices and choose “Mark correct answer” from the pop-up menu.

   

7. Save your new problem.

Later, when you click Make PDF, you’ll see that your multi-select question gets whatever label you specified on the Format tab. If you turn it into an online test, you’ll see the labels are turned into checkboxes.

Scoring, limitations

Problem-Attic will score multi-select questions automatically if you deliver them as an online test or give students the link to an answer sheet. (The answer sheet can go along with a slideshow or PDF document.)

The scoring of multi-select questions necessarily involves an algorithm. In Problem-Attic, a question is marked correct or incorrect. So the algorithm is simple: students must select all correct choices. If they do not select all of them, or if they select an additional non-correct choice, then Problem-Attic will consider the answer to be wrong. If you like, you can override this or give partial credit through manual scoring, which is explained here.

Currently, there is one important limitation with Problem-Attic’s automatic scoring. It cannot score questions that have more than one part. These are common with PARCC and SBAC, which are often written with Part A, Part B, etc. If you want automatic scoring, you should include in your document questions which have a single part or which prompt the student for just one answer. This limitation, by the way, is for all types of questions, not just multi-select.

Note: If you want to use automated scoring with a multi-part question from PARCC, SBAC or a state assessment, you do have the option of using Problem-Attic’s editor and breaking up the parts into separate questions.