27.0 Notes about Using the Needs Met Rating Interface

Result blocks are sometimes slow to load: Before assigning any ratings, please scan the task page to make sure that all result blocks with a Needs Met rating slider have loaded. Please note that result blocks may be slow to load and/or that the task page may need reloading if there are empty result blocks. If the result blocks fail to load after refreshing a few times, please release the task.

Sometimes a side will not have any result blocks: If one side displays the message “This side did not generate any results,” please do not release the task.

Do not refresh the task page after assigning ratings: If you refresh the page after assigning ratings, you will lose them and they will have to be entered again.

28.0 Using the “Report a Problem / Release this Task” Button

You will report problems and release tasks using the “Report a Problem / Release this Task” button on the rating interface. This is what the “Report a Problem / Release this Task” button looks like:

Clicking the red button displays the following reporting and release options:

Here are some examples for the “lack expertise” and “technical problem” release categories.

Lack expertise: These queries require specialized knowledge and may be difficult for some raters.

Proprietary and Confidential – Copyright 2015 156

Query	General Query Topic
[cortisol secretion shows a blank pattern of secretion]	Scientific query
[minecraft gun and vehicles 1.1 mod installer]	Gaming query
[python pandas index like matrix]	Technical query

Technical problem: These result blocks have question marks and other characters where text should be.

Note: Based on the number and/or type of tasks that you release, you may be asked to provide additional details about the reason for some of the releases.

29.0 Reporting Results with Duplicate Landing Pages

In addition to assigning Needs Met ratings to each result block, we would like you to tell us whether any results are duplicates (dupes).

29.1 Pre-Identified Duplicates

Some duplicate results are automatically detected and will be pre-identified for you. They will be highlighted by a grey or green background, and annotated by either “Same as...” or “Dupe of...” text right below the slider and above the result block.

Proprietary and Confidential – Copyright 2015 157

This is what these pre-identified dupes look like:

Same as… pre-identified duplicate result

Dupe of… pre-identified duplicate result

Please note:

• Results annotated by “Dupe of...” may be given different ratings/flags/comments.

• Results annotated by “Same as…” may not be given different ratings/flags/comments. Their Needs Met and E-A-T ratings, flags, and comments will be automatically transferred to each other.

• You cannot un-check dupes that have been automatically detected and pre-identified.