- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
14.4.3 English Language Results
The following rating guidance is for raters in non-English locales. You may stop reading this section if the language spoken in your locale is English!
Your Needs Met ratings need to reflect how helpful the result is for users in your locale.
We know that you can read English (you are reading this document!), but you should only give high Needs Met ratings to English results if users in your locale would expect or want them. Unless requested by the query, English results should be considered useless if most users in the locale can't read them.
Here are some examples using Korean (KR) as the task language. In Korea, most users don’t read English. Unless the query “asks for” English results, we will consider them unhelpful or even useless (FailsM):
|
Query and User Intent |
Result Block and LP, Needs Met Rating |
|
Explanation |
|
|
|
|
This Korean language Wikipedia |
|
|
|
|
|
page on Barack Obama is very |
|
|
|
Query: [버락 오바마] |
|
helpful for Korean (KR) users. |
|
|
|
User Location: Seoul, |
|
|
|
|
|
|
|
|
|
|
|
Korea |
|
|
|
|
|
User Intent: 버락 오바마 |
|
|
This English Wikipedia page |
|
|
|
|
about Barack Obama has similar |
|
|
|
is Barack Obama in |
|
|
|
|
|
|
|
content to the Korean Wikipedia |
|
|
|
Korean, so the user |
|
|
|
|
|
|
|
page. Although there are some |
|
|
|
wants to find information |
|
|
|
|
|
|
|
helpful images on the page, very |
|
|
|
about Barack Obama. |
|
|
|
|
|
|
|
few users would be able to read |
|
|
|
|
|
|
|
|
|
|
|
|
this page. |
|
|
|
|
|
Note: this query does not “ask |
|
|
|
|
|
|
|
|
|
|
|
for” English language results. |
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 125
|
Query and User Intent |
Result Block and LP, Needs Met Rating |
|
Explanation |
|
|
|
|
|
Although the query was typed in |
|
|
|
|
|
English, most Korean users |
|
|
|
|
|
would expect to see Korean |
|
|
|
|
|
language reviews or Korean |
|
|
Query: [titanic 1997] |
|
|
stores for purchasing the movie. |
|
|
|
|
|
|
|
|
User Location: Seoul, |
|
|
This Korean result on a Korean |
|
|
|
|
|
||
|
|
|
website is very helpful. |
|
|
|
Korea |
|
|
|
|
|
User Intent: The user |
|
|
This English language landing |
|
|
|
|
page has helpful content for |
|
|
|
wants to purchase a |
|
|
|
|
|
|
|
users who can read English. |
|
|
|
DVD or find information |
|
|
|
|
|
|
|
Although there are some helpful |
|
|
|
about the movie “Titanic,” |
|
|
|
|
|
|
|
images on the page, very few |
|
|
|
released in 1997. |
|
|
|
|
|
|
|
users would be able to read this |
|
|
|
|
|
|
|
|
|
|
|
|
page. |
|
|
|
|
|
Note: this query does not “ask |
|
|
|
|
|
|
|
|
|
|
|
for” English language results. |
|
|
|
|
|
|
|
|
|
|
|
Although the query was typed in |
|
|
|
|
|
English, most Korean users |
|
|
|
|
|
would expect to see the Korean |
|
|
Query: [samsung tablet] |
|
|
page on the Samsung website. |
|
|
|
|
|
|
|
|
User Location: Seoul, |
|
This Korean result is very helpful. |
|
|
|
|
|
|
|
|
|
Korea |
|
|
|
|
|
User Intent: The user |
|
|
|
|
|
|
|
This English page on the website |
|
|
|
wants to purchase, find |
|
|
of Samsung, the company that |
|
|
information, or go to the |
|
|
makes this tablet, has helpful |
|
|
Galaxy tablet page on |
|
|
content. Although there are |
|
|
the Samsung website. |
|
|
some helpful images on the |
|
|
|
|
|
page, very few users would be |
|
|
|
|
|
able to read this page. |
|
|
|
|
|
Note: this query does not “ask |
|
|
|
|
|
|
|
|
|
|
|
for” English language results. |
|
|
|
|
|
|
|
In some locales, English is one of the official languages or a commonly spoken language. In these locales, English websites are easy to use and could be helpful, depending on the query.
For example, the Singapore government recognizes four official languages: English, Malay, Chinese, and Tamil, but English is the first and most dominant language in Singapore.
Proprietary and Confidential – Copyright 2015 126
|
Query and User Intent |
Result Block and LP, Needs Met Rating |
Explanation |
|
|
|
|
This Wikipedia page in English |
|
|
|
|
about Obama would be very |
|
|
Query: [barack obama] |
|
helpful to users in Singapore. |
|
|
|
|
|
|
|
User Location: |
|
|
|
|
Singapore, Singapore |
|
|
|
|
|
|
|
|
|
User Intent: Find |
|
|
|
|
information about |
|
This Wikipedia page in |
|
|
Barack Obama. |
|
Chinese about Obama would |
|
|
|
|
also be very helpful to users in |
|
|
|
|
Singapore. |
|
|
|
|
|
|
|
|
|
|
|
