- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
Needs Met Rating and Freshness
Some queries demand very recent or “fresh” information. Users may be looking for “breaking news,” such as an important event or natural disaster happening right now. Here are different types of queries demanding current/recent results.
|
Type of Query |
|
Example Queries |
|
Explanation |
|
|
|
|
|
|
Assume users need the information right away. |
|
|
“Breaking news” |
|
[tornado], [tsunami] |
|
Imagine someone who needs immediate weather information |
|
|
queries |
|
|
|
||
|
|
|
|
because a big storm is coming. Information about last year's |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
weather would not be helpful. |
|
|
|
|
|
|
|
|
|
|
|
|
|
Assume users are looking for the most recent or current information |
|
|
Recurring event |
|
|
|
about the event. |
|
|
|
|
|
|
|
|
|
queries, such as |
|
[olympics], [american idol], |
For example, if the Olympics are happening right now, users |
|
|
|
elections, sports |
|
[redsox schedule], [tax forms], |
|
||
|
|
searching [olympics] want information about the current Olympics, |
|
|||
|
events, TV shows, |
|
[elections] |
|
||
|
|
not results from years ago. If the next Olympics are a few months |
|
|||
|
conferences, etc. |
|
|
|
||
|
|
|
away, users are probably interested in the |
|
||
|
|
|
|
|
||
|
|
|
|
upcoming Olympics. |
|
|
|
|
|
|
|
|
|
|
Current information |
|
[population of paris], [amount of |
|
Assume users are looking for the most current information, such as |
|
|
|
u.s. debt], [airfare from ny to sfo], |
|
|
||
|
queries |
|
|
prices or airfare. |
|
|
|
|
[next federal holiday] |
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Product queries |
|
[iphone], [toyota camry], |
|
Assume users are looking for information about the most recent |
|
|
|
[windows operating system] |
|
model/version for these product queries. |
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
When a query demands recent content, only pages with current, recent, or updated content should get high Needs Met ratings. For these queries, pages about past events, old product models and prices, outdated information, etc. are not helpful. They should be considered “stale” and given low Needs Met ratings. In some cases, stale results are useless and should be rated FailsM.
For some queries, there may be “newsy” or recent information user intent, as well as more “timeless” information user intent. Users issuing queries for celebrities or politicians may be interested in biographical information, or users may be looking for the latest news or gossip.
Freshness is generally less of a concern for E-A-T rating. “Stale” pages can have high E-A-T ratings. For example, some highly reputable news websites maintain “archival” content. However, unmaintained/abandoned “old” websites or unmaintained and inaccurate/misleading content is a reason for a low E-A-T rating.
Note: The date the page was created may be different from when the content was last updated or modified. When content is updated, the page will sometimes show the date of the update, not the date the page was created. Some websites always show the current date, regardless of when the content was last updated. If you are curious about the content of a page, see here to try the “Wayback Machine” on the Internet Archive. Not all pages are available, but this tool may help you understand how some pages were created and how their content has changed over time.
Proprietary and Confidential – Copyright 2015 141
|
Query, User Location, |
|
|
Result Block and LP, E-A-T Rating, Needs Met Rating |
|
|
Explanation |
|
|
|
|
User Intent |
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
Query: [nasa mars |
|
|
|
|
|
This LP has an article about the |
|
||
|
|
|
|
|
|
“Curiosity” rover landing, |
|
|||
|
landing] |
|
|
|
|
|
|
|||
|
|
|
|
|
|
published on the same day as |
|
|||
|
|
|
|
|
|
|
|
|
||
|
User Location: San |
|
|
|
|
|
the landing, which is the day this |
|
||
|
|
|
|
|
|
example was written. |
|
|||
|
Francisco, California |
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
||
|
User Intent: On the date |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
this example was written, |
|
|
|
|
|
|
|
|
|
|
NASA had just landed |
|
|
|
|
|
This LP has an article about the |
|
||
|
the “Curiosity” rover on |
|
|
|
|
|
|
|||
|
|
|
|
|
|
launching of the “Curiosity” rover, |
|
|||
|
Mars. Most users are |
|
|
|
|
|
|
|||
|
|
|
|
|
|
published nine months before its |
|
|||
|
probably looking for |
|
|
|
|
|
|
|||
|
|
|
|
|
|
landing on Mars. The content on |
|
|||
|
breaking news and |
|
|
|
|
|
|
|||
|
|
|
|
|
|
this page would be considered |
|
|||
|
pictures about the event. |
|
|
|
|
|
|
|||
|
|
|
|
|
|
stale for the query. |
|
|||
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
||
|
Query: [los angeles |
|
|
|
|
|
This LP shows traffic conditions |
|
||
|
traffic] |
|
|
|
|
|
in Los Angeles on August 7, |
|
||
|
|
|
|
|
|
|
|
2012, which is the day this |
|
|
|
User Location: Los |
|
|
|
|
|
example was written. |
|
||
|
Angeles, California |
|
|
|
|
|
|
|
|
|
|
User Intent: Users are |
|
|
|
|
|
|
|
|
|
|
looking for current |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
information about traffic |
|
|
|
|
|
|
|
|
|
|
conditions in Los |
|
|
|
|
|
|
|
|
|
|
Angeles. Even pages |
|
|
|
|
|
This LP shows traffic conditions |
|
||
|
about traffic conditions |
|
|
|
|
|
in Los Angeles on August 6, |
|
||
|
the day before would be |
|
|
|
|
|
2012. It is stale and useless for |
|
||
|
considered stale for the |
|
|
|
|
|
the query even though it is only |
|
||
|
query. |
|
|
|
|
|
one day old. |
|
||
|
|
|
|
|
|
|
||||
|
|
|
|
|
|
|
|
|
||
|
Query: [nobel peace |
|
|
|
|
|
This LP on the New York Times |
|
||
|
prize winner] |
|
|
|
|
|
website has information about |
|
||
|
|
|
|
|
|
|
|
the most recent winner of this |
|
|
|
User Location: San |
|
|
|
|
|
prize, based on the date when |
|
||
|
Diego, California |
|
|
|
|
|
this example was written. |
|
||
|
|
|
|
|
|
|
||||
|
User Intent: Find the |
|
|
|
|
|
|
|
|
|
|
name of the most recent |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
winner of this prize. This |
|
|
|
|
|
|
|
|
|
|
is a Know Simple query. |
|
|
|
|
|
|
|
|
|
|
At the time this example |
|
|
|
|
|
|
|
|
|
|
was originally written, |
|
|
|
|
|
This LP on the BBC website has |
|
||
|
2011 was the most |
|
|
|
|
|
|
|||
|
|
|
|
|
|
information about the 2006 |
|
|||
|
recent year that the |
|
|
|
|
|
|
|||
|
|
|
|
|
|
winner of this prize. This page is |
|
|||
|
Noble Peace Prize was |
|
|
|
|
|
|
|||
|
|
|
|
|
|
stale for the query. |
|
|||
|
awarded. |
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 |
142 |
|
||||
|
Query, User Location, |
|
|
Result Block and LP, E-A-T Rating, Needs Met Rating |
|
|
Explanation |
|
|
|
|
User Intent |
|
|
|
|
|
|
|||
|
|
|
|
|
|
|
|
|
|
|
|
Query: [arthritis] |
|
|
|
|
|
Medline Plus is an authoritative |
|
||
|
User Location: |
|
|
|
|
|
|
|||
|
|
|
|
|
|
website that regularly updates its |
|
|||
|
Philadelphia, |
|
|
|
|
|
content. |
|
||
|
Pennsylvania |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
User Intent: Users are |
|
|
|
|
|
|
|
|
|
|
looking for information |
|
|
|
|
|
|
|
|
|
|
about this disease. |
|
|
|
|
|
|
|
|
|
|
Because there is ongoing |
|
|
|
|
|
|
|
|
|
|
research done on |
|
|
|
|
|
|
|
|
|
|
treatments and cures for |
|
|
|
|
|
This 2006 article would be |
|
||
|
most diseases, users are |
|
|
|
|
|
|
|||
|
probably looking for |
|
|
|
|
|
considered stale for this medical |
|
||
|
authoritative, up-to-date |
|
|
|
|
|
query. |
|
||
|
information on arthritis. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
|
Query: [when was |
|
|
|
|
|
|
|
|
|
|
franklin d roosevelt born] |
|
|
|
|
|
This LP has the information |
|
||
|
|
|
|
|
|
|
|
|
||
|
User Location: Chicago, |
|
|
|
|
|
users are looking for. |
|
||
|
Illinois |
|
|
|
|
|
|
|
|
|
|
User Intent: Users want |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
|
to find this well |
|
|
|
|
|
|
|
|
|
|
understood fact. |
|
|
|
|
|
|
|
|
|
|
|
|
|
We couldn’t find a stale page for this query. |
|
|
|
|
||
|
|
|
|
|
|
|
|
|
|
|
