- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
Page Quality Considerations for Specific Types of Pages
Ratings for Encyclopedia Pages
There are many encyclopedia-type websites. Some are highly-respected publications which are standard references. Some are websites with content created and edited by anonymous users with no editorial oversight or fact checking.
In many cases, we may not know the specific author of the encyclopedia article, and must rely on results from reputation research of the encyclopedia when judging the expertise of an article. High and Highest quality ratings can only be used for encyclopedias with very good reputations for accuracy and expertise.
As a rater, you will frequently encounter Wikipedia pages. In general, Wikipedia does have a good reputation. Wikipedia is a very popular resource and is generally valued for accuracy. However, there is no single author or organization that vouches for the accuracy of Wikipedia articles. Individual Wikipedia articles should be evaluated on the basis of page level checks because the quality of pages on Wikipedia varies.
A Wikipedia article with a lot of detailed, information -rich MC, and external references can usually be rated in the High range. Some Wikipedia articles may even be rated as high as Highest, although this rating is usually considered too high for an article on a medical, financial, or legal topic, which requires an extremely high level of expertise. Wikipedia articles can get PQ ratings up to High if they have a lot of high quality MC, are well-researched, and do a good job citing their resources. Otherwise, a PQ rating in the Medium range is often appropriate. Naturally, Wikipedia articles with very little MC should get lower PQ ratings.
Ratings for Pages with Error Messages or No mc
Some pages load with content created by the webmaster, but have an error message or are missing MC.
Pages may lack MC for various reasons. Sometimes, the page is “broken” and the content does not load properly or at all. Sometimes, the content is no longer available and the page displays an error message with this information.
Many websites have a few “broken” or non -functioning pages. This is normal, and those individual non-functioning or broken pages on an otherwise maintained site should be rated Low quality. This is true even if other pages on the website are overall High or Highest quality.
-
Proprietary and Confidential – Copyright 2015
59
Sometimes exploring a website reveals that the individual page is not an isolated example, but rather a symptom of an unmaintained site (or possibly a deceptive or malicious site). When that is the case, the page should be rated Lowest quality.
However, not all pages with error messages are Low or Lowest quality pages. If the purpose of the page is to communicate that content has been removed or is no longer available, and the page does a good job of communicating this message, the overall PQ rating may be higher; it may be Medium or even High. The Page Quality rating will depend on the website level checks and the content of the page.
Here are some examples of “broken” or “error message” pages, as well as pages which are missing MC:
|
Webpage/Type of Content |
|
Characteristics of the Page |
|
|
Discussion |
|
|
|
|
Deliberately created with no |
|
• |
Large amount of prominent Ads and no |
|
|
This is an example of a page with no MC. You |
|
|
|
|
|
|
might think that the MC is “missing” due to a |
|
|
|||
|
MC |
|
|
attempt to help users |
|
|
|
|
|
|
|
|
|
|
problem with this particular page, but in fact, this |
|
|||
|
|
|
|
|
|
|
website has hundreds of pages that look the same |
|
|
|
|
|
|
|
|
|
way—no MC, just Ads. This website shows Ads |
|
|
|
|
|
|
|
|
|
with little or no attempt to help users, and should |
|
|
|
|
|
|
|
|
|
be rated Lowest quality. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
• |
No MC (probably due to a temporary |
|
|
|
|
|
|
Page didn't fully load and |
|
|
technical error) |
|
|
This page has no MC and no error message. It is |
|
|
|
has no MC |
|
• |
Sufficient SC |
|
|
|
||
|
|
|
|
an isolated example of a page with no MC or error |
|
||||
|
|
|
• |
Positive reputation |
|
|
|
||
|
|
|
|
|
message on a website for a reputable newspaper |
|
|||
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
for a town in Michigan. All of the navigation links |
|
||
|
|
|
|
|
|
work, and the page was later fixed. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This is an example of a “custom 404” page, |
|
|
|
|
|
|
|
|
|
alerting users that the URL they are trying to visit |
|
|
|
Error page with custom 404 |
|
• |
Nothing wrong, but nothing special |
|
|
no longer exists. Some websites do a nice job of |
|
|
|
|
|
|
alerting users about a problem and providing |
|
|
|||
|
message 1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
helpful tips. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This page is on a well-known merchant website |
|
|
|
|
|
|
|
|
|
with a good reputation. However, this particular |
|
|
|
|
|
|
|
|
|
page displays the bare minimum of content |
|
|
|
|
|
|
|
|
|
needed to explain the problem to users, and the |
|
|
|
|
|
|
|
|
|
only helpful content is a link to the homepage. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
• |
A satisfying amount of high quality MC for |
|
|
|
|
|
|
Error page with custom 404 |
|
|
its purpose |
|
|
This is an example of a “custom 404” page, |
|
|
|
|
• |
Helpful SC which improves the user |
|
|
|
|
||
|
message 2 |
|
|
|
|
|
|||
|
|
|
experience |
|
|
alerting users that the URL they are trying to visit |
|
||
|
|
|
|
|
|
|
|||
|
|
|
• |
Positive reputation |
|
|
no longer exists. This website does a nice job of |
|
|
|
|
|
|
|
|
|
explaining the issue and providing helpful tips, |
|
|
|
|
|
|
|
|
|
including a search box. |
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
Proprietary and Confidential – Copyright 2015 |
60 |
|
|||
|
Webpage/Type of Content |
|
Characteristics of the Page |
|
|
Discussion |
|
|
|
|
|
|
|
|
|
This is an example of a “custom 404” page. |
|
|
|
|
• |
A satisfying amount of high quality MC for |
|
|
These pages are designed to alert users that the |
|
|
Error page with custom 404 |
|
|
its purpose |
|
|
URL they are trying to visit no longer exists. The |
|
|
|
• |
Helpful SC which improves the user |
|
|
MC of this page is the cartoon, the caption, and |
|
|
|
message 3 |
|
|
|
|
|||
|
|
|
experience |
|
|
the search functionality, which is specific to the |
|
|
|
|
|
|
|
|
|
||
|
|
|
• |
Very positive reputation |
|
|
content of the website. It is clear that time, effort, |
|
|
|
|
|
|
|
|
and talent was involved in the creation of the MC. |
|
|
|
|
|
|
|
|
This publication has a very positive reputation and |
|
|
|
|
|
|
|
|
is specifically known for its cartoons, which allows |
|
|
|
|
|
|
|
|
us to go as high as High+ to Highest. |
|
|
|
|
|
|
|
|
|
|
