- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
9.0 Page Quality Rating: Important Considerations
At first, PQ rating may seem difficult. There are several aspects of the page and the website to look at and think about. This type of rating takes practice. Rereading sections of these guidelines and thinking about the examples may help when you encounter difficult rating tasks.
Important: Do not struggle with each PQ rating. Please give your best rating and move on. If you are having trouble deciding between two ratings, please use the lower rating. If you are torn between three ratings, choose the one in the middle.
Do not consider the country or location of the page or website for PQ rating. For example, English (US) raters should use the same PQ standards when rating pages from other English language websites (UK websites, Canadian websites, etc.) as they use when rating pages from U.S. websites. In other words, English (US) raters should not lower the PQ rating because the page location (UK, Canada) does not match the task location.
These guidelines are specific to “regular” webpages. Occasionally, you may be asked to rate a landing page which is not a webpage. For example, you may be asked to rate a PDF file, a PNG or JPEG image file, etc. When the landing page of the URL is not a webpage, some of the considerations in these guidelines may not apply. In this case, please use your judgment.
Finally, this Page Quality Rating Guideline does not completely cover every aspect of page quality. If you find pages which you truly believe to be High or Low quality, please rate them as such, even if the reason is based on something not covered in this document. Please use the comment section to explain your reasoning. As always, we ask you to use your judgment.
Instructions for Rating Page Quality Tasks
The Page Quality task page is broken up into several parts:
Some initial questions about the task landing page.
A “PQ grid” to record your observations about PQ characteristics of the landing page.
The Overall PQ rating slider which records your Overall PQ rating.
A comment box to explain your rating.
Some results to the initial questions will end the task early. If the page is Porn, Foreign Language, or Didn’t Load, you will not fill in the PQ grid or assign an overall rating. Didn’t Load should be used for pages where there is absolutely no content on the page created by the website. There is no MC, SC, or Ads on the page. You can see this Wikipedia article for descriptions of different types of error messages.
Similarly, if you respond that the page is malicious, harmful, deceptive, or lacking in purpose, you will also not fill in the PQ grid or assign an overall rating.
The PQ grid is designed to be your "note pad." It allows you to record your observations about the landing page and the website it belongs to.
The Top Three pq Considerations
The top three most important PQ considerations are:
Quality and quantity of Main Content. Examine the MC carefully. Given the purpose of the page, evaluate the quality and quantity of MC.
Level of expertise, authoritativeness, and trustworthiness (E-A-T) of the page and the website. The level of E-A-T is extremely important for YMYL pages.
Reputation of the website. The reputation of a website is very important when the website demands a high level of trust.
Proprietary and Confidential – Copyright 2015 58
These “top three” considerations will help you rate many or most pages. High or Highest quality ratings must be supported by evidence from at least one of these top three considerations.
E-A-T: Page or Website?
Several PQ characteristics are based on the landing page by itself, such as considerations about MC, SC, and page design, etc. Some PQ characteristics are assessed at the website level, such as contact information, reputation, and website maintenance.
However, E-A-T may be based on either the page alone, or the website, or even both.
Page level checks for E-A-T are important when a website has different authors on different pages. This is the case for article websites or websites like YouTube, which have user-generated content.
Website level checks for E-A-T are important in the following situations:
All content on the website is produced by the same person or organization. An example is a medical website which is produced by a reputable physician group.
The content of the website is produced by different authors or organizations, but the website has very active editorial standards. An example of this is a science journal with very high standards for publication.
The website has an extremely positive reputation from experts in the topic of the website, i.e., the website is acknowledged to be one of the most expert, authoritative, or trustworthy sources on the topic.
