- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
2.7.2 Sources of Reputation Information
Look for information written by a person, not statistics or other machine-compiled information. News articles, Wikipedia articles, blog posts, magazine articles, forum discussions, and ratings from independent organizations can all be sources of reputation information. Look for independent, credible sources of information.
Sometimes, you will find information about a website which is not related to its reputation. For example, pages with information about Internet traffic to the website do not provide evidence of positive or negative reputation. You can ignore this information since it's not helpful for Page Quality rating.
2.7.3 Customer Reviews of Stores/Businesses
Customer reviews can be helpful for assessing the reputation of a store or business. However, you should interpret these reviews with care, particularly if there are only a few. Be skeptical of both positive and negative user reviews. Anyone can write them, including the creator of the website or someone the store or business hires for this purpose. See here for a New York Times article on fake reviews and here for a Guardian article on fake reviews.
Proprietary and Confidential – Copyright 2015 16
When interpreting customer reviews, try to find as many as possible. Any store or website can get a few negative reviews. This is completely normal and expected. Large stores and companies have thousands of reviews and most receive some negative ones.
It is also important to read the reviews because the content of the reviews matter, not just the number. Credible, convincing reports of fraud and financial wrongdoing is evidence of extremely negative reputation. A single encounter with a rude clerk or the delayed receipt of a single package should not be considered negative reputation information. Please use your judgment.
2.7.4 How to Search for Reputation Information
Here is how to research the reputation of the website:
Identify the “homepage” of the website. For example, for the IBM website, ibm.com is the homepage.
Using ibm.com as an example, try one or more of the following searches on Google:
[ibm -site:ibm.com]: A search for IBM which excludes pages on ibm.com.
[“ibm.com” -site:ibm.com]: A search for “ibm.com” which excludes pages on ibm.com.
[ibm reviews -site:ibm.com] A search for reviews of IBM which excludes pages on ibm.com.
[“ibm.com” reviews -site:ibm.com]: A search for reviews of “ibm.com” which excludes pages on ibm.com.
Look for articles, reviews, forum posts, discussions, etc. written by people about the website. For businesses, there are many sources of reputation information and reviews. Here are some examples: Yelp, Better Business Bureau (a nonprofit organization that focuses on the trustworthiness of businesses and charities), Amazon, and Google Product Search. You can try searching on specific sites to find reviews. For example, you can try [ibm site:bbb.org] or [“ibm.com” site:bbb.org].
Note: You will sometimes find high ratings on the Better Business Bureau (BBB) website because there is very little data on the business, not because the business has a positive reputation. However, very low ratings on BBB are usually the result of multiple unresolved complaints. Please consider very low ratings on the BBB site to be evidence for a negative reputation.
See if there is a Wikipedia article or news article from a well-known news site. Wikipedia can be a good source of information about companies and organizations. For example, try [ibm site:en.wikipedia.org] or [“ibm.com” site:en.wikipedia.org]. News articles and Wikipedia articles can help you learn about a company and may include information specific to reputation, such as awards and other forms of recognition. Note that some Wikipedia articles include a message warning users that there are disagreements on some of the content, or that the content may be outdated. This may be an indication that additional research is necessary.
Here are some examples of reputation information:
|
Website |
|
Reputation Information About the Site |
|
Description |
|
|
|
|
Search results for [annualcreditreport.com |
|
Positive reputation information: Users in the U.S. can |
|
|
|
|
-site:annualcreditreport.com] |
|
|
|
|
|
|
|
|
obtain free credit reports on this website by providing their |
|
|
annualcreditreport.com |
|
Wikipedia article about |
|
Social Security Number. Notice that this Wikipedia article |
|
|
|
annualcreditreport.com |
|
tells us that “AnnualCreditReport.com is the only federally |
|
|
|
|
|
|
|
||
|
|
|
|
|
mandated and authorized source for obtaining a free credit |
|
|
|
|
Wall Street Journal article about |
|
report.” |
|
|
|
|
annualcreditreport.com |
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 17
|
Website |
|
Reputation Information About the Site |
|
Description |
|
|
|
|
|
Search results for [clevelandclinic.org] |
|
Positive reputation information: According to Wikipedia, |
|
|
|
|
|
Wikipedia article about clevelandclinic.org |
|
the Cleveland Clinic “is currently regarded as one of the top |
|
|
|
clevelandclinic.org |
|
|
4 hospitals in the United States as rated by U.S. News & |
|
||
|
|
|
|
|
World Report,” which you will also find in the article on the |
|
|
|
|
|
|
|
|
|
|
|
|
|
US News & World Report article about the |
|
best hospitals in the U.S. Users can trust medical |
|
|
|
|
|
best hospitals in the U.S. |
|
information on this website. |
|
|
|
|
|
|
|
|
|
|
|
|
|
Search results for [csmonitor.com - |
|
Positive reputation information: Notice the highlighted |
|
|
|
|
|
|
section in the Wikipedia article about The Christian Science |
|
||
|
|
|
site:csmonitor.com] |
|
|
||
|
csmonitor.com |
|
|
Monitor newspaper, which tells us that the newspaper has |
|
||
|
|
|
|
|
|
||
|
|
Wikipedia article about The Christian |
|
won seven Pulitzer Prize awards. From this information, we |
|
||
|
|
|
|
|
|||
|
|
|
|
can infer that the csmonitor.com website has a positive |
|
||
|
|
|
Science Monitor |
|
|
||
|
|
|
|
reputation. |
|
||
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Search results for [llbean.com reviews – |
|
|
|
|
|
|
|
site:llbean.com] |
|
|
|
|
|
|
|
LL Bean positive reviews 1 |
|
Positive reputation information: From the numerous |
|
|
|
llbean.com |
|
|
positive reviews, we can infer that llbean.com has a positive |
|
||
|
|
|
|
|
|
||
|
|
|
LL Bean positive reviews 2 |
|
reputation. |
|
|
|
|
|
|
|
|
||
|
|
|
LL Bean positive reviews 3 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Positive reputation information: We learn in the Wikipedia |
|
|
|
|
|
|
|
article that “Kernel.org is a main repository of source code |
|
|
|
|
Search results for [kernel.org – |
|
for the Linux kernel, the base of the popular Linux operating |
|
|
|
kernel.org |
|
site:kernel.org] |
|
system. It makes all versions of the source code available |
|
|
|
|
|
|
|
to all users. It also hosts various other projects, like Google |
|
|
|
|
|
|
|
|
|
|
|
|
|
Wikipedia article about kernel.org |
|
Android. The main purpose of the site is to host a repository |
|
|
|
|
|
|
|
|
for Linux kernel developers and maintainers of Linux |
|
|
|
|
|
|
|
distributions.” |
|
|
|
|
|
|
|
|
|
|
|
|
Search to find reputation information |
|
|
|
|
|
|
|
Search to find reviews |
|
Extremely negative reputation information: This business |
|
|
|
|
|
|
|
|
|
|
|
Site selling children’s |
|
BBB negative review |
has a BBB rating of F. There is a news article about |
|
||
|
|
financial fraud. There are many reviews on websites |
|
||||
|
jungle gym |
|
|
|
|
||
|
|
TrustLink negative reviews |
describing users sending money and not receiving anything. |
|
|||
|
|
|
|
||||
|
|
|
|
|
|
||
|
|
|
Negative news article |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Search to find reputation information |
|
|
|
|
|
Site selling products |
|
BBB page |
|
Extremely negative/malicious reputation information: |
|
|
|
related to eyewear |
|
|
|
|
This website engaged in criminal behavior such as |
|
|
|
|
Wikipedia article |
|
physically threatening users. |
|
|
|
|
|
New York Times article |
|
|
|
|
|
|
|
|
|
|
|
|
Proprietary and Confidential – Copyright 2015 18
|
Website |
|
Reputation Information About the Site |
Description |
|
|
|
|
Search to find scams related this |
|
|
|
|
|
organization |
|
|
|
Organization serving |
|
Negative review 1 |
Extremely negative reputation information: There are |
|
|
|
|
many detailed negative articles on news sites and charity |
|
|
|
the hospitalized |
|
|
|
|
|
|
Negative review 2 |
watchdog sites about this organization describing fraud and |
|
|
|
veteran community |
|
|
||
|
|
|
financial mishandling. |
|
|
|
|
|
Negative review 3 |
|
|
|
|
|
|
|
|
|
|
|
Negative review 4 |
|
|
|
|
|
|
|
|
