- •Releasing Tasks
- •Introduction to Page Quality Rating
- •Understanding Webpages and Websites
- •Important Definitions
- •What is the Purpose of a Webpage?
- •Your Money or Your Life (ymyl) Pages
- •Understanding Webpage Content
- •2.4.1 Identifying the Main Content (mc)
- •2.4.3 Identifying Advertisements/Monetization (Ads)
- •2.4.2 Identifying the Supplementary Content (sc)
- •2.4.4 Summary of the Parts of the Page
- •Understanding the Website
- •2.5.1 Finding the Homepage
- •2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page
- •2.5.3 Finding About Us, Contact Information, and Customer Service Information
- •Website Maintenance
- •Website Reputation
- •2.7.1 Reputation Research
- •2.7.2 Sources of Reputation Information
- •2.7.3 Customer Reviews of Stores/Businesses
- •2.7.4 How to Search for Reputation Information
- •2.7.5 What to Do When You Find No Reputation Information
- •Overall Page Quality Rating Scale
- •High Quality Pages
- •Characteristics of High Quality Pages
- •A Satisfying Amount of High Quality Main Content
- •A High Level of Expertise/Authoritativeness/Trustworthiness (e-a-t)
- •Positive Reputation
- •Helpful Supplementary Content
- •Functional Page Design
- •A Satisfying Amount of Website Information
- •A Well Cared For and Maintained Website
- •Examples of High Quality Pages
- •Highest Quality Pages
- •Very High Quality mc
- •Very High Level of e-a-t
- •Very Positive Reputation
- •Examples of Highest Quality Pages
- •6.0 Low Quality Pages
- •Low Quality Main Content
- •Unsatisfying Amount of Main Content
- •Lacking Expertise, Authoritativeness, or Trustworthiness (e-a-t)
- •Negative Reputation
- •Characteristics Which May Be Evidence of Low Quality
- •6.5.1 Unhelpful or Distracting Supplementary Content
- •6.5.2 Lacking Supplementary Content
- •6.5.3 Poor Page Design
- •6.5.4 Lacking Care and Maintenance
- •Unsatisfying Amount of Information about the Website
- •Examples of Low Quality Pages
- •7.0 Lowest Quality Pages
- •Harmful or Malicious Pages
- •Lack of Purpose Pages
- •Deceptive Pages
- •7.3.1 Deceptive Page Purpose
- •7.3.2 Deceptive Page Design
- •7.3.3 Sneaky Redirects
- •Lowest Quality Main Content
- •7.4.1 No Main Content
- •7.4.2 “Keyword Stuffed” Main Content
- •7.4.3 Gibberish or Meaningless Main Content
- •7.4.4 Automatically-Generated Main Content
- •7.4.5 Copied Main Content
- •7.4.6 More About Copied Content
- •7.4.7 How to Determine if Content is Copied
- •No Website Information
- •Highly Untrustworthy, Unreliable, Unauthoritative, Inaccurate, or Misleading
- •Abandoned Websites or Spammed Pages on a Website
- •Extremely Negative or Malicious Reputation
- •Examples of Lowest Quality Pages
- •8.0 Medium Quality Pages
- •Examples of Medium Quality Pages
- •9.0 Page Quality Rating: Important Considerations
- •Instructions for Rating Page Quality Tasks
- •The Top Three pq Considerations
- •Page Quality Considerations for Specific Types of Pages
- •Ratings for Encyclopedia Pages
- •Ratings for Pages with Error Messages or No mc
- •Ratings for Forums and q&a pages
- •Page Quality Rating faQs
- •Understanding Mobile Users, Mobile Queries, and Mobile Results
- •Important Rating Definitions and Ideas
- •Understanding the Query
- •Task Location (Locale) and User Location
- •Queries with an Explicit Location
- •Queries with Multiple Meanings
- •Query Meanings Can Change Over Time
- •Understanding User Intent
- •Know and Know Simple Queries
- •Do and Device Action Queries
- •Website Queries
- •Visit-in-Person Queries and User Location
- •Queries with Multiple User Intents
- •Understanding Result Blocks
- •Web Search Result Block Examples
- •Special Content Result Block Examples
- •Device Action Result Block Examples
- •How Device Action Results are Displayed in Rating Tasks
- •Rating Using the Needs Met Scale
- •Rating on Your Phone Issues
- •Rating Result Blocks: Block Content and Landing Pages
- •Fully Meets (FullyM)
- •Examples of Fully Meets (FullyM) Result Blocks
- •Examples of Result Blocks that Cannot be Fully Meets
- •Highly Meets (hm)
- •Examples of Highly Meets (hm) Result Blocks
- •Moderately Meets (mm)
- •Examples of Moderately Meets (mm) Result Blocks
- •Slightly Meets (sm)
- •Examples of Slightly Meets (sm) Result Blocks
- •Fails to Meet (FailsM)
- •Examples of Fails to Meet (FailsM) Result Blocks
- •14.0 Rating Porn, Foreign Language, Didn’t Load, and Hard to Use Results
- •Porn Flag
- •Needs Met Rating for Porn Results
- •Needs Met Rating for Clear Non-Porn Intent Queries
- •Needs Met Rating for Possible Porn Intent Queries
- •Needs Met Rating for Clear Porn Intent Queries
- •Reporting Illegal Images
- •Foreign Language Flag
- •14.4.1 Using the Foreign Language Flag
- •14.4.2 Needs Met Ratings for Foreign Language Results
- •14.4.3 English Language Results
- •Didn’t Load Flag
- •14.5.1 Using the Didn’t Load Flag
- •14.5.2 Needs Met Rating for Didn’t Load Results
- •Hard to Use Flag
- •14.6.1 Using the Hard to Use Flag
- •The Relationship between e-a-t and Needs Met
- •Rating Queries with Multiple Interpretations and Intents
- •Rating Queries with Both Website and Visit-in-Person Intent
- •Specificity of Queries and Landing Pages
- •Needs Met Rating and Freshness
- •Misspelled and Mistyped Queries and Results
- •Misspelled and Mistyped Queries
- •19.2 Name Queries
- •19.3 Spelling Suggestion Result Blocks
- •21.0 Product Queries: Action (Do) vs. Information (Know) Intent
- •22.0 Rating Visit-in-Person Intent Queries
- •26.0 Needs Met Task Page Screenshot
- •27.0 Notes about Using the Needs Met Rating Interface
- •28.0 Using the “Report a Problem / Release this Task” Button
- •29.0 Reporting Results with Duplicate Landing Pages
- •Rater-Identified Duplicates
Understanding Mobile Users, Mobile Queries, and Mobile Results
What do you do on your mobile smartphone?
People rely on their phones for many different tasks in different environments. Users may want to search the web or may want to tell the phone to do something specific. Keep in mind that tasks can be simple or complex, and may take multiple steps to complete. For example, a simple task may be to find the director of a movie. A complex task may be to find a movie’s showtimes nearby, purchase tickets, get directions, and then use the phone’s navigation to go to the theater.
We expect our phones to do a lot. At the same time, phones can be challenging to use, especially compared to a desktop computer or laptop:
Entering data may be cumbersome: typing is difficult on mobile smartphones, and when users speak to their phones instead of typing, voice recognition may not always be accurate.
Small screen sizes make it difficult to use some phone features, apps, and webpages.
Some webpages are difficult to use on a mobile phone. Website navigation can be difficult as menus and navigation links may be small. Webpages may require left-to-right scrolling to read text. Images may not fit on the screen. In addition, many mobile devices cannot access webpages with Flash or other similar features.
Internet connectivity can be slow and inconsistent for mobile users going in and out of networks. App opening, recognition of voice commands, and webpage load times can be very slow on a mobile phone.
Important: Mobile smartphones should make tasks easy, even for mobile users with a small screen device (i.e., size of smartphone, not a tablet). Users want results right away, at that moment, and may not be able to spend a lot of time to find what they are looking for.
In order to do mobile rating tasks, you must have experience using a mobile smartphone, which we’ll also refer to as a mobile phone in these guidelines. If you are not familiar with voice commands, device actions, or phone features, please take some time to experiment on a mobile smartphone. For example, you can try some of these voice commands:
iPhone Siri voice commands
Android voice commands, or say “OK Google” and then “Help” on an Android device
Proprietary and Confidential – Copyright 2015 67
Important Rating Definitions and Ideas
Query: This refers to the word(s) and/or number(s) that a user types or speaks into a mobile phone. In these guidelines, queries have square brackets around them. If a user says “navigate home,” we display: [navigate home]. If a user types “iPhone” in the search box, we display: [iphone].
There are many different types of queries because users ask their phones to do many things, from opening an app to calling a friend to searching the web.
User: The user is the person trying to accomplish something by typing or speaking into a mobile phone with a small screen (i.e., size of a smartphone, not a tablet).
User Intent : When a user types or speaks a query, he or she is trying to accomplish something. We refer to this goal as the user intent.
Locale: All queries have a locale, which is the language and location for the task. Locales are represented by a two-letter country code. For a current list of country codes, click here. We sometimes refer to the locale as the task location.
User Location: This tells us where the user is located.
Search Engine Results Page (SERP): The page a search engine shows after a user enters a query in the search box. The SERP is made up of result blocks.
Result: We will use the word result to refer to the result block and the landing page.
Result Block: This is an individual “block” which appears on the user’s phone in response to the query. The result block may display information in the block itself or contain links, or may do both.
The Landing Page (LP) is the page you see after you click a link in the result block.
Proprietary and Confidential – Copyright 2015 68
Device Actions : Mobile phones and other devices can respond to voice commands to perform many actions, such as setting an alarm or opening an app. This is a specific type of query which we’ll refer to as a Device Action query.
Device Action query: Specific type of query where users ask their phone to perform an action. These are frequently spoken commands used to complete actions which would normally require interaction with the screen or the device's controls.
Device Action result: The phone may respond to a Device Action query by performing an action, such as calling a phone number, etc.
