Duplicate content

What is duplicate content?

Duplicate content is content that appears at more than one URL, either word for word or with only small differences. It can happen within one site, such as the same product at several URLs, or across sites, such as an article republished elsewhere. Search engines usually index one version, the canonical, and filter the rest from results.

Most duplicate content is technical and accidental. The same page can be reached over http and https, with and without www, with and without a trailing slash, with URL parameters for tracking or sorting, through print versions, through faceted navigation, or through several category paths in a shop. A product sold in six colours may have six near-identical pages. To Google, each URL is a separate document until it works out they are the same.

Other duplication is editorial. Location pages that swap only the town name, manufacturer descriptions copied onto hundreds of shops, boilerplate text that outweighs the unique text on each page, and articles syndicated to partner sites all count. Near-duplicates are pages that are mostly the same, such as two category pages showing nearly identical product lists.

When Google finds duplicates, it groups them and picks one URL as canonical, using signals such as the canonical tag, redirects, which URL is linked internally, the sitemap, HTTPS and simple URL patterns. Your canonical tag is a strong hint, not a command; if the other signals disagree, Google may choose a different URL. Search Console’s URL Inspection tool shows both the canonical you declared and the one Google selected.

There is a widespread myth of a “duplicate content penalty”. Google has said for years that duplicate content on its own is not grounds for a penalty and that most of it is not deceptive. The exception is duplication meant to manipulate rankings, such as scraped content republished from other sites without adding value, or many doorway pages with the same text. Those fall under Google’s spam policies.

Why duplicate content matters for SEO

Even without a penalty, duplication costs you. Links and other signals can be split across several URLs instead of building up on one. Google may choose a different version from the one you want, so a parameter URL or an http page appears in results. On large sites, crawling duplicates wastes crawl budget and can delay new pages being found. Near-duplicate pages targeting the same search can compete with each other, which is keyword cannibalisation.

Cross-site duplication carries its own risk. If you syndicate an article and the partner’s copy is stronger, it may rank instead of your original. And if your product pages use the same manufacturer text as every other retailer, Google has little reason to show yours over theirs.

Fixing duplication is usually cheap compared with its effects: a few redirects, consistent canonicals and clean internal links can consolidate signals onto one URL per page.

How to fix duplicate content

1. Find it. Crawl the site and look for duplicate titles, meta descriptions and body text, and check the Search Console Pages report for “Duplicate without user-selected canonical” and “Duplicate, Google chose different canonical than user”.
2. Pick one preferred URL for every page: one protocol (https), one host (www or not), one trailing-slash style and no tracking parameters.
3. Redirect the other versions with 301 redirects where people do not need them, for example http to https and non-www to www.
4. Add a self-referencing canonical tag to every indexable page, and point duplicates that must stay live, such as parameter or print versions, at the preferred URL.
5. Make all your other signals agree: link internally only to preferred URLs, list only them in the XML sitemap, and use them in hreflang and structured data.
6. Merge or rewrite editorial duplicates. Combine near-identical pages into one stronger page and redirect the rest, and write unique descriptions for important products instead of using manufacturer text.
7. For language and regional versions, use hreflang. Pages translated into different languages are not duplicates.
8. For syndication, ask partners to noindex their copy. Google’s guidance is that a cross-domain canonical is not the most reliable fix here, because syndicated pages often differ from the original.

Duplicate content example

http://example.com/shoes, https://example.com/shoes, https://www.example.com/shoes, https://www.example.com/shoes/ and https://www.example.com/shoes/?utm_source=newsletter can all serve the same page. Choose https://www.example.com/shoes/ as the preferred URL, 301 redirect the http and non-www versions to it, add a canonical pointing to it on every variant, and link to it internally.