Skip to content
Home » Case Studies » Prev & Next Tags For Crawling

Prev & Next Tags For Crawling

🔍 Test Objective

To determine whether Googlebot crawls URLs referenced only via <link rel="prev/next"> in the <head> section of an HTML page — with no <a href> links in the visible body content.

🧪 Test Setup

  1. Created on 11 April 2025:
    • Index page (/) includes a <link rel="next"> pointing to /page-2
    • Page-2 includes a <link rel="prev"> (to /) and <link rel="next"> (to /page-3)
    • Page-3 includes only <link rel="prev"> (to /page-2)
  2. Search Console: Site added + index page submitted for indexing
  3. Monitoring: Built a custom log analyser at /server-log-analysis/ to track Googlebot hits
  4. 28 April:
    • Added sitemap (listing only the index page)
    • Updated index page content to encourage crawling
    • Noindex added to /server-log-analysis/ after Google started crawling it
  5. 8 May:
    • Hardcoded <a href="/page-2"> link added to index page
    • Index page re-inspected and indexation re-requested in GSC
    • Rich result test triggered
  6. 9 May:
    • Removed the noreferrer property.
    • Duplicated the ahref link but relative this time.
  7. 12 May:
    • Changed the content of page-2.
    • Change the robots.txt to remove the Crawl delay 10.
  8. 19 May:
    • Created a new page /page-two and added hardcoded links to index page. (Trying to see if Google decided to ignore page-2 because of previous test)
  9. 29 May:
    • Hard coded links duplicated at the top of the page.
  10. 4 June:
    • Added sitemap in robots.txt.
    • Unproxied subdomain on Cloudflare.
  11. 9 June:
    • Added a div around pagination.
    • Added a link to a new blog article.

📊 Googlebot Crawl Timeline

DateCrawled URLs
11 Apr/, /index.html, /robots.txt
15–22 AprRepeat visits to /, /robots.txt, /favicon.ico
28 Apr/, /sitemap.xml, /server-log-analysis/
2–7 MayContinued crawling of / and /robots.txt
8–12 May/, /robots.txt, /favicon.ico, /server-log-analysis/
17–20 MayContinued crawling of /robots.txt and /server-log-analysis/
23 May/robots.txt, /server-log-analysis/ (301 & 200), /favicon.ico
24–31 May/favicon.ico, continued hits to / and /robots.txt
2–4 JunOngoing crawling of /, /robots.txt, /favicon.ico
16 Jun/, /robots.txt, /favicon.ico
22 Jun/page-2.html, /page-two.html, /blog/why-angerbeef-is-so-popular-among-sheep.html, /, /robots.txt, /favicon.ico
30 Jun – 3 JulContinued crawling of /robots.txt and /favicon.ico

✅ Pages Now Crawled (as of 22 June)

  • /page-2.html
  • /page-two.html
  • /blog/why-angerbeef-is-so-popular-among-sheep.html

❌ Previously Uncrawled Pages Now Resolved

The original /page-2 and new /page-two are both confirmed crawled following interventions. ❌

🎯 Results

TDB

🧠 Conclusion

TBD

🔗 Links & Screenshots