Key Takeaways
- Ahrefs found 95.2% of 1,002,165 domains had 3XX redirect issues and 72.9% were missing a meta description. When almost every site fails an item, that item describes the crawler's defaults rather than your business.
- Vercel and MERJ found that none of the major AI crawlers render JavaScript. A client-side rendered pricing page can be indexed fine by Google and read as blank to GPTBot, ClaudeBot and PerplexityBot.
- Google's own page experience docs say chasing a perfect Core Web Vitals score "may not be the best use of your time" — fix the pages between a buyer and the booking form, and leave the 2019 archive alone.
- Google treats rel=canonical as a hint rather than a rule, so the check that matters is whether your commercial pages are the URLs Google actually selected.
- A finished audit is four to eight changes with a named owner and a ship date. A 150-row crawler export is a report about a crawler.
Order the checklist by what a failure costs you
A technical SEO audit checklist is only worth running if it is ordered by consequence. Most of the published ones are inventories instead: 100 items, 150 items, 200 items, sorted by whatever severity label the crawler shipped with. Point one at a healthy B2B site and you get a red dashboard, a PDF, and no change in booked sales conversations.
The ordering problem is measurable. When Ahrefs crawled 1,002,165 domains for its site audit study, 95.2% had 3XX redirect issues, 80.4% were missing alt attributes, 72.9% had a missing or empty meta description, and 72.3% served at least one slow page. An item that almost every site on the internet fails is describing the crawler's defaults rather than your business.
Google says something similar about its own guidance. The Search Central crawl budget guide opens by telling most readers to leave: "If your site doesn't have a large number of pages that change rapidly, or if your pages seem to be crawled the same day that they are published, you don't need to read this guide." Google scopes it to sites with a million-plus pages, or ten thousand-plus pages that change daily. Almost no B2B company in the $1-10M range is either of those, and yet crawl budget shows up on every checklist.
What follows is the audit reordered into four layers, each one a question about whether a machine can reach, read, and repeat what your commercial pages say. Work them in order and stop at the first one that fails, because a fix at layer three does nothing while layer one is broken.
Layer one: can a machine read the page without running your JavaScript
Google processes pages in three phases — crawling, then rendering, then indexing. Its JavaScript SEO documentation describes queuing pages for a headless Chromium that executes the JavaScript "once Google's resources allow," which may be seconds or considerably longer. Anything absent from that rendered HTML does not get indexed.
That has been the standard caveat for years, and for Google alone it is mostly a solved problem. The part that has changed is who else is reading. In a joint analysis with MERJ published in December 2024, Vercel logged crawler behaviour across its network and found that none of the major AI crawlers render JavaScript — GPTBot, ClaudeBot, PerplexityBot, Meta-ExternalAgent and Bytespider all fetch raw HTML and stop there. Over the same window Googlebot made roughly 4.5 billion fetches per month against GPTBot's 569 million and ClaudeBot's 370 million.
The share is moving, too. Cloudflare's July 2025 crawler analysis found Googlebot's crawl volume up 96% year over year, with GPTBot's slice of the AI-crawler cohort rising from 5% to 30% over the same period. Whatever the exact split is by the time you read this, the direction is settled: a growing fraction of the machines reading your site will not execute a line of your JavaScript.
So a client-side rendered pricing page can be indexed perfectly well by Google and be functionally blank to every AI assistant a buyer might ask. The audit check takes about a minute:
curl -s https://yoursite.com/pricing | grep -i "your value proposition"
If your positioning and your call-booking path are not in the raw HTML response, they do not exist for the retrieval systems that increasingly sit between a buyer and your site. That is a layer-one failure and nothing below it matters until it is fixed.
Layer two: does the page that should book a call exist, and is it the canonical
Most technical audits spend their duplicate-content section on parameter URLs and trailing slashes. The question worth asking first is whether the page a buyer needs even exists.
Ahrefs' study of roughly 14 billion pages found that 96.55% get no traffic from Google at all, with another 1.94% picking up between one and ten monthly visits. Most sites do not have a technical indexation problem. They have a large volume of pages nobody was ever going to search for, and a small number of commercial pages that were never built. Auditing the first group harder does not produce the second.
Where duplication genuinely bites is when it lands on the pages that convert. Google's canonicalization documentation is blunt that "indicating a canonical preference is a hint, not a rule" — Google clusters pages it considers equivalent, picks whichever it judges most complete, and can override the canonical you declared. It ranks the signals it weighs, with redirects strongest, rel="canonical" strong, and sitemap inclusion weak.
The layer-two check is short. List the pages that are supposed to produce a sales conversation. For each one, confirm it is indexed, confirm Google selected the URL you intended as canonical, and confirm no thinner variant is outranking it. Everything else in the duplicate-content section can wait.
The failure mode I see most often on B2B sites is a services page and a near-identical landing page built for an old campaign, both live, both partially optimised for the same term. Google clusters them, picks one, and it is rarely the one with the booking form. That is a ten-minute fix with a redirect, and it is invisible on a crawler report because neither page is technically broken.
Layer three: speed, but only between the buyer and the form
Core Web Vitals has three current metrics with published thresholds: Largest Contentful Paint at 2.5 seconds or less, Interaction to Next Paint at 200 milliseconds or less, and Cumulative Layout Shift at 0.1 or less, each measured at the 75th percentile of real visits. INP replaced First Input Delay in March 2024, which is worth checking if your audit template predates that.
Passing is not unusual, and not universal. The HTTP Archive's 2024 Web Almanac put 48% of mobile sites and 54% of desktop sites through the full Core Web Vitals assessment, drawn from roughly 17 million origins.
Google's own framing is the useful part for an audit. Its page experience documentation confirms that "Core Web Vitals are used by our ranking systems," then immediately adds that "trying to get a perfect score just for SEO reasons may not be the best use of your time," and that Search "always seeks to show the most relevant content, even if the page experience is sub-par."
Read that as permission to be selective. Take the four or five pages that stand between a buyer and a booked conversation and get those under the thresholds. Leaving a 2019 blog archive at a 3.4-second LCP costs you nothing you can measure.
One practical note that trips up a lot of audits: the score in a Lighthouse run is a lab simulation on synthetic hardware, while the numbers Google uses come from field data on a rolling 28-day window of real visits. A page can score 94 in Lighthouse and still fail its Core Web Vitals assessment because actual buyers arrive on mid-range Android phones over patchy connections. Audit against the Search Console report, then use the lab tools to work out which change to make.
Layer four: the markup that tells a retrieval system what it is looking at
Structured data is where the technical audit now earns most of its keep, because it is how a machine works out what a page is without reading it the way a person would.
Semrush analysed 5 million URLs cited by ChatGPT Search and Google AI Mode for a study published in January 2026, and found structured data on a consistent share of cited pages: Organization schema on 25% of ChatGPT-cited pages and 34% of AI Mode citations, Article schema on 20% and 26%, breadcrumb markup on 15% and 20%. The study is explicit that these are correlations and not proof of causation. Its conclusion is the line worth pinning above the audit: technical factors should be evaluated "less as a checklist, and more as a foundation" that makes visibility possible once the other signals line up.
One myth to retire while you are in the robots file. Google's crawler documentation states that Google-Extended "does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Blocking it will not hurt your rankings. It will remove your content from Gemini grounding, which for a B2B company trying to get named in AI answers is usually the wrong trade.
If schema is thin on your commercial pages, that is a contained, one-sprint fix. Our guide to schema markup for AI search covers which types are worth implementing and which are decoration.
What the finished audit should actually look like
The deliverable determines whether anything happens. A 150-row spreadsheet exported from a crawler gets forwarded, admired, and archived. I spent seven years running marketing for a firm that made the Inc. 5000 four years running, and the pattern held every time: the audit that got implemented was the one a developer could act on before the next sprint closed.
So the output of the four layers is four to eight changes, each with a named owner and a ship date. Something like:
- Server-render the pricing and services pages so raw HTML carries the offer.
- Fix the canonical on the duplicated demo page and 301 the older variant.
- Get LCP under 2.5 seconds on the two call-booking pages.
- Add Organization and Article schema sitewide.
Four lines, all shippable, all traceable to a reason. That is a technical SEO audit. The 150-item version is a report about a crawler.
The wider point is that technical work only pays when something is waiting to be found. A perfectly crawlable site with no commercial-intent pages ranks for nothing worth ranking for, which is why the technical pass belongs inside a Demand Engine rather than beside it. If you want the layer above this one, our B2B SEO audit template covers content and intent, the GEO audit walkthrough covers AI-search visibility specifically, and the SEO and AEO checklist condenses both into something you can work through in an afternoon.
