Technical SEO Audit: Find Crawl, Index and Site Health Problems

Technical issues are expensive for one reason: they affect everything downstream. Fixing a title tag might improve one page. Fixing an accidental noindex can restore your entire site.

They are also easy to introduce and hard to see. A developer pushes a build with the staging robots meta. A plugin update changes canonicals. A CDN change breaks the sitemap. The site looks the same in a browser, so no one notices until traffic falls.

Common technical SEO failures include:

• Important pages blocked by robots.txt or noindex.
• Canonical tags pointing to the wrong URL or to a redirecting URL.
• Redirect chains and loops.
• Broken internal links and 404s on pages that used to rank.
• Duplicate or missing titles and descriptions across templates.
• HTTP pages still serving content, or mixed content.
• Heading structures that skip levels or use several H1s without reason.
• Missing or invalid structured data.
• Slow, uncompressed, uncached pages.

None of these is glamorous. All of them are cheaper to find early.

The audit crawls your homepage and up to four additional pages from your sitemap, fetching each page and recording measurable facts. It also reads robots.txt and sitemap.xml and looks for broken internal links across the sample. The data is then analysed, with issues grouped by severity.

What is checked

Crawlability and indexability. Is the page reachable (status code)? Does robots meta or a header restrict indexing? Does the canonical point where it should? Are robots.txt rules and sitemap in place and sensible?

On-page structure. Title and description present and within useful lengths, duplicates across pages, one clear H1, logical heading order.

Links and images. Broken internal links, link counts, nofollow usage, images without alt text, oversized images and lazy loading.

Security and delivery. HTTPS, HSTS, compression, caching headers and security headers.

Mobile and international basics. Viewport tag and language attribute.

Structured and social data. JSON-LD types found, whether they parse, and Open Graph and Twitter completeness.

Stack. Technology detected helps you decide where to fix an issue (theme, plugin, server).

Prioritisation

Findings are sorted into critical issues, warnings and opportunities. Anything that could prevent indexing or crawling is critical. Cosmetic or marginal issues are not.

Examples of what the audit can surface and how to act on them.

Repeat after releases and migrations.

Post-migration check

A company migrates to a new CMS. The first audit finds that canonicals on product pages point to the old domain, and the sitemap still lists old URLs. The team fixes both within a day, avoiding months of confused indexing.

Staging leftovers

After a redesign, traffic drops. An audit shows a noindex robots meta on the homepage. It was in the staging theme and shipped to production. Removing it restores visibility.

Agency onboarding

An agency audits a new client, finds duplicate titles across service pages and no structured data, and uses the findings as the first month's roadmap.

Agencies: audit at onboarding and after each release. Web designers and developers: audit before handover and after launch. Businesses: audit after any theme, plugin or hosting change. Freelancers: use findings to scope technical retainers.

Specialist crawlers and log analysis go deeper for large sites. The audit tool is for fast, repeatable checks.

Benefits

• Catches costly configuration errors quickly.
• Plain-English explanations for non-specialists.
• Repeatable and stored over time.

Limitations

• Five-page sample.
• No Core Web Vitals measurement. Use PageSpeed Insights and Search Console.
• No JavaScript rendering analysis.
• No log analysis.
• AI explanations should be verified before risky changes such as editing robots.txt.

1. Audit after every significant release.
2. Always verify noindex and robots changes by hand.
3. Fix templates, not pages.
4. Keep sitemaps clean: indexable 200 URLs only.
5. Use Search Console alongside any audit.
6. Keep a changelog so regressions can be traced.

• Blocking CSS or JavaScript Google needs to render the page.
• Using robots.txt to remove pages from search (use noindex instead, and do not block a noindex page).
• Redirect chains after migrations.
• Fixing issues without re-checking.
• Treating audit scores as the goal.

Both files are short and a common source of costly mistakes. Open them in your browser.

robots.txt (yoursite.com/robots.txt)

• Is there a blanket "Disallow: /" under a user-agent that applies to search engines? That blocks the whole site.
• Are CSS and JavaScript directories blocked? Search engines may need them to render pages.
• Does it point to your sitemap?
• Remember it controls crawling, not indexing; a blocked URL can still appear in results if linked elsewhere. To keep a page out of results use noindex, and do not block the page in robots.txt, otherwise crawlers cannot see the noindex.

sitemap.xml

• Does it list only canonical, indexable URLs that return 200?
• Does it include redirected, noindexed or 404 URLs? Remove them.
• Is it submitted in Search Console, with no errors?
• Does the number of URLs match what you expect?

Most migration traffic losses are redirect problems.

1. Make a list of every URL that gets traffic or links before you change anything.
2. Map each old URL to its closest new equivalent.
3. Use 301 redirects for permanent moves, pointing straight to the final URL.
4. Check there are no chains (A to B to C) and no loops.
5. Keep redirects in place long enough for search engines to process them, typically a year or more.
6. After launch, test a sample of old URLs and check Search Console for crawl errors.
7. Update internal links to point to the new URLs directly.

Google documents site moves with URL changes in Search Central; follow its checklist for larger migrations.

These are hygiene items. They rarely move rankings directly, but missing HTTPS or serious security problems can harm users and trust, and compression and caching help speed.

Use this as a calendar.