Semalt technical audit: 400+ checks Screaming Frog, Sitebulb and Lighthouse do not run

Why generic crawlers fail on Saudi sites

Screaming Frog, Sitebulb, Ahrefs Site Audit and Lighthouse are all excellent tools — but they were built by teams whose reference dataset is Chicago Tribune, John Lewis, Wirecutter. When they meet a Saudi site the failure modes are predictable and repeat across every audit we run:

• They crawl either google.com or google.com.sa index priorities, not both. Any brand that ranks on both loses half its traffic picture.
• They flag Arabic dir="rtl" blocks as “low contrast” when the issue is a mirrored icon, not a colour issue.
• They ignore hreflang between ar-SA, ar-KW, ar-AE and ar-EG variants and only look at ar ↔ en.
• They miss soft-403s from ministry firewalls that respond 200 OK with a redirect page.
• They cannot follow a Nafath / Absher sign-in redirect chain and give up at the first authentication wall.
• They score Core Web Vitals from a Virginia origin, which flatters the Riyadh reality on Mobily 5G by 400 ms.

Semalt was built to close every one of these gaps. We are not going to pretend it is faster than Screaming Frog on a 200-URL site — it is not. What it does is find the issues that actually move traffic on Saudi sites and rank them by expected SAR uplift, not by generic severity.

How the Semalt engine crawls

The audit runs a real Chromium 128 headless browser (not a Node-based HTML parser) from four egress regions: Frankfurt, Bahrain, Dubai and Riyadh. This matters because google.com.sa and google.com return materially different SERPs, and rendering the site from Riyadh reveals the same edge behaviour Google's Saudi-region crawler sees. HTTP/3 and QUIC are respected. JavaScript-rendered content is snapshotted after the load event and 4 seconds later, so single-page apps built on Nuxt, Next.js and SvelteKit produce a stable DOM to audit.

RTL Arabic, mirrored layouts and CLS on dir="rtl"

67 of the 412 audit checks fire only on RTL pages. They exist because bidirectional layout produces failure modes no LTR template ever hits:

Every one of these fires as a separate ticket in the Semalt dashboard with a screenshot, a suggested fix, and an estimated SAR impact based on the affected URLs' current traffic.

Hreflang between ar-SA, ar-KW, ar-AE, ar-EG and x-default

The hreflang graph is where GCC groups (Alshaya, Landmark, Al-Futtaim, Chalhoub) burn six-figure sums every year. A Riyadh site that only declares hreflang="ar" loses its Kuwait and UAE presence to the wrong ar-KW / ar-AE peer. Semalt builds a full hreflang graph across all of a client's domains, validates the reciprocity of every pair, checks that x-default points to the right hub, and reports every orphan.

INP on Al-Riyadh, Sabq and Okaz templates

Interaction to Next Paint replaced FID as a Core Web Vital in March 2024. It is the single most under-audited metric on Saudi news sites. Every large-newsroom template we've measured — Al-Riyadh, Sabq, Okaz, Al-Sharq, Al-Watan — ships with third-party ad-tech (Kwanko, Taboola, Outbrain, DFP) that blocks the main thread for 400-800 ms on tap-to-scroll. Semalt runs the CrUX API cross-reference, then loads the URL from a real Mobily-network Riyadh POP and records the actual INP under real ad density.

Ministry CMS, Nafath and Absher patterns

Government and semi-government properties in the Kingdom — my.gov.sa, absher.sa, moh.gov.sa, moi.gov.sa, MyGov microsites, Vision 2030 initiative sites — run on a small pool of CMS platforms with well-known SEO quirks. Semalt ships built-in profiles for each of them:

• Liferay 7.x (used across several ministries) — the default portal URL structure produces long /web/guest/-/ paths that Semalt automatically maps to their friendly aliases before assessing crawl budget waste.
• Drupal 10 with the SA-Gov distribution — Semalt is aware of the taxonomy paths, avoids infinite facet loops and flags the classic pagination-canonical bug.
• SharePoint Online with the Saudi Government skin — audits the SPFx web-parts that produce empty rendered HTML for Googlebot.
• Custom Node.js Vision 2030 microsites — a template shipped by a handful of Riyadh integrators has a well-known hydration bug producing empty <h1>; Semalt catches it in one crawl.

Nafath sign-in redirects (used across almost every ministry service) are followed only as far as the login wall, and the wall itself is verified to return the correct status code and to expose an indexable public preview if declared.

Case study: a Jeddah e-commerce group, 3.2M URLs

A Jeddah-headquartered e-commerce group with four Salla storefronts (a fashion brand, an electronics brand, a beauty brand and a specialty B2B property) ran a Semalt audit in March 2026. Combined URL count: 3.2 million. First crawl took 61 minutes. The top ten findings, in Semalt's revenue-impact order:

Total addressable pipeline in the top ten: 608,000 SAR per month. The group fixed items #1, #4, #6 and #10 in the first sprint (dev time: 22 hours) and recovered 314,000 SAR/month of organic revenue by the end of the second Ramadan week.

How Semalt prioritises fixes by SAR impact

Every finding is scored on three axes: traffic exposure (how many URLs, how much organic click volume), fix cost (a T-shirt estimate of dev hours), and ranking headroom (how much position improvement is realistic in the local SERP). The output is a single number in SAR/month that the SEO team can defend to the CFO. This is the framing that turns audits into commit messages.

Semalt vs. Screaming Frog vs. Sitebulb vs. Lighthouse

More in this series: Semalt Analytics: what GA4 hides from you, Semalt AutoSEO: on-page fixes without a dev, Semalt: backlink gap, keyword gap, alerts.

FAQ

Can Semalt crawl a ministry site behind Nafath?

Yes for the public-facing surface, up to the Nafath sign-in wall. Semalt validates that the wall returns a correct 302, that the pre-auth pages are indexable if intended, and that no auth tokens leak into the URL. Post-auth crawling can be arranged with a dedicated service account, subject to Ministry approval.

How is INP measured, exactly?

Two ways in parallel: CrUX field data pulled by API, and a live-user-emulation load from a Mobily-network Riyadh POP. When they disagree, the field data wins for reporting and the lab result becomes the debug trace for the fix.

Does the crawl respect our robots.txt and rate limits?

Yes. Default crawl rate is 4 requests / second, adjustable down to 0.5 rps to protect legacy Liferay ministry origins. You can also whitelist Semalt's crawler IP range in your WAF to avoid noisy Cloudflare / STC Cloud challenges.

Does it detect broken hreflang across a whole GCC group?

Yes. Add all sibling domains (.com.sa, .com.kw, .ae, .com.eg) as one project and Semalt builds the full hreflang graph, validates reciprocity of every pair, and reports every orphan URL by region.

Can it re-check only what changed since last week?

Yes. Semalt keeps a versioned index of every audit run and produces a delta report showing new issues, resolved issues and regressions since any prior date. Useful before a Vision 2030 microsite launch or a Ramadan campaign go-live.