SERP Monitoring

The scenario

You rank third for "best invoicing software" on Monday. By Friday a competitor's AI Overview has eaten the top of the page, your organic listing is below the fold, and your demo signups have quietly dropped fourteen percent. Nobody on the team noticed for six days because nobody on the team manually checks the search result every morning - and the rank-tracker tool you're paying for only reports position, not layout.

This is the modern SERP problem. The page that Google shows for your keywords is not static. Positions move, AI Overviews appear and disappear, featured snippets swap owners, and People Also Ask blocks expand. Each of those shifts changes whether real users actually see your listing.

The problem

Tracking SERPs manually does not scale past a handful of keywords. Even at five or ten, the data goes stale within hours. Tools that automate it split into two camps that each leave gaps. Rank trackers (Semrush, Ahrefs, AccuRanker) report numeric position well but miss layout changes - your "position 3" might be position 3 on a page where positions 1 and 2 are now an AI Overview and a video carousel, and your CTR has cratered. Visual monitors (Visualping, SERPWoo) capture the layout but cost more per keyword and don't expose the structured fields you'd need to alert on programmatically.

What most SEO and growth teams actually want is the layered version: notify me when anything on the SERP for this keyword changes, with the structured fields attached so the alert is actionable.

How Verid solves it

Point Verid at the Google search URL for the keyword you care about. Tell it which xpath or CSS selectors map to the SERP elements you want to track - the top organic title, the AI Overview block, the first featured snippet, your own listing's position. Pick a check interval - every fifteen minutes for high-value commercial intent, hourly for the long tail.

Verid renders the page with a real Chrome browser (CloakBrowser-backed, so detection is not the failure mode here), extracts each field, diffs against the previous run, and fires a webhook the moment any one of them shifts. You define one monitor per keyword. There is no rank-tracker subscription to renew, no screenshot-diff dashboard to log into, no scraping cluster to maintain.

Build the monitor

Extraction config

{
  "method": "xpath",
  "fields": {
    "top_result_title":  "(//div[@id='search']//h3)[1]",
    "top_result_url":    "(//div[@id='search']//a[h3])[1]/@href",
    "ai_overview":       "//div[contains(@aria-label,'AI Overview')]",
    "featured_snippet":  "//div[@data-attrid='wa:/description']",
    "people_also_ask":   "//div[contains(@jsname,'Cpkphb')]//span[@role='heading']"
  }
}

Predicate

Fire on any change to any tracked field:

{ "type": "any_field_changes" }

If you only care when an AI Overview appears or disappears, narrow it:

{ "type": "field_changes", "field": "ai_overview" }

Create the monitor

curl:

curl -X POST https://api.verid.dev/v1/monitors \
  -H "Authorization: Bearer vrd_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "SERP - best invoicing software",
    "url": "https://www.google.com/search?q=best+invoicing+software&hl=en&gl=us&pws=0",
    "fetch_mode": "browser",
    "schedule_interval_seconds": 1800,
    "extract_config": {
      "method": "xpath",
      "fields": {
        "top_result_title":  "(//div[@id=\"search\"]//h3)[1]",
        "top_result_url":    "(//div[@id=\"search\"]//a[h3])[1]/@href",
        "ai_overview":       "//div[contains(@aria-label,\"AI Overview\")]",
        "featured_snippet":  "//div[@data-attrid=\"wa:/description\"]",
        "people_also_ask":   "//div[contains(@jsname,\"Cpkphb\")]//span[@role=\"heading\"]"
      }
    },
    "diff_predicate": { "type": "any_field_changes" },
    "deliveries": [
      { "type": "slack", "webhookUrl": "https://hooks.slack.com/services/..." }
    ]
  }'

SDK:

import { VeridClient } from '@verid.dev/sdk';

const client = new VeridClient({ apiKey: 'vrd_your_api_key' });

await client.monitors.create({
  name: 'SERP - best invoicing software',
  url: 'https://www.google.com/search?q=best+invoicing+software&hl=en&gl=us&pws=0',
  fetch_mode: 'browser',
  schedule_interval_seconds: 1800,
  extract_config: {
    method: 'xpath',
    fields: {
      top_result_title:  '(//div[@id="search"]//h3)[1]',
      top_result_url:    '(//div[@id="search"]//a[h3])[1]/@href',
      ai_overview:       '//div[contains(@aria-label,"AI Overview")]',
      featured_snippet:  '//div[@data-attrid="wa:/description"]',
      people_also_ask:   '//div[contains(@jsname,"Cpkphb")]//span[@role="heading"]',
    },
  },
  diff_predicate: { type: 'any_field_changes' },
  deliveries: [
    { type: 'slack', webhookUrl: 'https://hooks.slack.com/services/...' },
  ],
});

What the webhook delivers

{
  "id": "del_01H...",
  "version": "2026-05-01",
  "monitor_id": "9b1c…",
  "run_id": "7e2a…",
  "fired_at": "2026-05-24T08:30:00Z",
  "diff": {
    "fields_changed": ["ai_overview", "top_result_title"],
    "before": {
      "top_result_title":  "Top 10 Invoicing Tools for Small Business in 2026",
      "top_result_url":    "https://competitor-a.com/invoicing-tools-2026",
      "ai_overview":       "",
      "featured_snippet":  "Invoicing software automates billing, payment tracking…",
      "people_also_ask":   "What is the best free invoicing software?"
    },
    "after": {
      "top_result_title":  "AI-Powered Invoicing: 7 Tools Compared",
      "top_result_url":    "https://competitor-b.com/ai-invoicing-2026",
      "ai_overview":       "Modern invoicing tools combine automated billing with AI-driven payment prediction. Top options include…",
      "featured_snippet":  "Invoicing software automates billing, payment tracking…",
      "people_also_ask":   "What is the best free invoicing software?"
    }
  },
  "monitor": {
    "url": "https://www.google.com/search?q=best+invoicing+software&hl=en&gl=us&pws=0",
    "name": "SERP - best invoicing software"
  }
}

Your growth team consumes the webhook, sees that an AI Overview just appeared and a different competitor took the top organic slot, and triages: rewrite the targeted landing page, ask the SEO contractor to investigate, or just note it for the weekly review.

Caveats & tips

• Always set fetch_mode: "browser". Google's results are rendered by JavaScript and the raw HTML returned by a plain HTTP fetch is mostly inert. Browser mode is non-negotiable for SERPs.
• Pin the locale and disable personalization. Use hl=en&gl=us&pws=0 in the search URL (or whatever locale you actually care about). Without it, two runs from the same monitor can return different SERPs purely because the server-side state changed, and your diffs will be noisy.
• Selectors evolve - expect to update them. Google ships SERP layout changes regularly. The xpath above is illustrative and will eventually drift; when a field starts returning empty values in the dashboard, that's the signal to inspect the new DOM and adjust.
• Be polite with the check interval. One monitor per keyword at fifteen-to-thirty minute intervals is sustainable. Hammering the same keyword every minute from one IP is how you get challenged. For high-frequency tracking across many keywords, route through the residential proxy layer.
• Bing and DuckDuckGo work too. Same monitor pattern, different URL - bing.com/search?q=… or duckduckgo.com/?q=…. Bing is more aggressive with bot challenges from datacenter IPs, so it's the case where the residential proxy fallback earns its keep.
• Mind the ToS gray area. Google's terms technically prohibit automated SERP scraping. This is the same gray zone every commercial rank-tracker operates in. For your own keywords or competitive intelligence on a handful of terms, the practical risk is negligible; for industrial-scale rank tracking across thousands of keywords, talk to a lawyer.

Related use cases

If you're tracking how your own SEO metadata drifts on indexed pages, see SEO title and meta description drift. For watching competitor landing page copy directly, see competitor ad and landing copy changes. And for spotting when a competitor swaps their analytics or martech stack - often a leading indicator of strategy shifts - see competitor tech stack changes.