Sitemap.xml New URL Detection

The scenario

A competitor publishes a new landing page or product page. By the time you spot it via Google, they've had a week of search exposure on it. Their sitemap.xml gets updated within an hour of publish, though - and it's public. If you watched their sitemap for new URLs, you'd see new content the same day it ships.

The same pattern applies to internal SEO teams checking that pages they shipped actually made it into the sitemap, and to anyone aggregating "what's new" across a set of sites.

The problem

Sitemaps aren't designed for human consumption. They list every URL the site wants indexed, often thousands of them, and diffing two of them by hand is tedious. A purpose-built sitemap-diff tool exists but you'd want a SaaS-style alerting layer around it; otherwise you're back to scheduled scripts.

How Verid solves it

The sitemap-new-url template uses a regex extractor to count <loc> entries and capture the most recent URL. When the count goes up, new pages have appeared.

For deeper inspection - which specific URL is new - you can return the count and the first/last <loc> value as separate fields, then re-fetch the sitemap in your webhook handler to compute the precise diff.

Build the monitor

Extraction config

{
  "method": "regex",
  "fields": {
    "url_count": "<loc>",
    "latest_url": "<loc>([^<]+)</loc>(?![\\s\\S]*<loc>)"
  }
}

The url_count field uses the bare <loc> pattern - the regex engine returns the count of matches when the pattern has no capture group on count-style queries. The latest_url uses a negative-lookahead to capture the final <loc> value in the document.

Predicate

{ "type": "field_changes", "field": "url_count" }

A change in the count is the cleanest signal that something was added or removed.

Create the monitor

Using the template:

curl -X POST https://api.verid.dev/v1/monitors/from-template/sitemap-new-url \
  -H "Authorization: Bearer vrd_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Competitor sitemap",
    "url": "https://competitor.com/sitemap.xml",
    "deliveries": [
      { "type": "webhook", "url": "https://my-seo-tool.example.com/webhooks/sitemap" }
    ]
  }'

SDK:

import { VeridClient } from '@verid.dev/sdk';

const client = new VeridClient({ apiKey: 'vrd_your_api_key' });

await client.monitors.createFromTemplate('sitemap-new-url', {
  name: 'Competitor sitemap',
  url: 'https://competitor.com/sitemap.xml',
  deliveries: [
    { type: 'webhook', url: 'https://my-seo-tool.example.com/webhooks/sitemap' },
  ],
});

What the webhook delivers

{
  "id": "del_01H...",
  "fired_at": "2026-05-08T09:30:00Z",
  "diff": {
    "fields_changed": ["url_count", "latest_url"],
    "before": { "url_count": "2418", "latest_url": "https://competitor.com/blog/post-abc" },
    "after":  { "url_count": "2421", "latest_url": "https://competitor.com/blog/post-xyz" }
  }
}

Three new URLs were added. Your handler can re-pull the sitemap and compute the precise set diff if it needs the exact URLs.

Caveats & tips

• Sitemap index files. Larger sites use sitemap_index.xml which references multiple sub-sitemaps. Verid's regex against the index will give you a count of sub-sitemaps, not URLs. To monitor a real-content sitemap, target one of the referenced sitemaps directly - or run separate monitors per sub-sitemap.
• Hash-style alerts. If you only need "did the file change at all?", swap the extractor to full_page - Verid will hash the response and fire on any byte-level change. Cheaper and simpler.
• Polling interval. Once or twice a day is usually enough. SEO signal isn't real-time and the urgency rarely justifies a fast interval.

Related use cases

For RSS-style new-item detection, see RSS feed new items. For competitor copy tracking, see competitor ad & landing copy changes. For SEO title drift on specific pages, see SEO title & meta description drift.