Steel and glass funnel collecting glowing droplets that settle into layers sorted by size

Core Web Vitals Real User Monitoring: Design Your Own RUM Pipeline

•11 min read

Lighthouse tells you how a page behaves on one machine, once. Your users get slower phones, worse networks and late third-party scripts. Core Web Vitals real user monitoring (RUM) closes that gap by measuring LCP, INP and CLS in the browsers of actual visitors and sending the data to a backend you control. This guide is about designing that system: the client collector, the payload, the ingestion endpoint, the data model, the aggregation queries and the alerting, with the trade-offs at each step.

If you are looking for the optimization work itself (images, caching, scripts, fonts), start with our guide on how to improve website speed in 9 steps. This article is about knowing, with evidence, which fixes matter for your real traffic and whether a deploy made things worse.

Why build RUM when CrUX already exists

Google's Chrome UX Report (CrUX) is free and based on real Chrome users, but it has two limits that matter for engineering teams. First, the CrUX API values are calculated over the previous 28 days as a rolling window, so a regression shipped today blends in slowly and a fix takes weeks to show. Second, it only knows about URLs and origins: it cannot tell you which release, which page template, which A/B variant or which DOM element caused the problem.

Your options, roughly:

  • CrUX only. Zero build cost, no per-release or per-element detail, slow feedback.
  • A commercial RUM product. Fast to adopt, good dashboards; you pay per volume and accept their data model and retention.
  • Your own pipeline. More work, but you choose the dimensions (release, route template, tenant, experiment), keep the raw data and can join it with business events like checkout completion.

Building your own makes sense when you deploy often, need data per tenant or customer, or must keep the data in your infrastructure.

The targets you are measuring against

Keep the definitions straight before designing storage. INP replaced First Input Delay as a Core Web Vital on March 12, 2024, so a modern pipeline collects LCP, INP and CLS (plus TTFB as a diagnostic). The "good" thresholds are LCP of 2.5 s or less, INP under 200 ms and CLS under 0.1, assessed at the 75th percentile, with phones and desktop evaluated separately. Google explains that it chose the 75th percentile partly because it is less sensitive to outliers.

So: aggregate with percentiles, not averages, and make device class a first-class dimension.

Core Web Vitals real user monitoring: the architecture

A RUM pipeline has five parts:

  1. Collector in the page: measures metrics and attribution, batches them.
  2. Beacon endpoint: accepts small, untrusted POSTs, validates them, responds fast.
  3. Buffer: a queue or an in-memory batch so ingestion never blocks on the database.
  4. Store: one row per metric per page view, upserted by metric ID.
  5. Aggregation and alerting: p75 by route, device and release, with minimum sample sizes.

Start simple: one small service writing to Postgres, and swap the buffer and store later if volume demands it.

Step 1: the client collector

Do not write the metric logic yourself; INP and CLS have edge cases (back/forward cache, prerendering, background tabs) that Google's web-vitals library already handles. Use its attribution build, which adds an attribution object to each metric with diagnostic data and is imported from web-vitals/attribution instead of web-vitals. Recent releases also expose the target element in INP attribution.

import {onCLS, onINP, onLCP, onTTFB} from 'web-vitals/attribution';
const ENDPOINT = '/rum';
const SAMPLE_RATE = Number(window.__RUM_SAMPLE_RATE__ ?? 1);
// Sample per session, not per page view, so a user is either in or out.
function inSample() {
  let flag = sessionStorage.getItem('rum');
  if (flag === null) {
    flag = Math.random() < SAMPLE_RATE ? '1' : '0';
    sessionStorage.setItem('rum', flag);
  }
  return flag === '1';
}
const pending = new Map(); // keyed by metric.id: only the latest value is kept
function summarize({name, attribution: a = {}}) {
  switch (name) {
    case 'LCP': return {target: a.target, ttfb: a.timeToFirstByte,
      loadDelay: a.resourceLoadDelay, loadDuration: a.resourceLoadDuration,
      renderDelay: a.elementRenderDelay};
    case 'INP': return {target: a.interactionTarget, type: a.interactionType,
      inputDelay: a.inputDelay, processing: a.processingDuration,
      presentation: a.presentationDelay};
    case 'CLS': return {target: a.largestShiftTarget};
    default: return {};
  }
}
function enqueue(metric) {
  pending.set(metric.id, {
    id: metric.id, name: metric.name, value: metric.value,
    rating: metric.rating, nav: metric.navigationType,
    attr: summarize(metric),
  });
}
function flush() {
  if (pending.size === 0) return;
  const body = JSON.stringify({
    route: window.__ROUTE__,          // template, e.g. "/product/:slug"
    release: window.__RELEASE__,      // git SHA or build number
    ect: navigator.connection?.effectiveType ?? null,
    metrics: [...pending.values()],
  });
  pending.clear();
  const ok = navigator.sendBeacon?.(ENDPOINT, body);
  if (!ok) fetch(ENDPOINT, {method: 'POST', body, keepalive: true});
}
if (inSample()) {
  onLCP(enqueue); onINP(enqueue); onCLS(enqueue); onTTFB(enqueue);
  addEventListener('visibilitychange', () => {
    if (document.visibilityState === 'hidden') flush();
  });
}

Design notes:

  • Flush on visibilitychange to hidden, not on unload. Hidden is the last moment you can reliably send data on mobile, and INP and CLS keep changing until then.
  • The same metric can be reported more than once. If the user comes back to the tab and interacts again, INP for the same page view gets a new value with the same id. The library's own examples pass id precisely so events can be aggregated per page view. Your backend must treat id as an upsert key.
  • Send a string body to a first-party path. A string body goes out as text/plain (no CORS preflight), and first-party paths are less likely to be blocked.
  • Send the route template, not the raw URL. /product/:slug aggregates; /product/blue-shirt-xl?utm_source=… explodes your cardinality and can leak personal data in query strings.
  • Keep the attribution small: selectors and a few numbers, not full performance entries.

The attribution build is a little heavier than the standard build, so load the collector after the critical path, for example with a dynamic import() once the page is interactive.

Step 2: the beacon endpoint

Treat every beacon as hostile input and do as little as possible synchronously.

// Express-style pseudo-code
app.post('/rum', express.text({type: '*/*', limit: '16kb'}), (req, res) => {
  res.status(204).end();                 // respond first, never make the page wait
  let p;
  try { p = JSON.parse(req.body); } catch { return; }
  if (!Array.isArray(p.metrics) || p.metrics.length > 10) return;
  if (isBot(req.get('user-agent'))) return;
  const route = KNOWN_ROUTES.has(p.route) ? p.route : 'other';
  const device = deviceClass(req.get('user-agent'));    // 'mobile' | 'desktop' | 'tablet'
  const country = req.get('cf-ipcountry') ?? null;       // CDN geo header; never store the IP
  for (const m of p.metrics) {
    if (!ALLOWED.has(m.name)) continue;
    if (typeof m.value !== 'number' || m.value < 0 || m.value > MAX[m.name]) continue;
    buffer.push({
      metric_id: String(m.id).slice(0, 64), name: m.name, value: m.value,
      rating: m.rating, nav: m.nav, route, device, country,
      release: String(p.release ?? '').slice(0, 40),
      attr: truncateStrings(m.attr, 200),
    });
  }
});
setInterval(() => writeBatch(buffer.splice(0)), 5000); // batch inserts

Key decisions:

  • Allow-list routes and metric names. Anything else becomes other or is dropped. This is your main defense against cardinality explosions and junk data.
  • Cap values per metric (MAX) at a ceiling you choose; values beyond it are usually broken clocks or tabs left open.
  • Derive device class and country on the server. Do not trust the client for dimensions you can compute, and avoid storing IP addresses at all.
  • Rate-limit at the edge: a RUM endpoint is an open write API.
  • Batch writes. At higher volume, replace the in-memory buffer with a queue (SQS, Pub/Sub, Kafka, Redis Streams) so a database incident does not drop the service.

Step 3: the data model

Store one row per metric per page view. Keep it narrow and append-friendly.

CREATE TABLE rum_metric (
  metric_id    text PRIMARY KEY,           -- web-vitals id, unique per page view + metric
  received_at  timestamptz NOT NULL DEFAULT now(),
  name         text NOT NULL,              -- LCP | INP | CLS | TTFB
  value        double precision NOT NULL,
  rating       text,                       -- good | needs-improvement | poor
  nav          text,                       -- navigate | reload | back-forward-cache | prerender ...
  route        text NOT NULL,
  device       text NOT NULL,
  country      text,
  release      text,
  attr         jsonb
);
CREATE INDEX ON rum_metric (name, received_at);
-- Upsert so later reports for the same page view replace earlier ones
INSERT INTO rum_metric (metric_id, name, value, rating, nav, route, device, country, release, attr)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
ON CONFLICT (metric_id) DO UPDATE
SET value = EXCLUDED.value, rating = EXCLUDED.rating,
    attr = EXCLUDED.attr, received_at = EXCLUDED.received_at;

Storage trade-offs:

  • Postgres with raw rows is the simplest and exact. Partition by day and drop old partitions to control retention.
  • A columnar store (ClickHouse, BigQuery) handles much larger volumes and has built-in quantile functions, at the cost of another system to run. Columnar stores usually prefer appends over upserts; there you insert every report and keep the latest per metric_id at query time (for example with argMax in ClickHouse).
  • Pre-aggregated histograms (fixed buckets per metric, route, device and hour) are tiny and merge cheaply, but percentiles become approximate and you lose per-element attribution. A common hybrid: histograms for long-term trends, raw rows with short retention for debugging.

Step 4: aggregation queries that match how Google scores you

Compute p75 per metric, per device class, per route, with a minimum sample size so small groups do not trigger noise.

SELECT route, device,
       count(*) AS samples,
       percentile_cont(0.75) WITHIN GROUP (ORDER BY value) AS p75
FROM rum_metric
WHERE name = 'INP'
  AND received_at > now() - interval '7 days'
GROUP BY route, device
HAVING count(*) >= :min_samples
ORDER BY p75 DESC;

To find what to fix, group by attribution instead of by route:

SELECT attr->>'target' AS element,
       count(*) AS poor_interactions,
       avg((attr->>'processing')::float) AS avg_processing_ms
FROM rum_metric
WHERE name = 'INP' AND rating = 'poor'
  AND received_at > now() - interval '7 days'
GROUP BY 1
ORDER BY poor_interactions DESC
LIMIT 20;

The INP sub-parts tell you where the time went: a long input delay points to main-thread work that was already running (often third-party scripts), long processing points to your event handlers, and long presentation delay points to rendering cost after the handler. For LCP, comparing TTFB, resource load delay, load duration and render delay tells you whether to look at the server, the discovery of the hero image, its size, or render-blocking code. If you need to name the guilty script, the attribution build can also surface Long Animation Frame data, which only reports script entries above 5 ms.

Step 5: release-aware alerting

Owning the pipeline lets you catch regressions in hours, not weeks. Every row carries release, so compare the new build with the previous one:

for each (route, device, metric) with samples >= min_samples in both releases:
    delta = p75(new_release) - p75(previous_release)
    if delta > tolerance[metric] and p75(new_release) crosses the "good" threshold:
        alert(owner_of(route), route, device, metric, top attribution targets)

Compare like with like (same routes, device class and comparable time windows) and include the top attribution targets so whoever is on call starts from a selector, not a dashboard.

Common mistakes

  • Averaging instead of using percentiles. The mean hides the slow tail that the 75th percentile is designed to capture.
  • Mixing mobile and desktop in a single number, which hides mobile problems behind fast desktops.
  • Raw URLs as a dimension. Unbounded cardinality, slow queries, personal data in query strings.
  • Inserting every report as a new page view. Without upserting by id, page views with several INP updates are counted more than once.
  • Sending on unload or only once at load time, which misses late INP and CLS updates.
  • Loading the collector render-blocking, so the measurement becomes the regression.
  • Alerting without minimum samples, which trains the team to ignore alerts.

Implementation checklist

  • web-vitals attribution build loaded after the critical path
  • Session-level sampling with a configurable rate
  • Route template and release ID injected at build or render time
  • Flush on visibilitychange to hidden with sendBeacon and a keepalive fallback
  • First-party endpoint that answers 204 immediately
  • Allow-lists, value caps and size limits; dimensions derived server-side; no IP storage
  • Upsert (or latest-per-ID at query time) keyed by metric ID
  • p75 queries per metric, route and device with a minimum sample size
  • Release comparison alerts that include top attribution targets
  • Retention policy and a documented list of collected fields

When the pipeline lives inside a larger product, such as a multi-tenant store or a customer portal, design it with the rest of your observability stack; that is the kind of work we do in custom software development. For marketing sites, the same principles apply in our website design and development projects.

Frequently asked questions

Will my RUM numbers match CrUX exactly?

No. CrUX is built from real Chrome users over a rolling 28-day window; your pipeline uses your own browsers, sampling and windows. Use your data for diagnosis and CrUX as the external reference.

What sample rate should I use?

It depends on traffic and how fine-grained your groups are. Pick a rate that lets each important route and device class clear your minimum sample size, and keep it configurable.

Should I send RUM data to Google Analytics instead of my own backend?

It is a valid shortcut; the web-vitals documentation shows how to send metrics as analytics events. You trade control over dimensions, deduplication and retention for speed of setup.

Comments

No comments yet. Be the first to share what you think.

Leave a comment

It is never published.