← Back to the library
SecurityDeep dive · 9 min

A "fetch this URL" feature reaches internal services

“A link preview, webhook tester, image importer, or PDF renderer accepts a URL, and someone points it at localhost, an internal admin port, or the cloud metadata endpoint and gets the response back.”

THE PRINCIPLE

Validate where the connection actually goes, not what the URL string looks like, and assume the application check will one day be bypassed.

FIRST MOVES

  1. Prefer an allowlist of hosts and build the outbound URL yourself, with scheme, port, and path fixed by the application.
  2. Otherwise resolve the hostname and accept only public unicast addresses (reject private, loopback, link-local, unique-local, CGNAT, IPv4-mapped and NAT64 ranges, and IP-literal hosts that skip DNS), then connect only to the address you validated (DNS rebinding).
  3. Disable automatic redirects, or re-run the full check on every hop.
  4. Require IMDSv2 on EC2 (HttpTokens=required) and keep the PUT response hop limit at 1 unless containers need more.
  5. Run fetchers in a segment whose egress policy or proxy only allows the internet, never internal ranges.

TOOLS: THEN → NOW

Regex or deny-list on the URL stringHost allowlists plus resolved-IP validation and pinned connections
IMDSv1 reachable by any local requestIMDSv2 required, enforceable at the account level
Application checks onlyPlus an egress proxy or network policy for fetcher workloads

PATTERN SNAPSHOT

import { lookup } from "node:dns";
import type { LookupFunction } from "node:net";
import ipaddr from "ipaddr.js";
import { Agent, fetch } from "undici";

const ALLOWED_HOSTS = new Set(["images.partner.example", "cdn.partner.example"]);

// Validate inside the socket's own DNS lookup, so the checked address is the
// one actually connected to: no rebinding window between check and connect.
const publicOnly: LookupFunction = (hostname, options, callback) =>
  lookup(hostname, { ...options, all: true }, (err, addrs) => {
    if (err) return callback(err, "");
    // parse() keeps ::ffff:127.0.0.1 as "ipv4Mapped", so it is rejected too.
    const bad = addrs.find((a) => ipaddr.parse(a.address).range() !== "unicast");
    if (bad) return callback(new Error(`Blocked address ${bad.address}`), "");
    if (options.all) return callback(null, addrs);
    callback(null, addrs[0].address, addrs[0].family);
  });

const pinned = new Agent({ connect: { lookup: publicOnly } });

export async function fetchPartnerUrl(raw: string) {
  const url = new URL(raw);
  // Node skips lookup for IP-literal hosts; the allowlist keeps those out.
  if (url.protocol !== "https:" || url.port || url.username || !ALLOWED_HOSTS.has(url.hostname)) {
    throw new Error("Destination not allowed");
  }
  return fetch(url, { dispatcher: pinned, redirect: "error", signal: AbortSignal.timeout(10_000) });
}

CLOSE THE AI. EXPLAIN THIS.

Your allowlist checks the hostname and then calls fetch(url). How can DNS rebinding still reach 127.0.0.1, and what closes the gap?

HOW IT WORKS UNDERNEATH

SOURCES

Guide reviewed