The error "The URL … is not allowed: it must be a public http(s) address" means the address failed Wyregraf's safety check. To protect your server, URL Scrape only fetches pages the public internet can reach:
http://orhttps://only, on standard ports (80, 443, 8080, 8443).- No private or internal addresses —
localhost,127.0.0.1, LAN addresses (10.x,192.168.x,172.16-31.x), and other reserved ranges are refused. A public hostname that resolves to a private address is refused too. - Redirects are re-checked — a public URL that redirects to an internal address stops with "The URL redirected to an address that is not allowed."
This is deliberate (it prevents the scraper being used to probe your own network), so the common cases play out like this:
- Staging or intranet site? The scraper can't reach addresses only your network knows. Publish the page somewhere public, or paste its text into an Internal source embedding instead. Site developers can whitelist a known-safe internal host with the
pngx_wyregraf_url_guard_allowedfilter. - Blocked mid-scrape by a redirect? Check where the URL redirects (a login wall or a URL shortener hop is typical) and use the final public address directly.
A different message — "No content found, please try different url or selector" — isn't a block at all: the page fetched fine but the Selector matched nothing. Reset it to the default body and narrow from there.
