spyyyder - the power of the link

I made a very tiny tool called Spyyyder that grabs a page, tells me what links are on that page, and then allows me to drill down through each of them. For getting to grips with how a site is put together, it's a pretty useful tool.

It groups them by internal links, subdomains, links to external sites, and any mailto: links.

I find it useful when researching competitors. For example, sometimes you'll find versioned url slugs, and that gives you a thread to pull at. I'm surprised at what it's possible to learn just from looking at the links on a page.

Spyyyder is pretty dumb. There are a bunch of tools out there that do this, and show you a pretty chart or whatever - I wanted Spyyyder to be really simple, and as fast as possible.

Under the hood

I've been keen to get to grips with TanStack Query; this seemed like a good project for it. Each URL you search for is a query key, and this means that you're never refetching a URL that's been seen before. It makes for a pretty snappy experience as you explore.

A Cloudflare worker using HTMLRewriter takes care of the CORS limitations and compiles the list of links. Originally I built this on top of Supabase Edge Functions, but Worker's "scale to zero" works a lot better for this kind of project that keeping a Supabase project "warm".

Give it a go: https://spyyyder.vercel.app →