Sennable

Portfolio / Scraping & data pipelines

Digital Builder

A niche-research tool that fans one query out to four live public data sources — and refuses to display a number it didn't just fetch. A local rebuild of a product that gates the same research behind $44–$149 a month.

Built and run by Joel Wickremasinghe

running in production

21

bugs found and then independently confirmed by 25 adversarial review agents — none rejected

measured · review workflow artifact

Problem

What needed to exist

Choosing a niche is the decision that determines whether months of content work compound or evaporate — and the tools that inform it are subscription-gated and rate-limited. The commercial equivalent, digitalmaker.ai, caps its paid tier at 15 competitor scans a week and charges up to $149 a month for the privilege.

The deeper problem is what those tools display. Research products in this category habitually show confident, plausible-looking figures with no stated source, which is precisely how someone commits a year to a niche on the strength of a number nobody computed.

What we built

The system

A local Express server and a vanilla-JS front end — no cloud, no database, no build step, no account. One typed topic fans out in parallel to four live public sources: a 42-query Google Autocomplete sweep, Google Trends through its undocumented widget-token flow, twelve months of real Wikipedia readership, and YouTube view and channel data parsed out of the public page payload.

Those four feed three sub-scores — demand, keyword breadth, competition — that combine into a 0–100 opportunity score, which is then run through a published-benchmark ad-rate table by niche and audience geography. Four generators can turn the result into an ebook, planner, spreadsheet, or script, and the prompt builder targets ten AI models across text, image, and video.

The rule the whole thing is built around: no mock data anywhere. Every research figure is fetched at request time, and when a source fails the interface names the source that failed rather than quietly substituting a plausible value. Google Trends rate-limits this address often, so demand degrades to live Wikipedia readership — and the report says which one it used.

Architecture

Digital Builder — one query, four live sources

Digital Builder — one query, four live sourcesPARALLELFALLBACK PATHSEED TOPICtyped by operatorAUTOCOMPLETE42 probes/seedGOOGLE TRENDSwidget token flowWIKIPEDIA12mo readershipYOUTUBEpage payload parseSCORE ENGINEdemand · breadth · compEARNINGS MODELrpm × geo × tierREPORTsources labelled

Stack

Built with

  • Node.js 22.16
  • Express 4.22.2 (only dependency)
  • Vanilla JS · HTML · CSS
  • Hand-rolled SVG charts
  • Google Autocomplete
  • Google Trends (undocumented endpoints)
  • Wikipedia Pageviews API
  • YouTube page parsing
  • Anthropic Messages API (optional, user's key)

Outcome

Numbers, with provenance

21

bugs found and independently confirmed by 25 review agents (0 rejected) — 7 major, 14 minor, across 1.43M tokens and 187 tool calls

measured · review workflow artifact

4 / 4

live sources returning HTTP 200 when measured — autocomplete 2.67s, trends 3.32s, Wikipedia 1.69s, YouTube 1.69s

measured · timed HTTP calls, 2026-08-01

2.0–3.0s

end-to-end for a full four-source report across three test seeds (2,041 / 2,829 / 3,044 ms)

measured · timed /api/niche calls

392

unique keywords surfaced from 42 autocomplete probes for one seed — varies per run, because it is fetched live

measured · /api/keywords response

12 / 12

regression tests passing

measured · suite run 2026-08-01

9 / 9

text contrast ratios at or above 4.5:1 — WCAG AA across all three themes

measured · computed from live DOM

0

mock, cached, or seeded values in the research path — every figure is fetched at request time or its source is named as failed

architecture · no-mock-data rule, 11 API routes

1

npm dependency across 2,467 lines of application code — no framework, no bundler, no charting library

measured · npm ls --depth=0

[needs: users, sessions, or products built]

there is no telemetry and no run log — the tool doesn't record what it produced, so there is nothing to report

Every figure above is read from the system of record it describes. Gaps are marked, not filled in.

Also in this system

Supporting detail

The bug that proves the rule

The niche classifier used substring matching, so "ai" matched inside "email" and "domain" — which classified personal finance as Technology & Software, applied the wrong ad-rate band, and produced wrong earnings figures. Found and fixed on 1 August 2026 with whole-word matching and a test. It's worth stating plainly: this is exactly the class of silent error the no-mock-data rule exists to catch.

Degradation is designed, not accidental

Google Trends rate-limits this address frequently. Rather than fail or fabricate, the app scores demand from live Wikipedia readership and labels the report with which source it used. The honest cost: the demand score isn't always computed the same way between two runs of the same query, and the interface says so.

Reddit was cut, not faked

A fifth demand signal was planned. Reddit returns HTTP 403 to this machine on every endpoint tried, so it was removed rather than filled with a placeholder. A missing source is a smaller lie than an invented one.

What isn't proven

The four AI generators are wired to the Claude API but have never completed a run — no key is configured on that machine, so only the failure paths (400 on malformed, 401 on invalid) are tested. The ad-rate estimates have never been checked against a real AdSense account and are labelled as benchmarks, not predictions. And the regression suite lives outside version control, so a fresh clone loses it.

Need data you can actually defend?

Four live sources, zero mock values, and an interface that names the source that failed instead of hiding it. We build collection pipelines that degrade honestly — because a confident wrong number costs more than a visible gap.