Testing Methodology

How we verify that 126 converters actually do what their pages claim.

Fixture testing

During development, each tool family was tested by hand with three classes of sample files: ordinary documents, images and media; awkward edge cases (unusual page sizes, transparency, rotated pages, odd encodings, alternate streams and reversed page ranges); and deliberately broken inputs (truncated files, wrong extensions, encrypted PDFs, missing audio and zero-byte files). The engines are written to convert ordinary cases, handle awkward ones sensibly, and fail broken ones with a clear message instead of a wrong result. Committed fixtures exercise the browser file-inspection layer and the native media profiles through repeatable release scripts.

Engine-level verification

• Round-trips, enforced in the product on every conversion: Protect PDF re-opens its own output with your password and confirms it refuses to open without one before handing you the file; Unlock PDF confirms its output opens freely. If either check fails, the conversion fails.
• True redaction, enforced in the product: redacted pages are rebuilt as flattened images, and the output's text layer is then re-extracted — it must be empty on every marked page or the conversion fails instead of returning the file. You can confirm it yourself by running a redacted file through our PDF-to-Text tool.
• Repair: every repaired PDF is sanity-checked by rendering its first page before it is offered for download.
• Unit tests (npm run test:unit): 72 assertions covering the parts whose behaviour we publish claims about — the Settings Advisor returns byte-identical advice for identical inputs across every supported tool/goal combination (that is what "deterministic" means here, and it is checked, not asserted), every recommendation carries a reason and a trade-off, the error catalog maps real engine messages to the right codes and still returns a usable next step for an unknown failure, page-range parsing handles reversed and out-of-bounds ranges, and the data converters keep quoted fields intact while refusing YAML anchors.
• Strict data parsing: the data converters refuse rather than guess — malformed CSV/JSON/XML and unsupported YAML features (anchors, tags, directives) produce a clear error instead of silently mangled output. Conversions were spot-checked in both directions with sample files during development.
• Native media profiles: npm run test:media runs all six public profiles live against committed, copyright-free motion/tone fixtures through the same media-engine.js module the server uses. It checks EBML DocType or ISO-BMFF brands, SHA-256 evidence, promised codecs, a complete decode, duration tolerance, phone/camera rotation and temporary-file cleanup; the matrix exports hashes, codecs, bytes and timing as JSON/CSV.

Release gates

Every deploy runs a build-time audit that fails the release on: missing titles/descriptions/headings, wrong canonical URLs, duplicate titles, ads appearing on ad-free screens, more ad units than the per-page cap, broken internal links, near-duplicate page content, sitemap/robots/ads.txt inconsistencies, and any non-English locale leaking into production. A failed audit means the previous version keeps serving.

End-to-end checks

Around releases we manually drive the real product in a browser: drop a file, convert, open the download page, download the result. We have confirmed with the browser's network recorder that browser-side tools make no upload request, and repeat that check whenever a conversion engine changes. After each deploy, a committed smoke script (tools/smoke.sh in the project) checks the live site's canonical tag, robots.txt, ads.txt, the ad-free download page and one real server conversion.

Environment and limits

Manual testing is done on current Chrome (macOS), including mobile-viewport checks. Browser-engine behaviour is the same code served to visitors; native-media speed still depends on server hardware, which is why published elapsed time is evidence about the fixture run rather than a speed promise. Honest limits: we cannot test every dialect or codec, OCR ships 25 language models and reads one at a time, HDR and unusual media timestamps need source-specific review, and LibreOffice rendering of exotic Office features may differ from Microsoft Office — tool pages say so where it matters.

Published measurements live in our Conversion Labs — each report records its method, environment and limitations, and ships its raw data as a downloadable CSV. Product-level changes are tracked in the changelog.