Skip to content
pdfconversionapi
Menu

How we test

Every sample, timing and file size on our conversion pages comes from a script that calls the API, and records where it was made.

Last updated 2026-11-02

Contents (8 sections)

Limits and parameter behavior come from the code and from requests we sent. This page explains those scripts, what they record, and what we haven't tested yet.

The test server

The hosted API isn't open yet, so every test so far ran against a local container built from the same build file the hosted conversion pools are built from, with all converters in one image. Each sample and each timing records where it was made. So far that is a container on an Apple M1 Pro Mac, running aarch64 and limited to 4 CPUs and 4 GiB of memory.

It also records the image name and id. A build that changes the image gets a new id, so you can tell which build produced which file.

Sample files

Each conversion page offers the input file and the converted output for download. So far we wrote every sample on this site ourselves, and the companies, people and figures in them are made up. We publish them under CC0-1.0, so you can reuse them in your own tests. If a sample ever comes from another project's test files, its source line will name the project and its license.

Each sample's source line also says how its input was made. We used tools such as a desktop word processor, python-pptx, openpyxl and Pillow, hand-written HTML and SVG, and sometimes our own API, for example to render an SVG drawing as a PNG.

scripts/sample.mjs publishes a sample in one step:

  1. It copies the input into /samples/{page}/.
  2. It converts the input with the API, using the parameters shown on the page, and saves every output file.
  3. It makes the preview image with the same API. For a PDF, the first page becomes a JPG at 110 DPI.
  4. It writes manifest.json next to the files.

The manifest records the conversion, the parameters, the source and license of the input, the size and SHA-256 checksum of every input and output file, the environment and image id, and the date. You can check that the file you downloaded is the one the API produced:

Shell
curl -fsSO https://pdfconversionapi.com/samples/docx-to-pdf/manifest.json
curl -fsSO https://pdfconversionapi.com/samples/docx-to-pdf/output-quarterly-report.pdf
jq -r '.outputs[] | "\(.sha256)  \(.file)"' manifest.json | shasum -a 256 -c
Output
output-quarterly-report.pdf: OK

We don't edit output files after the API writes them. When we regenerate a sample, its manifest is rewritten with it.

Timings

The "Sample timing" on a conversion page comes from scripts/measure.mjs:

  • One warm-up request first, which isn't counted.
  • Then 5 timed requests, one after another, never in parallel.
  • The page shows the median of the 5 runs, and the measurement file keeps the slowest run as well.
  • The time is the ConversionTime the API reports: from the moment a conversion slot is free until the converter finishes. It leaves out the upload, any wait for a slot, building the response and the network.
  • Each measurement file records the input file and its size, the parameters, the output sizes, the environment and image id, and the date.

scripts/measure-all.mjs measures the pages one at a time, so two measurements never share the machine. We run it on a machine that isn't doing anything else.

Parameters

Parameter tables are written from our code. Before a conversion page is published, each parameter in its table is sent in a request to check that the output changes the way the page says, and each option the page lists as not supported is sent to check that the API refuses it.

You can check the rule yourself. If a request sets one of the options ConvertAPI documents for that conversion to a value other than its default, and our converter never reads that option, the API answers 400 and names it. For example, PDF to JPG doesn't support BitDepth:

Shell
curl -fsSO https://pdfconversionapi.com/samples/pdf-to-jpg/input-season-guide.pdf

curl -sS https://api.pdfconversionapi.com/v2/convert/pdf/to/jpg \
  -H "Authorization: Bearer $PDFCONVERSIONAPI_KEY" \
  -F File=@input-season-guide.pdf \
  -F BitDepth=8
Output
{"Code":400,"Message":"BitDepth is not supported yet for PDF → JPG"}

Options that ConvertAPI doesn't document for a conversion are ignored.

Each conversion page is stored with a list of the evidence behind its claims: the line of code, the test or the request. The list isn't shown on the page, and our content check refuses a page with fewer than three entries.

Code samples

Conversion pages show curl, Python and Node.js. scripts/test-samples.mjs runs all three for every conversion page before it is published:

  • It writes each sample to a file exactly as the page shows it.
  • It runs it in a new, empty folder that holds only the sample input.
  • It points it at our test server. The base URL is the only change.
  • It fails if a sample exits with an error or writes no output file.

Code in the docs and on pages like this one is run manually against the same test server before publishing, and the output shown under it is copied from that run.

We only show languages we can run this way. There are no PHP, C#, Java or Go samples because we have no runner for them yet.

What we compare our output with

The home page puts our Word to PDF output next to the same file converted by an open-source office suite. That is not Microsoft Word. We don't run Word, and a document can lay out differently there. We also never call a reference "the original": the original is the input file, which you can download.

For the checks behind each conversion page, the reference depends on the format:

  • Word, Excel, PowerPoint, CSV and DXF files: an open-source office suite, run offline from our own office image.
  • HTML, SVG, Markdown and email: an open-source web browser.
  • HEIC photos, and PDF pages rendered as images: the decoder and renderer built into macOS.
  • Other formats are checked against a standard tool for that format, for example dwebp for WebP and Pillow for PNG transparency.

Where both sides are images or PDFs, we render them at the same resolution, compare them pixel by pixel or side by side, and look at each area that differs. What we find goes into the page's list of what the conversion loses.

How we use AI

AI models help us research, draft and edit this site, including this page. They summarize sources, suggest outlines and first drafts, copy-edit, and write first versions of code and scripts.

A model never supplies a number, quote, test result, error message, biography or customer story, and we don't publish code that nobody ran. Numbers on this site come from script output, measurement files, the code or a dated source. A person on the team checks each claim against the code, a test or a source, runs the code samples, and reads the page in full before it is published. The tests are run by our scripts against our API, and the conclusions are ours.

Not tested yet

  • The hosted API. All timings and samples come from the local container.
  • Scheduled runs. Code samples run before a page is published, not yet every night.
  • Larger sets of real files. Each conversion page rests on one sample. We ran the engines behind our Word and PowerPoint conversions on 25 Word and RTF files and 20 PowerPoint decks from the Apache POI and Apache Tika test suites, and the engine behind our CAD conversions on 22 drawings from LibreDWG's test data, the ezdxf examples and that engine's own examples. Those runs used the engines' macOS builds, not our image, and we haven't published the files and results yet.
  • PDF to Word and PowerPoint layout. We have no benchmark for how closely our PDF to DOCX and PDF to PPTX output follows the PDF.
  • Microsoft Office as a reference. None of our comparisons use Word, Excel or PowerPoint output.
  • Outside review. Nobody outside the team has checked these tests.