How to Test an Address API's Accuracy

To test an address API's accuracy, take real addresses you can confirm exist, damage them in controlled ways (typos, missing fields, reordering), add a few fake ones, and send the same random sample to every API. Then score five numbers: how often it recovered the right address, how often it was wrong without warning, how often it flagged its own doubt, how often it returned nothing, and whether it refused the fakes.

Clean addresses tell you almost nothing. Every serious API gets a well-formed address right. The differences show up on the input real users type, and on what the API does when it is not sure.

The five numbers to measure

Step by step

1. Build a truth set

Start from real addresses: your own customer data, or a public list. Run each one in its clean form first and keep only the ones that resolve to a real deliverable address. In our own benchmark only 72.8% of a raw sample passed this screen. Scoring the rest would measure the quality of your list, not the API.

2. Damage it in controlled ways

Generate variants of each confirmed address, one damage type at a time, so you can report results per type:

3. Add fake addresses

Take confirmed addresses and change the house number to one that does not exist on that street. A good API refuses these. An API tuned to always return something will snap them to the nearest real address and call it a match.

4. Sample at random, run everything the same way

Draw your test inputs at random from the full set of variants, not by hand, and send the identical sample to every API you compare. Save the complete response, including any match level, confidence score and per-field flags, not just the address.

5. Score the five numbers, per damage type

A short scoring loop is enough. This one works with any API that returns a standardized address and some confidence signal:

def score(results):
    """results: list of dicts with 'expected', 'returned', 'confident' (bool), 'is_fake' (bool)"""
    real = [r for r in results if not r["is_fake"]]
    fake = [r for r in results if r["is_fake"]]
    n = len(real)
    recovered = sum(r["returned"] == r["expected"] for r in real)
    silent    = sum(r["returned"] and r["returned"] != r["expected"] and r["confident"] for r in real)
    flagged   = sum(r["returned"] and r["returned"] != r["expected"] and not r["confident"] for r in real)
    empty     = sum(not r["returned"] for r in real)
    refused   = sum(not r["returned"] for r in fake)
    return {
        "recovery":        recovered / n,
        "silent_wrong":    silent / n,
        "flagged_wrong":   flagged / n,
        "empty":           empty / n,
        "fake_refusal":    refused / max(len(fake), 1),
    }

Normalize both addresses the same way before comparing (case, abbreviations, ZIP+4 present or not), or you will count formatting differences as errors.

How many addresses you need

Small samples make close results look different. The margin of error at 95% confidence for a measured accuracy near 95%:

For silent substitutions, which should be rare, you need thousands of inputs to see them at all. At 1 in 700, a 100-address test will usually show zero.

Mistakes that make results meaningless

• Testing only clean addresses. Every API scores near 100% on them. You learn nothing.
• Counting any answer as a success. An API that always returns something will look great on recovery and hide its silent errors.
• Ignoring the confidence fields. The same wrong answer is harmless if it is flagged and costly if it is not.
• Using the vendor's demo addresses. They are chosen to work. Use your own data.
• Reporting one blended number. A mix heavy on formatting noise will beat a mix heavy on typos. Report per damage type.
• Scoring unconfirmed addresses. If the clean address is not real, no API can recover it. Screen first.

Worked example: our own results

We ran this method on the sthan.io parser and published the results in August 2026: 29,966 random variants of screened addresses, 26,943 of them scored against confirmed real addresses. Full details are in the benchmark write-up.

Run the same test on any API you are considering, including ours. You can start with the Address Verification API free tier of 100 requests a month, or the Address Parser for freeform input.