geowright
DocsTesting evidence

Testing evidence

The third-party apps geowright is tested against, what they changed in it, and the repeat-run bar its suite passes.

Two separate records: the apps geowright is tested against, and how often its complete suite passes in a row. Neither is a promise that every app or machine will behave the same way.

Apps it did not write

Tests against the library's own demo app carry its assumptions. So a separate suite runs 33 apps nobody working on geowright wrote, 135 specs in all:

  • 23 of MapLibre's own examples;
  • 3 react-map-gl apps and 1 deck.gl MapboxOverlay app;
  • 2 svelte-maplibre-gl apps and 3 vue-maplibre-gl apps;
  • 1 Protomaps PMTiles app.

Each is vendored as upstream plus exactly one added line, the hook. Where something was awkward, the library changed or the limit was written down; the apps were never bent to fit. They are test subjects, not users or endorsements.

What they found

17 worked untouched and 16 after a fix or addition in the library; 0 could not be tested. Those fixes were made by people who could see the failing test and the library's source at the same time, where an outside team might have filed a bug instead. So the fair reading is not a zero failure rate: roughly half of these unmodified apps exposed a real gap.

Among the changes they forced: aiming at 3D buildings where they are drawn, telling apart features with no id and identical properties, bounding the search on large shapes, working around MapLibre's viewport query on a globe, keeping modifier keys held during a drag, querying by layer type or source for wrappers that generate layer ids, scrolling a map below the fold into view, toHaveLayerPixels, and naming a missing icon image.

Some candidates were left out for licensing reasons. Apps that load an OpenStreetMap-based or commercial basemap run against a synthetic one with the same URLs, source and layer names, and label density kept within 0.6–2× of a real OpenFreeMap style's.

The repeat-run bar

Separately, the complete suite of 349 tests passed 20 repeats, 6,980 of 6,980 executions, with 0 flakes, 0 retries, and 13 opt-in tests skipped. It has passed this bar repeatedly as it grew. How the suite stays repeatable.

Versions

A version matrix runs one small page and a core set of specs against Mapbox GL JS 1.13 and MapLibre 2, 3, 4, 5 and 6, installed side by side. Globe, terrain, touch, drawing tools and the third-party apps are exercised on MapLibre 6 only.

Read more