Distance Calculator

How the Distance Calculator works, and what it assumes

Latitude and longitude are angles, not lengths, so the distance between two positions is never the Pythagorean distance between their degrees. A degree of latitude is about 111 km everywhere, but a degree of longitude runs from about 111 km at the equator to nothing at all at the poles. Any calculation that ignores that is wrong by a factor of 1/cos(latitude), which is already 2 at 60 degrees north.

Geodesic uses Vincenty's inverse formula on the WGS 84 ellipsoid: an iterative solution for the shortest path over an oblate spheroid, accurate to well under a millimetre. It has one known weakness, which is that the iteration does not converge for points on opposite sides of the earth; when that happens the tool falls back to the spherical figure and says so rather than returning a number it cannot stand behind.

Great circle uses the haversine formula on a sphere of the WGS 84 mean radius, 6,371,008.8 m. It is within about 0.5 % of the ellipsoidal answer, which is a few kilometres on an intercontinental route and centimetres across a town. Haversine is used rather than the spherical law of cosines because the law of cosines takes the cosine of a very small angle over short distances and loses most of its significant digits doing so.

Using the Distance Calculator

• Checking a distance produced by code that you suspect is treating degrees as if they were metres.
• Working out how far apart two sites, stations or assets are without opening a GIS.
• Getting a bearing for a radio path, a flight leg or a survey line.
• Deciding whether a simple spherical calculation is accurate enough for a job by comparing it with the ellipsoidal one.

A worked example

From London to Paris, 51.5072, -0.1276 to 48.8566, 2.3522:

Geodesic (WGS 84 ellipsoid)   343,897 m   343.897 km   213.688 mi   185.689 nmi
Great circle (sphere)         343,530 m   343.53 km    213.46 mi    185.492 nmi
Initial bearing               148.05° (SSE)
Final bearing                 149.95° (SSE)

The two methods differ by 367 m, about 0.1 %. The bearings differ from each other because a shortest path over a curved surface continuously changes heading: only a course due north, due south or along the equator arrives on the exact reciprocal of the bearing it set off on.

Where this comes up in GIS work

1. Enter the first position in the From box, in decimal degrees, DMS or degrees and decimal minutes.
2. Enter the second position in the To box.
3. Choose Geodesic for the ellipsoidal answer or Great circle for the spherical one.
4. Read the distance in each unit, and the bearings underneath.
5. Select Copy beside any unit to put that value on the clipboard.

When the output looks wrong

The distance is far larger than expected

The usual cause is a position with latitude and longitude swapped, which moves a point across the world. Check that each value is plausible: latitude never exceeds 90. Writing hemisphere letters removes the ambiguity entirely.

The two methods give slightly different answers

They should. The great-circle figure is computed on a sphere and the geodesic figure on the WGS 84 ellipsoid, which is about 21 km flatter at the poles than at the equator. The difference reaches roughly 0.5 %, which is several kilometres on an intercontinental route.

These points are very nearly antipodal

Vincenty's method does not converge for points on opposite sides of the earth, a known limitation of the algorithm rather than a fault in the input. The great-circle distance is shown instead and the result says so.

The final bearing is not the reverse of the initial bearing

On a sphere or an ellipsoid it usually is not. A great-circle path continuously changes its heading, so only a course due north, due south or along the equator has a final bearing exactly 180 degrees from its initial one.

Questions about the Distance Calculator