mapbench

How to Calculate Distance Between Two Coordinates

The haversine formula explained step by step, with worked examples in kilometres, miles and nautical miles — and the cases where the spherical answer is not good enough.

Aryan KhanNovember 2, 20263 min read
In brief

The haversine formula explained step by step, with worked examples in kilometres, miles and nautical miles — and the cases where the spherical answer is not good enough.

  • New York: 40.7128° N, 74.0060° W → φ₁ = 0.7107 rad, λ₁ = −1.2917 rad
  • London: 51.5074° N, 0.1278° W → φ₂ = 0.8989 rad, λ₂ = −0.0022 rad
  • Δφ = 0.1882 rad, Δλ = 1.2895 rad
  • a ≈ 0.0693, c ≈ 0.9289, so distance ≈ 5,570 km

Two coordinate pairs describe two points on a sphere, and the distance between them is an arc, not a straight line on paper. The standard way to compute it is the haversine formula: it turns both latitudes and the longitude difference into a central angle, then multiplies that angle by Earth's radius. You can do it by hand in a few minutes, or enter the coordinates into a straight-line distance calculator and read the result in kilometres, miles and nautical miles at once.

The formula, in the form you can actually type

Convert every angle to radians first. With φ₁ and φ₂ as the two latitudes and Δλ as the longitude difference, haversine is: a = sin²(Δφ/2) + cos φ₁ · cos φ₂ · sin²(Δλ/2), then c = 2 · atan2(√a, √(1−a)), and distance = R · c. Using the mean Earth radius R = 6,371.0088 km gives the WGS84-consistent answer most mapping tools report.

A worked example: New York to London

  • New York: 40.7128° N, 74.0060° W → φ₁ = 0.7107 rad, λ₁ = −1.2917 rad
  • London: 51.5074° N, 0.1278° W → φ₂ = 0.8989 rad, λ₂ = −0.0022 rad
  • Δφ = 0.1882 rad, Δλ = 1.2895 rad
  • a ≈ 0.0693, c ≈ 0.9289, so distance ≈ 5,570 km
  • In statute miles that is about 3,461; in nautical miles about 3,008

Why the sphere is usually good enough — and when it is not

Earth is not a sphere; it is an oblate ellipsoid, flattened at the poles by about 0.3%. Haversine on a sphere therefore differs from a full geodesic solution such as Vincenty or Karney's algorithm by up to roughly 0.5% on long routes, and much less on short ones. That is larger than the ambiguity in where a city centre sits and smaller than the difference between a road route and a great circle, so for everyday comparison the sphere is fine.

It stops being fine when the number has legal, engineering or navigational weight. Surveying, property boundaries, flight planning under a regulatory standard and marine passage planning all use ellipsoidal or datum-specific methods. In those cases the sphere result is a sanity check on the real figure, not a substitute for it.

Three mistakes that show up constantly

  • Swapped axes. GeoJSON and many APIs store [longitude, latitude]; spreadsheets and humans usually say 'lat, long'. A swap puts the point in the ocean, not an error message.
  • Mixed units. One decimal-degree pair and one DMS pair in the same formula silently produces nonsense. Convert everything to decimal radians first.
  • Great-circle versus road distance. A route is always equal to or longer than the arc. Comparing the two is useful; confusing them is the most common reporting error in geographic work.

Which output to quote

Kilometres for international or scientific work, statute miles for US road context, nautical miles for anything that floats or flies. Report the unit, the Earth radius you used and the coordinates to the precision you actually trust — five decimal places is roughly metre-scale, which is already finer than most consumer GPS fixes.

Useful tools for this topic

Continue with the actual map tools

AK

About the author

Aryan Khan edits MapBench with a focus on practical map calculations, coordinate systems, routing concepts, open geographic data and clear limitations. Guides are written to help a reader make a better decision, not just to repeat a definition.

Related reading