Great-Circle Distance⚠ unverified

Understanding the Parameters

• A great circle is the sphere's "straight line" — the intersection of the sphere with any plane through its centre (the equator, and every line of longitude, are great circles; latitude circles except the equator are not). The shorter arc of the great circle through two points is the geodesic — the shortest surface path.

• Central angle $\Delta\sigma$ — the angle, at the Earth's centre, between the two radius vectors to the points. Distance is simply radius times this angle (in radians): $d = R\,\Delta\sigma$. One degree of central angle is about $111\ \text{km}$; one arcminute is one nautical mile by definition ($\approx 1852\ \text{m}$) — which is exactly why nautical miles are the natural unit of navigation.

• Why haversine, not the plain law of cosines — the spherical law of cosines, $\Delta\sigma = \arccos(\sin\phi_1\sin\phi_2 + \cos\phi_1\cos\phi_2\cos\Delta\lambda)$, is mathematically equivalent but numerically fragile: for nearby points $\Delta\sigma$ is tiny, its cosine is $\approx 1$, and $\arccos$ near 1 amplifies floating-point round-off badly. The haversine form uses $\sin^2(\cdot/2)$ and $\arcsin$, which stay well-conditioned for small angles — critical for computing short legs and cross-track errors.

• Latitude and longitude in degrees — geographic coordinates; the formula converts them to radians internally. $\Delta\phi = \phi_2-\phi_1$ and $\Delta\lambda = \lambda_2-\lambda_1$ are the coordinate differences.

• Sphere vs ellipsoid — the Earth is an oblate spheroid (equatorial radius $6378\ \text{km}$, polar $6357\ \text{km}$), so the single mean radius introduces up to ~0.5 % error. For flight planning that is negligible; for survey-grade geodesy, the ellipsoidal Vincenty formulae are used instead.

Derivation (Approaching a Proof)

The arc-length relation $d = R\,\Delta\sigma$ is definitional (arc length = radius × subtended angle, in radians). The real content is finding the central angle $\Delta\sigma$ from latitude/longitude — the haversine formula.

Place both points on a unit sphere and write their positions as 3-D unit vectors. The haversine of an angle is $\operatorname{hav}(\theta) = \sin^2(\theta/2) = (1-\cos\theta)/2$. Consider the two points with latitudes $\phi_1, \phi_2$ and longitude difference $\Delta\lambda$. The spherical law of cosines gives the central angle:

$$\cos\Delta\sigma = \sin\phi_1\sin\phi_2 + \cos\phi_1\cos\phi_2\cos\Delta\lambda.$$

Convert to haversine form using $\operatorname{hav}(\Delta\sigma) = (1-\cos\Delta\sigma)/2$. Substituting and applying the identity $\operatorname{hav}(\Delta\lambda) = (1-\cos\Delta\lambda)/2$ together with $\operatorname{hav}(\Delta\phi)$ for the latitude difference, the cross terms collapse (a standard spherical- trigonometry manipulation) to the compact law of haversines:

$$\operatorname{hav}(\Delta\sigma) = \operatorname{hav}(\Delta\phi) + \cos\phi_1\cos\phi_2\,\operatorname{hav}(\Delta\lambda).$$

Writing the haversines explicitly as $\sin^2(\cdot/2)$:

$$\sin^2\!\frac{\Delta\sigma}{2} = \sin^2\!\frac{\Delta\phi}{2} + \cos\phi_1\cos\phi_2\,\sin^2\!\frac{\Delta\lambda}{2} \equiv a.$$

Solve for $\Delta\sigma$ by inverting the half-angle sine (using $\arcsin$, which is well-conditioned near 0):

$$\Delta\sigma = 2\arcsin\!\sqrt{a},$$

and multiply by the radius:

$$d = R\,\Delta\sigma = 2R\arcsin\!\sqrt{\sin^2\!\tfrac{\Delta\phi}{2} + \cos\phi_1\cos\phi_2\sin^2\!\tfrac{\Delta\lambda}{2}}. \qquad\blacksquare$$

This is exactly the implemented formula. The genius of the haversine arrangement is purely numerical: it moves all the small-angle behaviour into $\sin^2$ and $\arcsin$, avoiding the catastrophic cancellation that $\arccos(\approx 1)$ would suffer.

Sanity check. From the equator point $(0^\circ,0^\circ)$ to $(0^\circ,1^\circ)$: $\Delta\phi=0$, $\cos\phi_1\cos\phi_2=1$, $a=\sin^2(0.5^\circ)$, so $d = 2R\cdot 0.5^\circ$ in radians $= 6{,}371{,}000\times(1^\circ\text{ in rad}) = 6{,}371{,}000\times 0.017453 \approx 111{,}195\ \text{m}$ — the familiar "$1^\circ \approx 111\ \text{km}$." ✓

Dimensional check. $\Delta\sigma$ is an angle in radians (dimensionless), so $d = R\,\Delta\sigma$ carries the metres of $R$ ✓.

History and Development

• Spherical trigonometry. The mathematics of great circles is ancient — Menelaus of Alexandria (~100 CE) and medieval Islamic astronomers developed spherical trigonometry for astronomy and for finding the qibla (the direction to Mecca), itself a great-circle bearing problem.

• The haversine (19th century). The haversine function was tabulated specifically for navigation because it made the law of cosines computable by hand with logarithm tables while avoiding the sign/precision pitfalls of small angles. José de Mendoza y Ríos (1795) and later navigators popularised haversine tables; the name "haversine" (half versed sine) dates from this era. It survived into the computer age precisely for its numerical stability, famously discussed in the navigation-software literature (R. W. Sinnott, 1984).

• Great-circle routing in aviation. Long-range flight made great-circle navigation routine: a New York–Tokyo flight arcs far north over the Arctic, looking bizarre on a Mercator map but tracing the true shortest path. Flight-management systems compute great-circle legs continuously, and the initial great-circle bearing changes along the route (unlike a constant-heading rhumb line).

• From sphere to ellipsoid. For precise geodesy the sphere was replaced by the WGS-84 ellipsoid, and Thaddeus Vincenty's 1975 iterative formulae give millimetre-level distances on it. The spherical haversine remains the workhorse for aviation, where its ~0.5 % error is immaterial.

Related Concepts: Initial Heading, Cross Track Error, Time To Go, Ground Track Velocity, Gravitational Force

Notes: Shortest surface path = arc of a great circle (sphere's geodesic). $d=R\,\Delta\sigma$; the implementation gets $\Delta\sigma$ from the haversine formula (numerically robust for small angles, unlike $\arccos$ of the law of cosines). Inputs in degrees; $R$ = mean radius $6{,}371$ km. Spherical model (~0.5 % vs the WGS-84 ellipsoid / Vincenty). $1'$ of arc $\equiv 1$ nautical mile; $1^\circ\approx111$ km.