Discovery and Rediscovery in Spatial Data
Knowing what is where, how to get there, what it's like, and why
Polar expeditions have a fascinating and inspiring history. Both South Pole and North Pole, before being visited, were places that were known to exist as something of a distinct point in space, and could even be plotted on a globe or other map, as if the location of either also was no mystery. To navigate to either, at a very high level, one only needed to follow the magnetic compass to north or south. The only remaining aspect of a polar expedition—with the need to search and scour the earth for the destination removed—was to get there.
There are probably many things in the realms of industry, technology, research, science and beyond where conceptually something is known to be possible, or to exist, or generally that the math works out. Coming to such conclusions is not always easy, and sometimes is a result of a long period of trial and error, bricolage, and cumulative knowledge building. In terms of plotting the North and South Poles, it’s arguable that all of recorded human history, with the development of mathematics, navigation, the will to explore and record observations, the advancement of seafaring and medicine, was the foundational work toward making a Polar expedition possible. Doing the final stretch of work, however, becomes a logistical operation, and exercise in courage and endurance, and a battle against forces of nature.
The poles are not just one single thing. Years and years ago, my first deeper introduction to truly using maps to navigate was in the form of what the United States military calls “land navigation”, where new recruits or cadets like me learn to plot longitudes and latitudes, paying close attention to the scale of the map, and then plan routes on how to reach a point in the forest or on a mountainside from the other side of a landscape. Estimating the route often involves making use of roads or trails where possible—far faster than traveling over open ground with stone and vegetation, and a practice called “handrailing”. The final stretch can be very different, when the navigator is a few hundred meters distance from the target, and crossing the open ground becomes the only route.
Using the magnetic compass in hand, one must align to the map, knowing they need to face a certain bearing and then count their steps, which are equivalent to some distance in meters. On the map, north is zero degrees, called “grid north”—basically “up” on the map (which is projected onto a flat surface and not a sphere). This is similar to but different from true north, which where the line of longitude extends to the pole on a globe. The compass, however, points to magnetic north, wherever the magnetic field pulls the compass arrow. This is not on the tippy top of the globe.
The magnetic pole even wanders over time, so wherever a polar explorer arrived—at the place where the compass would be pointing always south while surrounding the pole—it would be a different location than someone one hundred years later. Navigating with reference to the wrong direction, for example not converting the degrees toward magnetic north to a grid north angle, could lead a land navigator to the wrong location, drifting too far left or right even over distances measured in single digit kilometers. Knowing that north is not always north can very well make or break the results of a land navigation exam and waste precious time, or get someone lost entirely.
But where did polar expeditions go? To which pole? Roald Amundsen, the legendary Norwegian explorer, reached the geographic South Pole in 1911—not the magnetic pole. Robert Falcon Scott reached it in 1912, while racing Amundsen and trailing behind. In 1926 Amundsen would also fly over the geographic North Pole. The expeditions on foot relied on various forms of navigation, including use of a compass until it became unreliable, but also keeping track of the stars and the sun and making frequent measurements, while also calculating longitude and latitude with a chronometer and a sextant, keeping course toward 90 degrees south (or north). The methods used here were an incredible feat of using the on-paper information about the world—the map—to actual arrive at the location that was mapped but not yet observed.
As magnetic poles continue to drift, they need to be remeasured to confirm their changing locations. It is a familiar theme in the field of geospatial data, geodesy, earth science, and GIS. The world’s landcover and the observations on it also frequently change. Businesses open and close, natural disasters and war can wipe away what was built, vegetation and coastlines morph and move. Borders need resurveying, ground control points need recalibration, the shape of the earth’s surface even warps.
While these things are constantly remeasured, all of them had once instance, somewhere in history, where they were known for the first time—the first observation or measurement. This can be as simple as the time that a particular fountain, bridge, or hardware store was added to OpenStreetMap, or discovery of new mountain peaks, or a new passage through an archipelago.
Discovery is one thing, and mapping is another. The first visit to Antarctica may very well have been by a Māori or Polynesian seafarer, whose oral tradition has possible references to the frozen southern landmass, while the first proclaimed sightings to the wider world were by both a Russian and America expedition, and the first mapping or charting was a British expedition, all around the same time. British Captain James Cook, of fame for his Pacific voyages to not just map and explorer but to help make astronomical observations of the transit of Venus in support of a project back home to calculate the distance from the earth to the sun.
Later, James Clark Ross mapped various parts of the Antarctica coast (which areas being named for him like the Ross Sea), and the United States launched Operation Highjump in 1946 with a naval and air presence, mapping the coast, and finally in the 1970s satellite images began to capture earth observations.
The important aspect of all of this is that documentation, charting, plotting, mapping, recording, and other ways of gathering measurements and evidence help to make knowledge about a location and what is there become retrievable data for others in the future. There are likely many places on earth, as well as aspects of science and technology, where someone discovered something but never recorded it, and the knowledge was never transmitted. In other cases, knowledge was recorded but lost, for example historical or geographic records in the burned Library of Alexandra or the destroyed Mayan codices.
A valuable aspect of consumer apps like Google Maps or Perplexity today is that they help solve a very particular human problem: how to change the unknown into the known. When you want to know the location of pizzerias around you, how to reach one by foot or car, or inspect the coastline of a lake for interesting beaches, Google Maps is a powerful tool. Without it, this information would be completely unknown, and you would have to find it by other means—probably possible, but with more friction and time. It could take months of living somewhere to know all the pizzerias or beaches, and get an idea of the best ones for your taste, while navigation is something you would have to find by trial and error and improvisation, or by shadowing a local, a taxi driver, or running painstaking calculations on your own with a paper map and a ruler.
With chat-enable LLMs, there is a lot of data and an underlying model—similar to Google Maps having data and a model of the earth, as well as search algorithms and more—but the icing on the cake is the user interface that allows you to simply ask a question. You can check who was President of the United States in 1893, ask about economic statistics of Japan, or get instructions to cook pesto lasagna. Sure, it can hallucinate—and map data an be wrong—but the process of discovering answers that tend to be good is faster and perhaps more reliable than a purely human way of doing it. Someone in 1925 may have made several telephone calls, visited a library or two, opened encyclopedias and cookbooks, went to visit university archives, written a few letters, and knocked around the neighbors’ houses to get answers to such questions.
You can withhold judgment on whether these are good things or not, but they are certainly advances in human capability. In relative terms, these types of technology have vastly closed the gap in time and effort required to turn “not knowing” into “knowing”, and it is almost as if we are able to will ourselves into having any knowledge we want. Of course, complex things are harder to truly understand just by having source material rapidly retrieve, and skills still require learning—but discrete facts like where something is, how far away, what it is like (the vibe), and how to get there, are almost instantly acquired.
For us to acquire information so instantly like this, the user interface has to be there, but so does the underlying data. That data must be discovered and captured by someone for the first time ever, before it can be copied a billion times and mirrored into the minds of all humanity. Every single day new observations go into Google Maps and OpenStreetMap for the very first time—locations of things on the face of the earth which may have existed already for decades or centuries, but never been mapped so that someone else can find it without a more informal and social inquiry.
This is slightly different from new events unfolding in real time, where new history is written in every news article—only because it is happening now. This too is remarkable, and of course it is remarkable that volcanic eruption on a remote Pacific island can be observed quickly by various sensors and be shared in social media within the hour, rather than perhaps going completely unobserved, or being witnessed by locals or passersby and perhaps a decade later entering the record of some diary transcribed into a public archive. That is a current event being made retrievable in real time, but mapping things that may have existed for a long time, but remained outside of retrievable digital knowledge, is a different situation, when something out there is very real in a physical sense but finally becomes official in its digitally captured existence.
Taking the physical world and its attributes, and digitizing them is an act of rediscovery. It takes something that may be known in the minds of few, and opens the potential for it to be known instantly by everyone. Transfer of knowledge from human to human is an ancient and powerful way to interact, and very organic. Using the digital database as an intermediary—human to database, database to millions and billions—is new, less organic, yet vastly more precise and immutable. Locations and facts enter a sort of knowledge graph in this latter way, their past and present status being tracked, and essentially becoming not only knowable to anyone with access, but unable to be forgotten.
Decentralized GIS datasets have existed for many decades, and collections of paper maps and other written records with a location element have been around for thousands of years. The ability to coalesce these into a single graph of knowledge is much more recent, and also incomplete. The process of finding what information is out there somewhere, and then conflating it against other instances of the same thing, is still quite computationally complex. Google Maps is a “one map” concept, as well as OpenStreetMap, where any single entity can have many upstream sources from different times and different observers, but eventually Google Maps and OSM aim to be comprehensive and unified.
Anything entering these map databases for the first time becomes an eternal piece of data. Sometimes that data is out there waiting to be rediscovered, in an old hard drive or court document or land record. Other times, nobody ever wrote down the location of something, like a colloquially known hot spring, a mountain with no written or spoken name, or an ancient rock art site that no living person has ever seen, but may be stumbled upon one day soon—all these can be mapped and made immortal, discovered and charted for what may be, or feel like, the first time.
There is a whole world out there, we can known so much about it instantly. It is only the logistics of getting there—making the journey and seeing with our own eyes—that stand between the instantly sense of knowing where and what, and the more gradual experience of understanding and experiencing, that data cannot convey in full.
PS: I made an Antarctica t-shirt for myself inspired by the research for this post, you can get your own here.
I was very interested in the Arctic topics after listening to the Belgica Expedition and Roald Amundsen episodes of the Explorer’s Podcast, which has tons of other fascinating episodes.