Microsoft Flight Simulator and the quest to replace Earth

1 day ago 2

Early on in Terry Pratchett's novel The Light Fantastic, a spell is cast to map the world. It begins as a "fireball of occult potentiality," dangling in the Great Hall of the Unseen University, which evolves into a ghostly "embryo universe." The embryo expands "lightly as a thought," with spectral continents "sleeting" through walls and people. It surges across the landscape until the entire population and geography of the Disc is exactly duplicated and enclosed by a shimmering shadow-self of "shining threads that followed every movement."

Real-life software developers, including video game developers, have been casting spells like this for decades. One of them is the Microsoft Flight Simulator series, whose latest incarnation is a full-scale, always-online representation of the Earth's surface, derived from a mixture of satellite imagery and terrain generation supplied by Microsoft's Bing mapping and Azure AI technologies. Like the embryo universe from The Light Fantastic, this Earth is getting more Earth-like over time, albeit much slower. As of 2024, you can land your aircraft and perambulate around a 3D landscape that, while riven with uncanny omissions and glitches, does a fair job of satisfying the digital tourist. What was once a game about slipping surly bonds has become that most despised of genres, a walking simulator.

One question I have for Microsoft Flight Simulator head Jorg Neumann is whether he envisages an upper limit for the simulation's fidelity. At what point will this phantasmagorical planet be Earth-like enough? It sounds like there's not much further to be gained from bumping the resolution of the environment. "We increased the resolution of the ground terrain 4000 times, between 2020 and 2024," Neumann tells me in an interview, just before the 2024 edition's release. "And when we turned on the terrain wireframe, at first we actually thought we had the textures, because the wireframe is so dense, that it actually looks like a mesh with textures. Do we really need more resolution than a few millimetres? No, I think on that side, we're fine."

An evening view of the statue of Christ the Redeemer on a mountainside above Rio de Janeiro, Brazil, as depicted in Microsoft Flight Simulator 2024. There's also a blimp on the righthand side flying over the city lights. Image credit: Microsoft

Neumann does, however, see potential for growth in terms of the technology that supplies the imagery on which Microsoft Flight Simulator is based. Satellites, he suggests, will eventually be able to fly lower than they currently do, and will be armed with better optical instruments. The 2024 game calls upon 50 centimetre resolution photography, but Neumann says that 30 centimetre resolution imagery is already available for certain regions. Drone photography, meanwhile, might allow for footage that would be impractical to obtain by human pilots.

"Right now, you either take satellite imagery which is really high and then you have a bunch of refraction in the atmosphere, or you fly a plane," he continues. "For the plane you need to hire a pilot, there needs to be enough fuel, you know - it's kind of limited from the logistics perspective, what you can get. So that's why some places on Earth are much harder to get higher resolution data for, but with these drones that people fly around nowadays, maybe - we think that there's going to be another step."

The prospect of incorporating user drone footage is a new threshold in Flight Simulator's relationship with its community - and thereby, a reminder of its debts to non-game consumer mapping software such as Bing Maps. As with those smartphone mapping tools, the latest versions of the game depend on an intricate, "participatory" relationship between a corporation, which creates or licenses the terrain data, and users, who fill out, hone, adorn and perhaps even deface the maps, albeit within firm constraints.

As Scott McQuire explains in a 2019 paper, corporate digital mapping tools are unique in being both imposed from above and contested from below, a tug of war that unfolds in real-time as the software is used, and that largely works to the benefit of the corporation. Amongst other factors, he links the success of the market-leader Google Maps to a strategy of releasing the tools to third parties, from rival entrepreneurs to ordinary users, then making their various alterations part of the product. Google Maps today is both highly editable, with the ability to add reviews and personal photography, and a passive data-harvesting service that learns and profits from the motions of billions of users worldwide.

If the editability and dynamism of digital maps allows for a measure of "popular cartography", McQuire notes, it's important not to succumb to any utopian people-power fantasies. "Public participation, such as the provision of co-created content, was crucial to the success of Google Maps," he writes. "But it is Google's structured shaping of such participation - enabling it while setting constraints on it - through a combination of technical, cultural and legal protocols that defines its particular enterprise." Digital maps may be user-modifiable, but they also exert technical and legal control over any changes you make to them. And if there is a commitment to fidelity, this is shaded by commercial priorities, such as the injection of advertising and thus, the definition of public space in terms of market forces.

A blue helicopter flies over a rainy city. Image credit: Rock Paper Shotgun/Xbox Game Studios

In Microsoft Flight Simulator, you are always, in a way, tinkering with the map, much as you're contributing data to Bing Maps every time you prompt it for directions. The game generates and streams terrain to you on demand, in ways familiar from other video games - for example, it displays more detail in the direction you're facing, with generated terrain data then beamed to other players visiting the same region to conserve processing power.

Whilst fundamentally a bid to minimise installation sizes (which might otherwise stray into the realm of hundreds of gigabytes), the reliance on streaming is a show of how playing Microsoft Flight Simulator is an act of "co-curation." Being generated on demand, the game's Earth is an evolving, combined reflection of the flightpaths and fascinations of its players. But this reflection is, nonetheless, tightly circumscribed. As Neumann explains, much of the data in Microsoft Flight Simulator is "rented" and refreshed over time, so even if it were physically practical, the developers aren't permitted to ship the game's Earth as a "permanent" download. "We are not really allowed to download it anyways. It's kind of how Bing Maps works - if you look at Bing, you don't download those textures either, right?"

These mundane legalities notwithstanding, the relationship between player and developers in Microsoft Flight Simulator is often a playful one, with an emphasis on active collaboration. Neumann recalls showing a trailer and being contacted by a player immediately afterward as he left the Los Angeles Convention Centre. "It was some person I'd never even heard of, I don't have any idea how they got my phone number, and they said, hey, I just watched your trailer and I'm from Hobart, Australia and it looks great, but there's something wrong with the colours."

The developers have been fielding feedback like this for years - as Neumann notes, one of the first things players do in Microsoft Flight Simulator is plot a course for their own neighbourhoods. He's keen to make such exchanges evermore the heart of the game, hence the interest in drones.

"I actually had an initiative, early on in 2024's development, that I called 'Let's build the world together'," he says. "So for example our lead artist, he lives in a little village in France, and he took a drone and he captured his area in France. And theoretically, we could be able to merge that into our world." There are immense practical difficulties to "stitching" drone photography into the world, not least the question of lighting in a simulation with a real-time day-night cycle. But Neumann is sure they can be overcome. "I think in the long term, we're going to revisit this, because I think it's the right thing to do."

The emphasis on community goes hand-in-hand with an ethic of preservation. Neumann sees Microsoft Flight Simulator as a digital archive, beginning with its dozens upon dozens of new and old licensed aircraft. "I go to tons of museums, and a lot of the curators were literally saying, hey, can you help us preserve this for prosperity?" he says. "We've scanned their planes, and we put it in the sim, and it kind of sits there forever, basically."

A huge aircraft takes off next to a lake. Image credit: Rock Paper Shotgun/Xbox Game Studios

The preservation angle extends to places and periods, fossilised within the amber of older versions of the game. "We have New York from 2015, we have New York from 2020, we have New York from 2024, and that timeline is really cool, you know?" Neumann goes on. He posits a system that would let people switch between eras in the game, so as to witness the geographical and architectural changes from sequel to sequel. "I think that's neat because I always say, I have two daughters and despite my best efforts they have no interest in looking at an atlas, ever, and I think something like this can actually help revitalise the curiosity about our planet."

In particular, this might allow Microsoft Flight Simulator to visualise the effects of climate change, a slow-moving apocalypse that remains intangible to a lot of people, and is thus easy to cynically decry as make-believe. Microsoft Flight Simulator 2024 might be a tool, he goes on, for recording environmental alterations caused by rising global temperatures and knock-on effects.

"We need to think about it, but there is, certainly in my head, a plan to do some kind of planetary visualisation," Neumann goes on. "For example, NASA has an entire array that looks just at pack ice. There's another array that is public, actually, for wildfires. Can we get that? One of the things that we have in Flight Simulator is that we can basically take any real-time data source and put it into the sim, and make that a reality."

There's a familiar, sinister cast to all this benevolent talk of curiosity and preservation - which, lest we forget, works primarily to the benefit of one of the world's largest corporations. It's less to do with Microsoft Flight Simulator, in fairness, than with the age-old imperial politics of maps. Maps aren't just neutral navigational aids: they are skewed representations that emphasise certain details and minimise or erase others, teaching the voyager to interact with their surroundings in certain ways. Networked digital maps that seem participatory and available to modification, but are in practice directly or algorithmically governed by big business, are only the latest weaponisation of mapping as a projection of power.

Neumann's enthusiasm for ever-increasing fidelity and naturalism is both innocent in itself, and in keeping with the rapacity of colonial pioneers venturing into "uncharted" territory. "I jokingly said at our an event a few weeks ago that I actually was really pushing to get butterflies in the game, because I want people to go outside the plane and actually collect some stuff," he remarks. "I'm a collector, right? I collect all kinds of stuff, like rocks and things. Why not? Could be cool. I have my Indiana Jones hat on, you know. I always wanted to fly my double decker down to some river in the Amazon and have mosquitoes around me, and it needs to sound just right - we want to immerse players. We want it to feel as close to the real thing as we can get."

One contribution of Microsoft Flight Simulator is to offer up an Earth in which such Jonesian urges can be sated without people flying actual planes down to endangered rainforests in search of butterflies. But these representations aren't immaterial, of course - Microsoft Flight Simulator's reliance on on-demand machine generation and streaming technology forms part of the same investment in data centres and generative AI that is even now threatening Microsoft's plans for reducing emissions. To build this world together adds to the burden placed upon the simulation's inspiration; to return to my Light Fantastic analogy, it's as though the Earth were slowly collapsing within its shimmering, silver skin. Microsoft Flight Simulator's emissions footprint might be trivial by comparison to, say, big agriculture, but in simulating the whole Earth for the enjoyment of explorers and collectors, it exemplifies the idea of Earth as a place to plunder.

Watch on YouTube

With all that in mind, it's reassuring to hear Neumann suggest that Microsoft Flight Simulator's simulation might, in fact, be subject to deliberate limits. If there's a feeling of social responsibility here, it's as much to do with what not to show. For example, the real-time locations of endangered animals.

There are many creatures in Microsoft Flight Simulator 2024, but they appear to be generic entities borrowed from the game's ironically-named stablemate Planet Zoo, and loosely assigned to certain latitudes. The developers could have gone much further. "For 2024, we licensed this thing called Spire, and they have a transponder signal of every ship on Earth, every 30 seconds," Neumann says. "So we now have every ship on Earth every 30 seconds. We could do that with environmental things as well.

"I was pushing hard to get a bunch of animals in, because the world needs to be alive - it makes people happy when they encounter animals in the world," he goes on. "There is actually data where you could show certain animal types with transponders, and I shied away from using it because I didn't want to have any poachers get any data that's useful." There's an "angle of responsibility" to Microsoft Flight Simulator 2024, he asserts. In practice, it's not just about making it "feel as close to the real thing as we can get".

I finally got round to playing Microsoft Flight Simulator 2024 last week and, to be triumphantly contrarian, I really dislike the aeroplane aspect of it. The planes are wonderfully realised, and yes, I'm sure hobbyists will relish their detailed interiors and handling models, but for me, they're just white noise between the player and the game's intricately recreated Earth, which I am both disturbed and fascinated by.

The first thing I did in the game was engage freecam, slide clear of my aircraft's cockpit, and swoop down to visit my old stomping grounds in West Yorkshire. I alighted on a main road between Leeds and Bradford like an absent-minded angel, and spent a few moments ghosting between streetlights, peering at the nearby fields and watching boxy cars bumble past. I found my surroundings totally unrecognisable, which filled me with relief. Not yet: the spell is still being perfected. There is still a more Earth-like Earth than this.

Read Entire Article