Virtual Visiting

Why is there no playable 3D representation of at least the major cities in the world? That is a question I recently asked myself, prompted by an article posted on Hacker News. A student has vibe coded “San Francisco - The Game”, a third person exploration game which lets the player walk through a full 3D representation of San Francisco built from Apple Maps data. The result looks somewhat similar to Google Earth Pro Studio’s first person mode.1 It is certainly impressive, but nowhere near triple A game quality. There are lots of artifacts like wobbly edges, blurred textures and incorrect volumes, which give the landscapes a dreamlike atmosphere.

This is quite surprising to me, since there are fictional virtual worlds of considerable size, like The Lord of the Rings Online. With the hype around using Gaussian splatting to create 3D volumetric data from 2D information on Hacker News, I would have expected a playable Earth simulation to be only a matter of dedicating compute resources to it. At least for partial coverage, turning Google Street View panorama images into 3D levels should be doable. Apparently, there is already research in that direction, with two obstacles being the large distance of 10 m between Street View images and the masking of cars, people and other obstructions.

I would greatly appreciate an opportunity to visit faraway places in triple A game quality. Yes, what I am looking for is in fact some kind of walking simulator. But it would be the perfect calm game to play online with friends to relax after a work day. And it would have the additional benefit of learning about other countries, cultures and places which actually matter in the world, instead of fantasy lore which only matters to a potentially nerdy subsection of society. Traveling with a small CO₂ footprint, feeling connected worldwide in a world increasingly split by nationalism - it really seems like a worthwhile project to me.

And let’s face it: only very few of us are able to travel to all the places they would like to visit in their lifetime. So it’s quite ok to visit a few only virtually.

Because this idea of virtual visiting has fascinated me so much, I have cobbled together a temporary solution. It doesn’t require the fancy hardware Google Earth VR needs and still provides a decent experience: I have vibe coded a Chromium browser plugin which lets you use a gamepad to navigate Google Street View, similar to what you would do in a sandbox game. The code was produced by Claude Opus 5.5 and is available open source on my GitHub page. For increased immersion, it plays atmospheric sound recordings from close to the current position in Street View. TTS-generated readings of Wikipedia articles can be played to get tourist guide style information during visits.

I am currently playing through a three day visit itinerary generated with Claude and enjoy it very much. You can find the Tokyo with Hana guide here. The map’s buttons will need zooming in to the route and manually dragging the Street View figure from the lower right to the starting point. I hope to improve this in a future version and maybe even create a more game-like integration of itinerary and Street View.

  1. If you are wondering, that’s what you get when you download the desktop app, open Street View somewhere and then toggle to the 3D view with the switch in the top right. ↩