Google's Global Localization is Every Traveller's Dream

The trouble with GPS

We're all familiar with the little blue dot that pinpoints your location on Google Maps. This process relies on GPS, which in turn relies on the delay of radio signals from dedicated satellites to nail down your exact position.

But GPS is fallible. Travellers wandering around big cities: London, New York, or San Francisco, for example, are often left frustrated. How is it that these tech capitals struggle most with localisation? Well, it's actually really difficult to pinpoint your location when the signals you need to do so are blocked and delayed by tall buildings. Signals get reflected and your access to the sky is limited.

The result is something we're all familiar with, too: Wildly inaccurate placements on the map; your little blue dot in the middle of a nearby river (worrying) or on a completely different street to the one you're actually standing on (inconvenient and counterproductive).

The below image illustrates the signal issues caused in dense urban areas. It's taken from a post published by Tilman Reinhardt?, a software engineer at Google Maps.

https://4.bp.blogspot.com/-hc7L9kuwZ8s/XF284i61CzI/AAAAAAAADvg/HUjU83Fg3js8oa4XBJjPXcmd0iUq8OZnQCLcBGAs/s1600/image4.png
GPS signals bouncing off facades in an urban environment.

Apart from generally getting things wrong, GPS has another major limitation that impacts travellers every day: It determines location, not orientation.

The direction you're facing is pretty important when you are exploring a foreign place. It's no good saying north or south if you don't know where those are. Travellers don't tend to carry compasses these days. And the sensors in smartphones designed to provide orientation can often be undone by nearby interference.

As Reinhardt? explains, "Magnetic objects such as cars, pipes, buildings, and even electrical wires inside the phone, resulting in errors that can be inaccurate by up to 180 degrees."

Read more: Google Street View Expands to South African National Parks

The next step: Google's 'Global Localization'

Google's solution to inaccurate GPS and orientation is to combine a host of the company's technologies into one seriously powerful - and rather intuitive - navigation system.

"We're experimenting with a way to solve this problem using a technique we call global localization," writes Reinhardt. "...which combines Visual Positioning Service (VPS), Street View, and machine learning to more accurately identify position and orientation. Using the smartphone camera as a sensor, this technology enables a more powerful and intuitive way to help people quickly determine which way to go."

Being at the overlap of travel and technology, we thought it would be a good idea to delve into this concept in more detail.

So let's start with VPS, Google's Visual Positioning System.

'Visual Positioning' in this sense is pretty literal. Just imagine you are walking in an area you are not very familiar with or haven't been to in a while. The first thing we do it orient ourselves with landmarks that are familiar: that church spire in the distance, the McDonalds on the corner, the train station on the next street.

Google's Visual Positioning System is based on this same intuitive method.

'Global localization' relies on a combination of techniques and technologies. A key component of the solution is the camera attached to your smartphone.

Rather than using GPS, VPS uses imagery to see the world as we do.

Its first task is to create a map by analysing your route, taking a series of images which have a known location and analyzing them for key visual features. This might be the outline of buildings or bridges, or other permanent city fixtures. The system then creates a large scale and searchable index of those visual features.

Next is the really clever part. To localize the device (and therefore its user), VPS compares the features in imagery taken live from the phone to those in the VPS index.

It's a great concept. The global localization system will have a huge database of images and recognisable urban fixtures. Then, when you're in a location with GPS instability, it can compare the real world through your smartphone's camera and use that information to guide you with perfect accuracy.

Of course, the accuracy of any system like this is entirely based on the quality of that image database. Where can Google possibly go to get a location-specific database of landmarks and urban areas?

Of course: Street View.

Street View gets a greater purpose

We can all remember seeing Google's odd little cars driving around our streets, mapping roads and building up to the eventual launch of Street View. The option to take a virtual walk on Google Maps has become something we all take for granted.

"To deliver global localization with VPS," says Reinhardt, "we connected it with Street View data, making use of information gathered and tested from over 93 countries across the globe. This rich dataset provides trillions of strong reference points to apply triangulation, helping more accurately determine the position of a device and guide people towards their destination."

But it's not that simple of course. When your phone shares live images with Google to try and get a sense of your location, the view will no doubt be different than from Street View's huge dataset.

Think about the angle of the image, the weather and the lighting. The building colours may have changed, too. The challenge is to devise a system that's able to filter out these temporary changes and recognise landmarks for what they are, no matter the time of day of the minor changes that may have occurred over time.

Which is where artificial intelligence comes in. Or to be more specific: Machine learning.

"That's why a core ingredient in this new approach is applying machine learning to automatically decide which features to pay attention to, prioritizing features that are likely to be permanent parts of the scene and ignoring things like trees, dynamic light movement, and construction that are likely transient. This is just one of the many ways in which we use machine learning to improve accuracy," says Reinhardt.

Read more: Report Digs into Travel Technology Trends for 2018

Combining Machine Learning with Augmented Reality

Google being Google, it's not enough to use Street View, your phone's camera and machine learning to provide a new way to guide you around locations. The search engine giant also wants to incorporate augmented reality (AR) into the process.

If you're not familiar with AR, it's essentially the practice of overlaying virtual images onto the real world. Pokemon Go is arguably the biggest mainstream example of it to date. But it's not restricted to the world of entertainment, as Google wants to show.

If we imagine that Global localization will become an additional option within Google Maps that users can turn on when they need it, how does AR fit into the picture?

Well, with increased precision comes more possibilities. "One of the newest features we're testing is the ability to use ARCore, Google's platform for building augmented reality experiences, to overlay directions right on top of Google Maps when someone is in walking navigation mode. With this feature, a quick glance at your phone shows you exactly which direction you need to go," explains Reinhardt.

That is a nice idea, right?

AI-powered camera experiences from Google are nothing new. Google Lens allows you to search for what you see, for example. But Reinhardt argues that "the ability to overlay directions over the real world environment offers an exciting and useful way to use the technology that already exists in your pocket."

Read more: Deep Dive: The Travel Industry's Social Media Marketing Landscape

Google is central to the future of travel

As the original online behemoth, Google is the established point of reference for anybody using the internet. But who could have foreseen that the search engine giant would go from aggregating website searches to becoming a huge player in the global travel industry?

As we well know, appeasing Google is key for any successful travel agent, particularly with more trips researched and booked online through computers and mobile devices every year. The company is increasingly packaging its own travel deals as well, rather than simply being a conduit for other agents.

But Google is also becoming a major player in the travel experience itself. You have Google Translate - the go-to tool for anyone travelling through a foreign country and trying to pick up the language as they go. You also have Street View and Google Maps: navigational tools that make travel more accessible than it's ever been before.

And now, Google wants to take that to the next level with Global localization, combining existing data from Street View with machine learning and augmented reality to turn your smartphone camera into an active part of the navigation process.

No more unstable blue location dots. No more walking up the street the wrong way as part of an annoying orientation process. Just easy, accurate directions.

We can't wait to try it.