Newly published research from the University of California, Merced, is paving the way towards a fascinating new AI use case: applying machine learning to satellite imagery to visualise views on the ground.
The technology will be especially useful for mapping and classifying large rural areas where aerial imagery is abundant, but on-the-ground imagery is sparse or non-existent.
The paper claims that the researchers are the first to use this method to translate satellite imagery into ground-level predictions. So how does it work?
A game of snap
To predict ground-level views from satellite photos, the team used a machine learning system known as a conditional generative adversarial network (cGAN). This combines two neural networks: a ‘generator’ and a ‘discriminator’.
The AI was shown thousands of pairs of images, with each one comprising a satellite image and a ground-level photograph of the same location. By comparing 16,000 such pairs, cGAN was trained to visualise what objects looked like on the ground from the aerial photographs.
The ‘generator’ neural net was then fed with 4,000 new satellite images and told to create fake ground-level images based on each one, and on what it had learned from the training data. The generator then learned via feedback from the ‘discriminator’ (which had access to the real ground-level views) to produce more accurate images.
The theory is that the process iteratively improves: the discriminator logs the differences between the genuine and AI-generated images, and the two neural nets play a competitive game to produce more accurate results – the adversarial element in the technology’s name.
Machine-made images
The resulting fake ground-level images are natural looking and structurally similar to the real images, but of course lack genuine details; they’re merely a prediction of ground-level views, based on the aerial data.
But given their ability to represent land types and features such as roads, the dense feature maps that can be generated using this framework are more effective for land-cover classification than existing human approaches.
As a result, the new AI-enhanced process could make the work of geographers much easier by supplementing the sparse image libraries of rural areas that they’ve had to depend on for years for this type of classification work. Short of travelling to far-flung locations themselves and taking thousands of photographs, this is the most accurate method developed so far.
The new technique is able to successfully determine land use 73 percent of the time, while the currently used human interpolation method is correct in just 65 percent of cases. However, the research team aims to improve on the system’s performance even more, and explore other machine-learning methods.
Lead researcher Xueqing Deng said:
We plan to develop cGANs that can generate more detailed ground-level views that can be used directly for image classification. The training of the cGANs is still very unstable. We will therefore also investigate other techniques and architectures to make the training of cGANS for our particular problem more stable.
Internet of Business says
The falling costs of satellite deployment and the new-found computer vision capabilities of AI have seen an influx of new research and investment in this space. For example, buouyed by acquisitions, DigitalGlobe and BlackSky are rapidly applying AI to their satellite imagery platforms.
Machine learning models are now being fed with data from a huge variety of sources, including social media, content from news outlets, radio communications, and earthquake sensors. The aim is to complement satellite imagery and allow faster action to support business operations and humanitarian efforts on the ground, while also responding to criminal activities and natural disasters.
Meanwhile, the Earth’s orbit is increasingly the domain of private space-faring companies, such as SpaceX, Virgin Galactic, and OneWeb, which are launching privately owned satellite constellations to help sustain our soaring data and connectivity needs.
As George Whitesides, CEO Virgin Galactic and The Spaceship Company, once said:
New space technologies developed by private companies will bring about fundamental change that will influence business and our personal lives.
AI and machine learning methods will play a key role in this new space age, by analysing the vast amounts of data that satellites create and relay.
This latest research out of the University of California is a clear sign of the field’s direction and early potential.
Elsewhere, Stanford University is combining satellite imagery and machine learning to predict poverty, while closer to the ground, Google’s controversial work in the AI interpretation of drone imagery for the Pentagon has caused a storm of controversy, both inside and outside the company.