Andrew Hobbs delves into Google’s latest edge computing developments at Cloud Next 2018, and sits down with Product Lead Indranil Chakraborty to discuss how LG is driving remarkable results with Google’s new Edge TPU.
Internet of Business says
In July this year, Google announced its new Edge TPU and Cloud IoT Edge products. Edge TPU is Google’s purpose-built ASIC. It sits on a 40x40mm system on module (SOM) that delivers high performance in a small physical and power footprint, enabling high-accuracy machine learning inferences at the edge.
It represents the company’s response to the need for more and more data streams to be processed at the point of origin, bypassing the latency and bandwidth issues that cloud solutions introduce. AI models trained in the cloud increasingly need to be run at the edge.
Edge TPU complements Google Cloud services to provide an end-to-end, cloud-to-edge, hardware-plus-software infrastructure, and can be used for a growing number of industrial use-cases such as predictive maintenance, anomaly detection, machine vision, robotics, and voice recognition.
Manufacturing, healthcare, retail, and the supply chain sectors represent just a few areas where ASICs such as this will find a foothold over the next few years.
Indranil Chakraborty on Edge TPU
I spoke to Indranil Chakraborty, Product Lead for Google Cloud IoT Core and Edge TPU, and the originator of the Cloud IoT Core product, about how Edge TPU is already making an impact.
From the off, he emphasised the quest for efficiency that underpinned the design process:
The great thing about Edge TPU is that, when we designed it, we really focused on high performance-per-dollar and performance-per-watt.
The Cloud IoT Edge software runtime extends the data processing and machine learning capabilities of Google Cloud to Edge TPU. It can run on cameras, gateways – basically anything with compute capability.
“Cloud IoT Edge can execute machine learning models on Edge TPU or on CPUS and GPUs. It uses TensorFlow Lite so that, if you use Edge TPU, you can get the benefit of it, but if you don’t, you can still run these models.”
Edge at LG
Chakraborty gives the example of their recent partnership with electronics giant LG. LG already used a visual model to identify faults in LCD panels, as part of the QA process, but it only had 50 percent accuracy, meaning panels often had to be checked manually.
A Google engineer was able to use the Google Cloud ML engine to train the model with LG’s labelled image data. As a result, the processes fault-detection accuracy increased from 50 percent to 99.9 percent, which has ultimately meant a saving of $20 million a year.
Yet, LG’s problem wasn’t as straightforward as retraining its model:
They were taking 200 high-resolution pictures of the LCD panels every 0.8 seconds. This time constraint meant they couldn’t send the data to the cloud for predictions, so they worked with us to use the Edge TPU and Cloud IoT Edge on the camera.
“They trained the model for detecting defects in the cloud and actually did the prediction and detection on the camera itself. They now get high accuracy and low latency all on the assembly line itself, so they don’t need additional people to monitor it.”
Google have made it easy for end-users to build proof-of-concepts using Edge TPU by creating a development board too. This has a System-on-Module, including the Edge TPU, a crypto microchip, and several other processors too. The crypto microchip contains your private key, allowing you to securely connect with the cloud.
Chakraborty claims that Google’s Edge TPU is currently the only machine learning accelerator chip for edge devices. However, potential adopters should bear in mind that, at present, the ASIC requires compiler services that are only available with Google Cloud.
Despite the capabilities of purpose-built edge chips such as this, we won’t see all data analytics shift to the edge. It’s largely a question of whether latency and bandwidth constraints make the desired processes unsuitable for the cloud.
Even when edge computing is the answer, machine learning models will still be trained in the cloud, where more resource-intensive CPUs and GPUs can be leveraged.
“That’s where we see the split between edge and cloud being more relevant – where you train the model in-cloud, you push it down onto the local inference [on the Edge TPU], but periodically you may, for example, have pictures where the confidence interval of inference is not high enough, so you send it back to the cloud, manually label it, use it to retrain the model and then push it down to the device again,” explains Chakraborty.
We think the cloud will always play a role as the central aggregator, but some processing can be done locally, as well.
The Edge TPU road-map
Google is starting to rollout Edge TPU to more early-access customers from today, so that they can start prototyping, but final pricing has yet to be announced.
While real-world tests to-date have focused on image-based work, there are many potential applications, such as voice-detection, text processing and vibration-analysis.
Google believes Edge TPU will be suitable for use-cases across industries.
“Our hope is that we get to a point where the chip itself can address multiple scenarios. We’ll see how it works but, at this point, we have designed Edge TPU as a general chip for doing machine learning on the edge. With customer feedback, we’ll get to the point where we can ask, ‘Do we need to create an additional chip for a specific application?’”
Google is working with just a few select customers at the moment, but the next two quarters will see this group expand, before it’s finally made publicly available.
Already the team is looking beyond this, though:
“We are working with a couple of partners in the carrier and networking spaces, where the intent is to look at the data within the access point itself and try to use Edge TPU to detect anomalies and certain patterns for additional insights. It’s pretty early but I think at some point we’ll have more announcements to make.”
To learn more on the power of Edge computing, IoTBuild is taking place on 13-14 November 2018, Olympia Conference Centre, London