What’s the buzz?
Built in academia and accelerated to market with the help of business giants, the field of AI has been developing fast and triggered a lot of hype. It was forecasted that AI will power innovation in many domains, from search engines and voice assistants to facial-recognition and object detection systems to change our lives once and for all. Today, billions of people use AI daily, inside their smartphones, speakers, cameras, cars, online, inside chatbots, or on personalized web pages and applications.
Yet despite this success, the fact remains, deploying AI has certain limitations in production environments. For example, self-driving cars have become more capable are not at the point of being safe enough to deploy on everyday streets. Hence, the question pops up.
“Is today’s AI technology really as reliable and world-changing as we imagine?”
To answer this question, let’s look at what AI is, and more specifically, deep learning. Fundamentally, AI is about using computer algorithms to make decisions, so humans don’t have to. In an ideal world, these decisions are made instantly, enhancing how we interact with the things we use every day. For instance, in self-driving cars, it’s the difference between being the driver and being a passenger.
With AI, the percentage of time the decision is correct is called “Accuracy”. Accuracy goes from being important to being critical, depending on the situation. To increase accuracy, AI needs to learn. This is called “Training” and requires a lot of data. So, the better the training, the better the accuracy, the better the decision making.
Once a suitable accuracy is achieved, you can run your AI. The live decision making is called “Inference”; data is received, processed and a decision is made. For example, a camera captures real-time video, the AI detects a person, and the car applies the brakes. The time delay between the data capture and the reaction is called “Latency”. In the example above, by speeding up the decision-making, the reaction time of the car increases.
Reality check
Here is a real-life example. Let’s imagine a driver who sits in their self-driving car, looking down at his dashboard as it makes its way down a suburban road. Suddenly a person runs across the street during their morning jog, which causes the car to come to an abrupt halt, right in front of the jogger. In situations like that, reaction time is a critical factor.
The crux of the matter
Although AI can be powerful and enable novel technologies, it also can be intensive, troublesome, and challenging to deploy. The latest AI systems’ demand for computing power can be expensive. Deep neural networks have a large memory footprint, significant power consumption, and require high compute power. Decisioning across all that data (Inference) requires vast computing resources, memory, and energy to run. You can find this computing power in the cloud, but connectivity and the time delay (“Latency”) are a non-starter in critical decision making. You can bring this computer power closer to the Edge but deploying AI on the Edge is no small task.
So, what needs to be done to enable AI adoption at the Edge?
AI models need to be faster, smaller, and more cost-effective while maintaining accuracy to run on edge devices. Deeplite was created out of this need enabled AI to run more efficiently. We’ve built our own AI solution that can Automatically make AI way smaller without losing the accuracy so you can embed it into small, inexpensive hardware without losing the reliability.
Where the rubber meets the road
In our demo above, the original vehicle detection model was optimized by Deeplite’s optimization engine and deployed the new model within the BlackBerry QNX platform. While driving, the Blackberry QNX Advanced driver-assistance system captured the video data feed, which was then processed through the Deeplite optimized AI model. With this combination, we significantly increased the throughput performance of the AI, enabling the vehicle to react up to 3 times faster.
As we saw in the video, increasing the throughput and size of the model also uses much less energy. This is important in general but hugely advantageous with battery-powered AI like electric vehicles. As shown in the video, not only does Deeplite make self-driving cars safer, it also significantly reduces battery consumption, giving drivers a lot more time on the road.
This video is an excellent example of our AI technology, processing a real-time video feed from Blackberry QNX. In a self-driving car, reaction time is everything. That self-driving car, a drone inspecting crops, a camera looking for defects in steel, your phone authenticating your identity. This is AI interacting in the real world, what we call the Edge. With Deeplite, complex AI can now run on any device on tiny chips. We can embed AI in that drone, in that camera, on that phone. We can embed dozens of AI models into that self-driving car. We can get AI out of the lab into the real world.
We enable AI for everyday life.
