I am Vincent of Sterblue (https://www.sterblue.com). We build software for drones to inspect power lines and wind turbines automatically.
We are three drone enthusiasts. I met Nicolas in 2006 during our Aerospace Engineering studies at ISAE, in France. Small drones were still in their infancy and we eagerly enrolled in the brand-new student micro-drone club. Nicolas later met Geoffrey when working in India for a large aerospace company, while I was working on my PhD in Computer Science.
In 2015, we shared a feeling of disappointment that the vision people had for drones back in 2006 had not fully happened yet. For example, utility companies need to inspect millions of power line pylons. They still mostly perform inspections using helicopters or rope access technicians. They sometimes use drones for short distance inspections, but they could not scale it up to inspect all of their grid.
We decided to try to make drone inspections happen in the renewable energy industry.
In the beginning, we were super optimistic: we started by designing and patenting actual drone hardware configurations, building and testing drone prototypes. Then we realised that hardware is really hard! and worse, that new drones are not what this industry needs right now. Even with super high-performance drones, you still need a top-gun-level pilot to capture precise photos 8 hours a day from a drone flying high in the wind. And then you need to spend ages trying to find small anomalies on thousands of captured images.
So our approach became to automate most of the inspection process, using off-the-shelf DJI drones. We built two core technologies in the last 2 years to try to get there.
The first one is a mission planning and execution engine based on ideas developed during my PhD. It allows drones to fly on complex 3D trajectories that wrap tightly around structures, flying the drone sometimes less than 3m away from the objects to inspect. Most other drones companies fly high above the objects to inspect, so their apps show trajectories on 2D maps. Our app shows trajectories on 3D views because our trajectories are intricated closely around objects. As an example, we had to take into account the fact that wind turbine blades bend under their own weight. Here is an image of power grid inspection with our software: https://imgur.com/KRE1Oce
The second is a deep learning framework built on top of Tensorflow, PyTorch and Caffe2. It allows to detect 130 classes of defects on images: corrosion, broken equipment, and other safety hazards. The typical use case for drones today is photogrammetry, in order to create 3D models from captured images. This is not super useful for wind turbine and power line inspection, so we focus on our thing: finding defects directly on images, just like human operators do now. For several classes of common defects, our software has 98% recall and 80% precision. We run up to 18 different neural networks on some images in order to find defects on all kinds of equipment. Here is an image of our algorithm detecting an anomaly on a wind turbine: https://imgur.com/iGIxtAZ
These two technologies are linked to a GraphQL API and cloud platform. Our customers get to visualize inspections results on PDF reports and 3D interfaces online. Our UIs are made using React and React Native. Here is an image of an HTML report for distribution grid inspection: https://imgur.com/dkznc6o
In the last 6 months we had successful pilot projects with utilities in 7 different countries, and some have even started to use our software in production mode now. This is exciting for us. But we still have a lot to learn, and we’d like to hear your feedback and ideas in this space!
by Andrey
Azimov