Computer Vision for UAS Applications

Computer vision in UAS is often perceived as a universal function: install a camera, add a neural network and receive automatic responses. In a real project, everything is more complicated. The algorithm works only when the sensor, optics, stabilization, computer, lighting, flight altitude and method for checking the result are consistent with each other.

For AERIS, computer vision is not a separate module, but part of an aerial robotic system. It should help the mission: find defects, highlight objects, count changes, aid navigation, or reduce the amount of data that the operator has to review manually.

Start with the mission requirements

You need to start with the question of what decision should be made after image processing. Signs of defects are important for infrastructure monitoring. For perimeter security - detection of objects and events. For navigation - stable landmarks and position assessment. For inspection - repeatability of shooting and the ability to compare the condition of an object over time.

• describe the target object and the signs that need to be detected;
• set requirements for accuracy, update frequency and acceptable errors;
• understand whether onboard processing is needed or post-processing is sufficient;
• determine whether the operator should see the stream, events, or finished report;
• collect a data set in advance for testing algorithms.

Components of a complete solution

A working computer vision system consists of a camera, a gimbal or mount, an onboard computer, program model, communication channel and validation procedures. If even one part is chosen incorrectly, the quality of the entire system decreases.

• camera and optics set the initial data quality;
• stabilization and vibration isolation reduce smearing and geometric distortion;
• the computer determines the delay, power consumption and temperature;
• the algorithm must be trained and tested under similar shooting conditions;
• The ground loop must store the results so that they can be used in production.

Key engineering constraints

The main limitations of computer vision for UAS are related to mass, power, heat, data path, and environmental variability.

• a model that works well in test photos may degrade at a different height, angle or lighting;
• transmission of full video in real time often requires more channel than transmission of events;
• edge-computer increases autonomy, but adds heat and consumption;
• night work, haze, snow, glare and shadows require separate testing;
• the quality of the result depends on synchronization with coordinates and telemetry.

Practical application

In real-life operations, computer vision must be built into the route. The platform flies at a given altitude and speed, the camera receives a predictable frame, the algorithm highlights features, and the operator receives not a “to-look” stream, but understandable events or areas of attention.

For industrial applications, repeatability is especially important: if the mission is carried out regularly, the system should provide comparable data. Therefore, the route, camera settings, report format and quality control must be described in advance.

Common mistakes

Most errors arise from trying to buy an “AI camera” without designing a scenario.

• choose an algorithm before understanding the data and shooting conditions;
• do not test the model on real customer objects;
• forget about lighting, weather conditions and seasonality;
• do not take into account processing delay and communication channel limitation;
• do not describe what is considered a correct or incorrect response.

Checklist before starting the project

It is helpful to answer a few questions before a project.

• what objects or defects need to be detected;
• whether work is needed online or after landing;
• what shooting conditions will be typical and extreme;
• which computer is acceptable in terms of mass and power supply;
• how the quality of the algorithm will be measured;
• how the result will end up in the customer’s system.

Practical conclusion: It makes sense to implement computer vision as a combination of sensor, computer, data and operational process. Then the UAS becomes not just a camera carrier, but a decision-making tool.