The introduction of video analytics for surveillance was revolutionary. Since then these applications have evolved and changed the way video surveillance is installed, monitored, and reviewed. The premise of these software applications is to optimize the human resource and focus the attention on those events that are relevant based on a set of predefined rules or conditions.
Today, most manufacturers provide one or more video analytics as part of their solution. Moreover, there are companies that focus specifically on video analytics development. Not all analytics have been designed the same way, however, and depending on the situation, one video analytic type can provide better results than others. At the same time, there are important factors that can support or undermine the video analytic performance.
How do video analytics work?
No matter the type of video analytic solution, the process they follow is very similar. Video analytics use video as input to compare against the condition(s) or rule(s) defined on the software engine. Depending on the type of analytic, the engine will look for specific information on the image. If the event is matched with the rule, then the analytic will trigger an output or action. Imagine that you are interested in identifying all events where an object crosses a virtual fence defined on the video analytic. If the event matches the condition, the operator should get a visual notification while the system logs the event on the server.
As part of the output, the video analytic generates meta-data, which is information in the form of text to describe what happened on the event. Meta-data can be used to trigger an alarm, classify events, run reports and search for information (i.e. forensic analysis).
An important point is that video analytics require a learning period before the engine can start detecting events. This learning period will give the analytic a baseline to gather information and compare it against the condition. Furthermore, some video analytics will require a minimum video resolution to operate properly, at least x number of frames per second to run the analysis and in some cases a camera angle to obtain better results.
What are the different types of video analytics?
If we classify the types of video analytics available in the market, we can cluster them in three main groups. Basic video analytics, advanced video analytics, and deep learning video analytics.
Basic video analytics were developed more than 15 years ago. These video analytics are designed to use pixel changes to trigger an event. An example of a basic video analytic is simple motion detection. When a group of pixels on the image change its color, the engine will consider this event as something moving on the scene resulting in motion detection.
Another example is camera sabotage/tampering. In this case, the camera will take a snapshot of the scene and if most of the pixels on the image change, then the analytic will consider this as tampering.
The disadvantage of basic video analytics is the number of false positives you could get since the engine is very basic, especially in complex scenes. Shadows, trees, animals or weather conditions can trigger the analytic, reducing its benefits.
Advanced analytics were developed more recently. These analytics use Binary Large Objects (BLOB), which is a group of connected pixels. This type of analytic look for shapes and forms reducing the number of false positives. Compared to basic video analytics, this type also provides additional settings to finetune the engine such as perspectives, camera angle, type of scene, number of frames for analysis, among others. Finally, algorithms using advanced analytics can reduce the number of false positives by narrowing down the number of events that can trigger the analytic (i.e. Object classification).
Artificial Intelligence (AI) or deep learning video analytics is the most recent technology used for video analysis. These video analytics take advantage of neural networks to train the analytic and produce an outcome. The more events the analytic is trained with, the higher the accuracy to detect relevant events. The learning process can be manual (human intervention to mark the events that are relevant) or automatic using a library of events gathered from different sources. Deep learning analytics will provide more advanced analysis such as feature searching or multiple conditions.
Deep learning analytics offer the ability to sift through information and compare against conditions faster and more accurately.
Server Analytics vs. Edge Analytics vs. Cloud Analytics
When implementing video analytics, you should decide if the analytic will reside on the camera (edge analytics), on the server, or on the cloud. Most camera manufacturers offer video analytics on the camera either for free or for an additional fee. In contrast, video analytic manufacturers offer video analytics either on the camera, on the server, or on the cloud.
It’s important to remember that for more advanced analysis you will require more processing, thus not all advanced video analytics or deep learning can run on the edge, but on a server, the cloud, or a hybrid. If the video analytic can run on the camera, most probably will require high-end camera models since these cameras will have better processors.
Finally, another important factor is the Video Management System (VMS) compatibility with the video analytic. Not all camera manufacturers or third-party video analytics manufacturers have integration with all VMSs. In some cases, the integration might be limited to sending alarms to the VMS rather than transmitting the meta-data to the VMS.
What other factors should be considered when implementing Video Analytics?
The video analytic can only deliver a good analysis if the video provided is of good quality. A barrel effect (optic distortion) on economic lens can be an issue when the subjects move to the corners of the image. Cameras without a true wide dynamic range will provide a poor image when there is a high contrast on the scene reducing the detection capabilities of the video analytic. A high shutter speed could be a key feature when objects move fast on the scene or when the camera is zoomed in on a small area of the object (i.e. vehicle’s license plate) reducing blurred images.
Video analytics, either on the server side or on the cloud, will require bandwidth to receive the camera streams and processing power to run the analysis. The advantage of this architecture is that you don’t require a specific type of camera chip, the disadvantage is that bandwidth usage will be higher. Depending on the number of video analytics, you might require more than one processor on the sever or increase your internet bandwidth if using cloud analytics. If bandwidth is an issue, evaluate if edge video analytics can meet your project requirements.
Measuring video analytic accuracy is a relative subject, especially when there is no standard in the industry for it. A manufacturer can increase the video analytic accuracy if the sample is constrained to a few trials (i.e. 10 trials and 9 of them were positive = 90% accuracy vs. 100 trials and 9 of them positive = 9% accuracy). Accuracy could be measured only on real events, eliminating false positives from the sample, or if the statistic is obtained under ideal conditions (i.e. camera angle, controlled lighting, simple background, little activity on the scene, a set distance from the camera, specific lens or camera type).
The only accuracy statistic you should trust is a trial period based on your conditions and settings.
Conclusion: Four important points about video analytics for video surveillance
There are important differences in video analytics and using the right video analytic in your implementation is crucial to obtain the best results. In most projects a combination of different types of video analytics will be the way to go, whereas in very specific situations one type can be more suitable than others.
Compare and test the video analytic before choosing a manufacturer. Even though two manufacturers claim to have the same type of video analytic, the outcome on the same scenario might differ considerably since accuracy can be measured differently. Not all the scenes are ideal, especially when using video analytics outdoors. Lighting, camera angle, weather conditions, and the image quality are some examples of factors that commonly affect the results of a video analytic.
Remember that all analytics will require configuration and fine tuning. Assess the ease of use and the level of training required to make changes and obtain results.
An integration between the Video Management System and the video analytic is important. Furthermore, an integration that includes meta-data support will give you more usability and the possibility to expand its applications.