Big Data and Security: an overview


Following on from ICD Shanghai's recent afternoon seminar themed big data in security, this article will draw on the information shared by our partners Axis Communications and Milestone to give an overview of this current trend in security.


Looking specifically at big data in video surveillance, we aim to give you understanding of why large amounts of data can be advantageous to your security systems, as well as an overview of how to go about collecting and managing this data to enhance your video surveillance systems and your overall security solution.

What is big data?

The term big data is used to describe any collection of data that is both large and complex, and is therefore difficult to process through traditional data processing applications.


Big data is typically characterized by the ‘4 Vs’:

  • Volume
  • Velocity
  • Variety
  • Value


It is clear that big data has already become part of our everyday lives, as we each receive, transmit, analyze and store large amounts of data on a daily basis, both at work and in our personal time. News feeds, emails, social media, communications and blogs list just a few of the channels and sources of big data; there are many more applications and processes which add to this exponential growing amount of data around us.


Although this data can be used effectively in our work and our everyday lives, the sheer amount of it poses challenges in terms of how to capture, extract, analyze, search and store all the information.


This is true also for data in security: in general, we are seeing an increasing number of security devises adopted in any one comprehensive security solution. For example, within video surveillance systems alone, not only are more cameras typically installed around a site, but enhanced technology means that each camera is collecting a larger amount of data than before.



According to a study conducted in Europe, “Eighty percent of surveillance video submitted to the police is unusable!” (Swedish Police Central Imaging Group White Paper on Image Usability)


Are we therefore collecting an increasing amount of data for it to end up being unusable and ineffective for serving the principle purposes of video surveillance: to solve crime and to maintain a safe and secure environment?  


So how can we reduce this figure and ensure that a larger portion of footage and images captured via surveillance cameras is useful?


The study goes on to highlight that significant improvements can be made through simply better thinking through the placement of surveillance cameras. Moreover, huge improvements can also be made by specific camera features designed to help overcome difficulties in capturing high quality, usable images, such as challenging lighting scenarios. This is where big data comes in. 


Capturing big data to improve image quality and clarity

Improved technologies can ensure that the best image quality possible is obtained and that the video captured will help end users optimize each camera and their overall video surveillance solution. This ensures that the large amount of data captured is used to the end-user's advantage and is not wasted.


Image resolution and frame rate

The most obvious way in which image usability is increased is through capturing higher resolution images. For example, HD images guarantee better color representation and a guaranteed resolution to improve the overall quality of the image, as well as higher frame rates which result in clearer images of moving objects. With a larger number of pixels and an increased number of frames per second, the amount of data collected over any defined period by an HD camera is therefore greater than that of a lower resolution camera.


Electronic image stabilization

Another feature that ensures high quality, usable images are captured during the challenging situation of a camera which is subject to frequent and severe movements or vibrations is EIS. Incredible useful for cameras in vulnerable outside locations, this feature can produce outstanding results (see the video on this page).  


Wide Dynamic Range

WDR is a special feature in surveillance cameras which enables scenes wherein both low and high levels of lighting are simultaneously present to be captured in one final effective image. This feature is essential to capture usable images where there are significant contrasts in light, such as a scene with strong background lighting or where there are rapid and constant changes in light. For more information about WDR and how it works, read this post.



De-fog mode

The latest in hi-tech surveillance cameras also offer impressive features which dramatically improve image quality in foggy or polluted conditions. De-fog functions, as described in this post, use complex algorithms and data analysis to determine the level of fog at a scene before adjusting the image captured, therefore also relying on the camera to capture big data.


What is true about all of these features, and other new capabilities of surveillance cameras, is that because of the increased quality of the image, and the technique and sophistication with which it is captured, the image size tends to be bigger and more complex, leading to larger amounts of data collected which in turn needs to be processed, transmitted, stored and possibly searched or analyzed. 

Extracting and analyzing big data for your advantage

So it's clear to see how collecting big data can be to the advantage of your security solution, but once it's been captured, how can you ensure that is can be used effectively and is not still ultimately wasted or 'lost in the system'?


The key is in video management software solutions and video analytics, whether in the camera or part of VMS software. Such software can enhance the end-user's ability to organize, process, search and store big data in security immensely. Below are some examples of how:


Event detection

VMS can be set to detect certain events or breaches of security within your video surveillance system, such as a person crossing a virtual boundary, loitering, crowd formation, items placed in, or taken from an area or even someone moving in the wrong direction of traffic. Once detected, the software will send an alert to the site's security manager for verification, typically via email. The major advantage of this technology is that it saves huge amounts of man-power which would otherwise be needed to watch camera footage from a large number of devises.  


Statistics, Business Intelligence and People Counting

VMS systems can also be set with to record and track data and trends collected from surveillance footage that is not necessarily related to security. Such data includes that from features such as people counting, queue management, tracking of customer flow and frequented shopping areas. This information can prove incredibly valuable to businesses (particularly retail) as it provides business intelligence, insight and statistics related to customer trends which is otherwise laborious or near impossible to collect. 


Identification, recognition and tracking

Sophisticated object recognition features within video management software are also able to not only recognize whether an object is, say, a person, a dog or a car, but also include specific identification, such as license plate recognition and facial recognition (forensics). Once again, this is an incredibly useful function whether or not your solution contains a large number of cameras as it can help detect and track criminals and specific vehicles. Such features can also be used on production lines or in logistics facilities to effectively track goods through a system.


Fast and effective searches

One of the most valuable features that VMS solutions can offer is the ability to carry out fast and effective searches. Because the video surveillance footage has already been processed and analyzed by the software, it become incredibly easy to carry out searches based on simple search phrases of time frames. Not only does this produce the desired result of finding the footage required, but such software is also designed to be user friendly and intuitive, making it favorable for end-users. 


For more information about video analytics and how it can enhance the effectiveness of your security solution, you may be interested to read this white paper.



Big data is unlikely to lose significance or volume any time soon. Already part of our work and personal lives, it is impossible to ignore the affects of this trend on video surveillance solutions and security systems in general.


Whether you manage the security of a site or monitor traffic systems, or whether you work on a production line or in a logistics center, the chances are that there is an increasing amount of video surveillance and security related data that your system collects and stores every day. This data, if captured and managed successfully, can dramatically improve the usability and overall efficiency of your security solution, so it is essential that end users understand how to best utilize the technologies and software available on the market to capture, extract and use big data effectively.


For more information about anything mentioned in this post or if you have any questions, feel free to follow the relevant links or to contact us directly on 




For more tips on security, follow ICD on our LinkedIn Company Page.