A Complete Guide to Video Analytics

A Complete Guide to Video Analytics

This guide covers the origins of video analytics, video content analysis, the use of video intelligence in multiple industry sectors, stand-out analytics advances and future AI applications.

What are video analytics?

Video analysis is the processing of video footage in real-time and the transformation of captured information into intelligent data. The analytics process automatically generates descriptions of video activity (or ‘metadata’) to detect and track objects such as people and vehicles.

Video analysis software is generally installed to generate objective data intended to inform decision making and ‘best next actions.’ Today, ‘next actions’ cover a vast range of possibilities, from security interventions to procedural updates and lifestyle management.

Video analytics software is fast and accurate – often supported by artificial intelligence – and have largely replaced manual video footage review, which was and is inefficient, costly, and prone to human error.

Video analysis is most frequently deployed to overcome human limitations when it comes to coping accurately with large volumes of data.

Video analytics solutions are also used for video data mining, which involves the analysis of historical data to discover insights such as trends and patterns.

Video content analysis

Video content analysis enables organisations to maximise the value of CCTV video footage. Video content analysis helps organisations to analyse content for performance improvement, customer behaviour and preferences, to understand potential risks, and to develop valuable training materials.

Facit’s video content analysis tools help customers across a range of important business matters and intelligence gathering.

How can Facit help you with video content analysis?

Facit’s video content analysis tools help customers across a range of important business matters and intelligence gathering.

Many businesses record and store large volumes of video footage but, without content analysis, the video footage they hold represents an untapped resource.

Video analysis can be used to:

• Track customer visiting times

• Analyse customer behaviour

• Monitor product and attraction popularity

• Develop ‘real-world’ training tools

• Assess health and safety risks

Trend analysis and performance analysis are important to any organisation., and are particularly valuable in sectors such as:

• Retail

• Transport

• Tourism

• Healthcare

• Law enforcement

• Public space and facilities management

Consider the value to a retailer of insights into what makes a particular product or area of a store popular or unpopular. Consider the value to clinicians of redacted video of live healthcare procedures for staff training. Consider the value of video analysis in redacted video of events when training police officers, to illustrate the most effective ways to deal with a variety of situations.

The evolution of video analytics: surveillance roots stimulate video analytics debate

Since the widespread introduction of CCTV in the 1970s, surveillance of the public has been met with a mix of sceptical and positive reviews. Opinion is split between those who see video analytics as an intrusion on privacy and those who value its effectiveness for preventing and punishing criminal activity.

However, over the years since video analytics solutions were introduced, the use of what is known as smart video technology has spread to inform the analysis of every type of activity, in business, sports and people’s lifestyles, and has extended far beyond security operations.

How do video analytics work?

Video content analysis refers to the automated processing and analysis of video streams in real-time to extract meaningful information.

Video content analysis can also be used to analyse past events and, when redacted to comply with privacy regulations, to develop training materials.

Analytics technology leverages techniques from computer vision, machine learning and artificial intelligence to understand and interpret video data. Depending on the particular use case, the architecture of a solution may vary.

When conducted in real time, video content analysis isconfigured in the system to trigger alerts for specific events and incidents that occur in the moment, or in post processing, by performing advanced searches to facilitate extensive analysis.

Key techniques in video content analysis

1. Object detection and tracking

   Object detection: Identifying and locating objects within a frame. Common methods include using convolutional neural networks (CNNs) like YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector).

   Object tracking: Following the detected objects across multiple frames. Algorithms like SORT (Simple Online and Realtime Tracking) or Deep SORT are popular.

2. Activity and behaviour analysis

   Detecting and understanding specific actions or behaviours of objects. For example, recognising if someone is loitering, falling or fighting.

3. Facial recognition

   Identifying or verifying a person’s identity using their facial features. Techniques involve deep learning models like those based on FaceNet architecture.

4. Motion detection

   Identifying sections in video footage where motion is occurring, which is often used in surveillance and security applications.

5. Scene-change detection

   Identifying changes in a scene, such as lighting changes and camera movements.

6. Anomaly detection

   Detecting unusual patterns or behaviours that deviate from the norm, which is useful in security for spotting suspicious activities.

7. Optical character recognition (OCR)

   Extracting text from video frames is useful for reading license plates, signage and other textual information.

8. Video summaries

   Creating concise summaries of video content by identifying and compiling key frames or segments.

Video content analysis and privacy regulations

Some video analytics tools, for example, Facit’s queue management CCTV plug-in solution, track objects as opposed to capturing people’s personal data such as faces. Object counting does not pose a risk of breaching privacy regulations (e.g., GDPR).

However, CCTV footage of staff, customers and the general public that shows people’s faces, licence plates and any other personally identifying information must be treated responsibly so that it complies with privacy regulations.

Before video can be shared with or viewed by third parties, all personal data must be redacted (masked or removed). For example, video that is submitted as evidence in court must be redacted to remove all but the subject(s) of interest.

Similarly, videos that are used as training materials must be redacted to protect the privacy rights of the people recorded, unless it is possible to obtain the express consent of all the people in the scene.

Facit’s Identity Cloak redacts personal data in video in real-time and post-event to ensure privacy compliance.

Applications for video content analysis

Surveillance and security
Automated monitoring of public and private spaces for threats, suspicious activities and incidents. Examples include intrusion detection, perimeter protection and crowd monitoring.

Retail analytics
Analysing customer behaviour in stores to optimise layout, improve customer service and increase sales. Applications include heat mapping, queue management and demographic analysis.

Traffic and transportation
Monitoring and managing traffic flow, detecting accidents and enforcing traffic rules. Use cases include vehicle counting, speed detection and congestion analysis. Monitoring behaviour and safety on trains and buses.

Healthcare
Monitoring patients in hospitals for fall detection, patient movement, and to ensure compliance with medical protocols. Telemedicine applications for remote patient monitoring.

Smart cities
Integrating video analytics for urban management, such as monitoring public spaces, improving public safety and managing utilities. Applications include waste management, energy use monitoring and public transport management.

Industrial automation
Monitoring production lines for quality control, detecting defects and ensuring safety compliance. Use cases include counting products, identifying defects and monitoring worker safety.

Video content analysis considerations

• Data privacy and security
  Handling and storing video data securely to ensure compliance with data protection regulations.

• Accuracy and reliability
  Ensuring that algorithms can perform accurately in diverse conditions, for example varying lighting and complex backgrounds.

• Scalability
  Developing systems that can scale to handle large volumes of video data from multiple sources.

Video content analysis is a rapidly evolving field with vast potential across multiple industries. Its success hinges on advancements in AI and machine learning, as well as on addressing the ethical and technical challenges associated with video data.

Feeding video data to the system

The data to be analysed can be ingested from various video sources. The most common are CCTV cameras, traffic cameras and online video feeds. The majority of video sources can be integrated into the analytics solution.

The breadth of the video coverage and the amount of data collected determine the breadth and, to some extent, the accuracy of the analytics.

How do people use video content analysis to improve performance?

Video content analysis (VCA) has become an essential tool for improving performance in various fields, including sports, education, business and law enforcement.

VCA in sports:

Technical skills

• Motion analysis: Coaches and athletes use video to break down movements frame by frame to identify areas for improvement in technique, such as the swing of a golf club.

• Biomechanics: Analysing the mechanics of movements to ensure efficiency and prevent injury.

Tactical Analysis

• Game review: Teams review footage of games to study opponents' strategies and develop counter-strategies.

• Positional play: Players can observe their positioning and movement in relation to their teammates and opponents to improve decision-making.

Performance metrics

• Statistical analysis: Combining video with statistical data to evaluate performance metrics such as speed, agility and accuracy.

• Progress tracking: Comparing current performance with past videos to track improvement over time.

VCA in education:

Teaching and learning

• Self-observation: Teachers can record their lessons to review and refine their teaching methods in order to improve engagement and clarity.

• Student feedback: Students can record presentations or practice sessions to receive feedback from teachers and peers.

Skill development

• Repetitive practice: Students can watch instructional videos repeatedly to master complex concepts or skills.

VCA in business:

The latest video content analysis functions with AI and machine learning that enable automated video content analysis to identify patterns and provide business insights.

By employing video content analysis, individuals and organisations can gain valuable insights into performance in order to make operational improvements.

How is video content analysis used in retail?

Video analysis in retail is used variously to enhance operational efficiency, customer experience and security. Key applications include:

Customer behaviour analysis

• Foot traffic analysis: Retailers use video analytics to monitor and analyse the flow of customers in and out of a store and in different store locations. Analysis helps in understanding peak hours, popular products and optimal staff resourcing.

• Dwell time: By measuring how long customers spend in different areas, retailers can identify which displays or products attract the most attention and optimise store layout accordingly.

Queue management

• Video analytics can monitor queue lengths and waiting times at checkout counters. When queues get too long, the system can alert staff to open additional counters in order to reduce customer wait times, improve customer satisfaction and prevent purchase abandonment.

Loss prevention and security

• Video content analysis helps in identifying suspicious behaviours such as loitering, shoplifting and employee theft. Advanced systems can alert security personnel in real-time to improve loss prevention efforts.

Customer demographics

• Advanced video analytics can estimate customer demographics such as age, gender and even mood. Demographic data helps in tailoring marketing strategies and personalised customer experiences.

Heat mapping

• Video content analysis can create heat maps of customer movement and engagement within a store. Heat maps help retailers to understand high-traffic and low-traffic areas and optimise product placements and store layouts.

In-store advertising and promotions

• By analysing customer behaviour and demographics, retailers can tailor in-store advertising to influence specific audiences effectively by improving the relevance and impact of promotions.

Compliance and safety

• Video analytics can ensure that safety protocols are followed, such as monitoring occupancy or, during a pandemic, social distancing or mask-wearing. VCA can also help stores to comply with regulatory requirements.

Staff performance

• Video analysis can be used to monitor staff interactions with customers to help in assessing and improving customer service quality. Video content analysis helps to identify training needs and footage can be redacted to become training resources.

Omnichannel integration

• Integrating video content analysis with other data sources such as mobile apps, loyalty schemes and online behaviour can provide a comprehensive view of customer preferences and behaviours, which enables retailers to enhance omnichannel retail strategies.

By adopting video content analysis, retailers can improve operational efficiency, enhance customer experience, reduce losses and stimulate sales growth.

Retailers benefit most from video analytics

Retail is the industry sector that has benefited most from the use of video analytics services. Smart video analysis used in smart zones and retail is so advanced that retailers have the ability to gain a deep understanding of consumer behaviour.

Bricks-and-mortar retailers can make the physical space within which they operate work better for them by capturing unprecedented levels of insight and business intelligence. Such data was historically very difficult to acquire. Today, retailers have at their fingertips actionable data on traffic flow, workforce management, store design and merchandising, supply, security and compliance.

Video intelligence helps retail businesses to align their targets for personnel, finances, overheads and margins, with new evidence-based data.

How retail video analytics may become commonplace in the future is indicated by emerging outlets such as Amazon Go. Video analytics eliminate check-outs to simplify the customer’s shopping experience and enable customers to walk out of the store having been charged automatically for the items they have selected.

Central processing vs edge processing

Video analysis software can be run centrally on servers located in a monitoring station, in other words central processing. Alternatively, analysis can be embedded in the cameras themselves, which is known as edge processing.

The selection of central and edge processing is often determined by bandwidth and storage considerations. Legacy systems catered only for central processing, which lends itself to extensive post processing, while edge processing can be pre-configured only to send data related to specific event types.

Defining situations and training models

After installation, it is possible to introduce models and train the system to become increasingly accurate at identifying events. For example, recognising vehicle types such as moving and stationary trucks, cars and motorbikes. In this instance, the data can be used to detect a possible crash.

Classifying objects such as people and vehicles, animals and man-made structures, in combination with local geographical data, enables the system to become smarter over time.

Image datasets are available that simplify the training of new models, while pre-trained models are available to fine-tune models for a specific use case.

Growing public understanding of the value of video analytics

Video analytic software is now capable of producing insights and reports that facilitate increased operational efficiency across all business types, by measuring and optimising what we do and how we act.

Perhaps video analytics entered the public’s consciousness most recently and most comprehensively through its development in sports.

Football fans cannot escape the controversy and ongoing human errors that surround the use of the otherwise accurate VAR (video assisted referee). Then again, technological change has always been viewed with scepticism, until it proves its value.

Used correctly, video analysis of athletic performance provides objective insights into anything observable, repeatable and improvable with a player’s movement and technique. Technology helps trainers to plan sports development and enables managers to make tactical decisions based on live data, such as distance covered, pitch heatmaps, passing accuracy, and more.

The application of video analytics in the wider community

Almost every industry around the world has reaped the rewards of wider video analysis. Those benefitting include operators making use of video intelligence in retail, commercial offices, warehouses and workspaces, as well as in the travel, transport, leisure, education and medical sectors.

The benefits of healthcare video analytics

Healthcare staff from surgeons to nurses benefit in many ways from the use of smart video analytics services. Surgeons use video analytics as a way of detecting minute mistakes, which improves clinical procedures over time in a way that was not previously possible.

The training of healthcare professionals is improved by the advent of remote procedure observation. Hospitals are generally able to improve the way they operate owing to regular insights into staff processes, and staff interaction, for example with people and equipment.

Video analytics in healthcare support incremental improvements that contribute to the health of patients and senior management’s awareness of best practice.

Away from hospitals, video analytics solutions are deployed to manage the health and safety of potentially vulnerable people, where cameras can detect in real time if a person has fallen, such as an elderly person living alone or a remote worker operating in a hazardous environment.

How is video content analysis used in healthcare?

Video analysis in healthcare leverages advanced technologies to enhance patient care, improve medical training and streamline operations. Key applications include:

Patient monitoring and diagnosis

• Remote patient monitoring: Video analysis helps in monitoring patients remotely, especially those with chronic conditions. Cameras can track vital signs, detect falls and alert caregivers to potential emergencies.

• Behavioural analysis: For patients with neurological conditions or mental health issues, video analysis can aid in observation and analyse behaviour patterns to help diagnose conditions like autism, dementia and depression.

• Postoperative monitoring: Continuous video monitoring can track a patient's recovery process and detect complications early to ensure timely intervention.

Surgical assistance and training

• Surgical navigation: During surgeries, real-time video analysis assists surgeons by providing augmented reality overlays that enhance precision in complex procedures.

• Training and simulation: Video recordings of surgeries are used for educational purposes to enable medical students and professionals to learn from real cases and improve their skills through detailed review and analysis.

• Error reduction: Video analysis can identify and highlight deviations from standard procedures, which helps to minimise human error and improve patient safety.

Rehabilitation and therapy

• Physical therapy: Video analysis can track a patient’s movements during rehabilitation exercises and provide real-time feedback to maximise recovery and prevent injury.

• Virtual reality therapy: Combining video analysis with VR, patients can undergo immersive therapy sessions that are monitored and adjusted based on real-time feedback from the system.

Operational efficiency

• Workflow optimisation: Hospitals and clinics use video analysis to monitor the flow of patients, staff and equipment, which helps to identify bottlenecks and optimise processes.

• Security and safety: Ensuring the safety of patients and staff is critical. Video surveillance systems enhanced with video content analysis can detect unauthorized access, monitor hazardous areas and ensure compliance with safety protocols.

Research and development

• Clinical trials: Video analysis is used to monitor participants in clinical trials to ensure accurate data collection on the efficacy and safety of new treatments.

• Biomedical research: High-resolution video microscopy and imaging techniques are used to observe cellular and molecular processes, which helps clinicians to understand diseases and development new therapies.

Telemedicine

• Virtual consultations: Video analysis can enhance telemedicine by providing tools for remote examination, such as analysing wounds or monitoring respiratory conditions.

• Patient engagement: Tools such as emotion detection help in understanding patient concerns and improving doctor-patient communication during virtual visits.

Elderly care

• Activity monitoring: For elderly patients, especially those living alone, video analysis can monitor daily activities to ensure that they are following routines and to detect any unusual behaviour that might indicate health issues.

• Fall detection: Advanced video systems can detect falls in real-time, which facilitates an immediate assistance and reduces the risk of serious injury.

Infection control

• Hygiene compliance: Video content analysis can be used to monitor and ensure compliance with hand hygiene practices and cleaning protocols in healthcare facilities, which is critical for preventing hospital-acquired infections.

Video analysis in healthcare is a multi-faceted tool that enhances patient care, improves training and operations, and supports research and development, all of which contribute to a more efficient and effective healthcare system.

Video analysis in education supports continual innovation

The education sector relies on continual innovation and adaptation. Individuals must find new ways of learning about themselves and improve in a cycle of continuous professional development.

Teachers are asked to become more self-aware and improve the way they present their lessons in order to enhance the student experience and learning environment.

Video analytics services enable experts to provide constructive feedback to teachers at every stage of their development in order to help them become and remain effective educators.

Transport: video content analytics keep us on the move

American transportation entrepreneur, Robin Chase, co-founder of Zipcar, said:

“Transportation is the centre of the world! It is the glue of our daily lives. When it goes well, we don't see it. When it goes wrong, it negatively colours our day, makes us feel angry and impotent, and curtails our possibilities.”

Transport is a necessity for the vast majority of people. When transport operators have issues, they have a measurable impact and are widely publicised. As a result, it is natural that video analysis is integrated into the control and monitoring of most transport types.

Rail and air travel, motoring, and interactions at transport hubs have all been enhanced by the use of video analysis. For example, video analysis combined with improved artificial intelligence improves traffic flow management and reduces congestion. Video analysis is also used to detect suspicious behaviour and anticipate potential dangers before they develop.

Analytics at work: how Transport for London (TfL) makes widespread use of video analytics

TfL’s explanation of its use of video analytics is informative and encouraging for privacy sceptics. The bulk of TfL’s CCTV operations support TfL’s statutory functions, in particular, to deliver “integrated, efficient and economic transport facilities and services to, from and within Greater London.” The main purposes of TfL’s CCTV operations and video analytics include:

• Protecting the health and safety of employees, customers and members of the public

• Protecting property and other infrastructure

• The management and investigation of major incidents

• Preventing and detecting crime and antisocial behaviour

• Realtime traffic monitoring

• Supporting the efficient management and operation of our road and rail networks

Video analytics are also used to create a visual summary of the operational changes TfL makes from time to time. Examples include: improvements to the layout of road junctions, or cycle lanes, or where traffic signal timings ae amended. Videos are used to demonstrate how changes contribute to the Mayor's Transport Strategy and other initiatives such as the Healthy Streets Approach.

TfL emphasises that the focus of any photographs or videos is the road network itself and not individual pedestrians, vehicles or cyclists. When personal data is captured inadvertently, redaction (blurring) is used to minimise the extent that people and cars are recognisable.

Video analytics are key in in the development of smart cities when an increase in traffic can result in an increase in accidents and traffic jams if adequate traffic management measures are not taken.

Security video analytics are in a class of their own

As we suggested at the outset, video surveillance was the original home of video analytics. It is therefore not surprising that major advances in security applications have involved advanced analytics.

Facial recognition and license plate recognition are used to identify people and vehicles in real-time in order to make informed decisions. The reasons for using video intelligence may range from searching for a suspect to detecting missing cars and persons, in both ‘live’ and stored video footage.

Crowd management is another key function of security systems. Video analysis tools can help to manage busy shopping malls, hospitals, stadiums and transport hubs. Video intelligence is used to trigger alerts when an occupancy threshold is reached, or when prohibited movement is detected.

One of the latest developments in video analytics is camera analytics software, or ‘edge’ processing. ‘Edge’ processing eliminates the need for central servers as the video processing is embedded in the cameras themselves.

How is video content analysis used in law enforcement?

Video content analysis in law enforcement is a powerful tool that is used to enhance public safety, solve crimes and increase the efficiency of police work.

1. Crime prevention

Surveillance systems: Continuous monitoring through surveillance cameras in public areas helps to deter criminal activity. Advanced video content analysis can identify suspicious behaviours in real time to facilitate immediate intervention.

2. Crime investigation

Evidence collection: Video footage from security cameras, body-worn cameras, dashcams and public/private CCTV systems provide crucial evidence in criminal investigations.

Automated facial recognition systems can identify suspects, missing persons or individuals of interest from video footage.

Object tracking tools can track the movement of vehicles, persons or objects across multiple cameras to reconstruct events associated with a crime.

License plate recognition can be used to identify vehicle license plates to track vehicles involved in criminal activities.

3. Real-time monitoring

Live feeds: Law enforcement agencies monitor live feeds to respond swiftly to incidents, manage crowds and ensure public safety during events.

Automated alerts: Systems trigger alerts for predefined events, such as a person entering a restricted area or an unattended bag in a public space.

4. Post-event analysis

Behavioural analysis: Reviewing footage to analyse the behaviour of individuals in the context of a crime helps in identifying patterns that help to predict future offences.

Incident reconstruction: Reconstructing the sequence of events associated with a crime to understand how it occurred and identify all parties involved.

Forensic video content analysis: Enhancing and clarifying video footage to ensure that minute details can be observed, which might be crucial for court evidence.

5. Training and performance evaluation

Officer training: Analysing footage from training exercises to improve the skills and techniques of law enforcement officers.

Performance review: Reviewing footage from body-worn cameras to evaluate officers' interactions with the public and adherence to protocols.

Training materials: Use of redacted event recordings during officer orientation to specific event types, such as crowd control, riots and large-scale accidents.

6. Technological Integration

AI and machine learning: Implementing AI algorithms to detect and categorise activities automatically in order to reduce the workload on human operators.

Data integration: Combining video data with other data sources (e.g., emergency phone calls) to provide a comprehensive view of incidents and trends.

Video analytics concerns and considerations

• Privacy concerns: Ensuring the right balance between surveillance and privacy rights

• Legal and ethical issues: Navigating the legalities around the use of video evidence and ensuring that it is admissible in court.

Video content analysis is a crucial component of modern law enforcement that aids in everything from real-time crime prevention to detailed post-incident investigations.

Video content analysis improves AI functionality

As well as the use of video content analysis to train people, VCA is also invaluable in improving the capabilities and accuracy of artificial intelligence technologies.

Algorithms can be refined to interpret nuanced scenes and activities through the use of video content analysis.

Facit: analytics, operations, compliance

Facit enables customers to maximise the value of their video footage, from queue management and heat mapping, to loss prevention and fall detection.

Our content analysis solutions are simple to install and operate, and integrate with customers’ business intelligence suites to generate a wide range of actionable insights.

If you are interested to learn more about advances in video content analysis, or are considering maximising intelligence available to you via CCTV data, we would be delighted to discuss video analytics software, operations management and compliance automation. We can help you lead the AI revolution.