What happened?
Video surveillance is undergoing a fundamental technological transformation. For decades, the core task of camera systems was to generate, transmit, and record the best possible images. This focus is increasingly shifting: modern systems are now expected to recognize relevant situations themselves, evaluate image information directly within the camera, and link different sensors with one another. The camera is thus evolving from a pure image source into an intelligent sensor within a broader security architecture.
The Details
On the classic camera side, technical development is far from complete. Dr.-Ing. Markus Müller of Fraunhofer IOSB, Department of Video Analysis Systems, expects further progress in geometric and radiometric resolution. In his assessment, thermal imaging sensors will also become more powerful and more cost-effective, enabling more effective overall systems. It is likely that systems with two or three different channels will be used more frequently in the future – for example, visual-optical cameras combined with near-infrared and thermal infrared. This development can already be observed on the market: multisensor cameras are becoming more widespread, and improved low-light technology not only delivers better images in low-light conditions but also improves the conditions for more reliable automatic object and event detection.
Edge Computing and AI in the Camera
At the same time, the system architecture is changing. Instead of transmitting all video streams centrally to a server, cameras are taking on a growing share of processing themselves – edge computing is considered one of the industry's most important trends. Modern cameras can detect people or vehicles, classify objects, and generate metadata before information is passed on to a video management system or a cloud platform.
Research on behavior recognition at Fraunhofer IOSB shows how far automatic interpretation can go. Digital skeleton models, or body keypoints, are generated from detected persons, and their movement behavior is then analyzed. The system is intended to detect actions such as hitting or kicking and alert an operator to a potentially relevant situation. The identity of the observed person is not required for this – a digital skeleton has no identity, origin, skin color, or gender.
Sensor Fusion in Perimeter Protection
This development is particularly evident in perimeter protection. Large outdoor areas place high demands on range, detection reliability, and the avoidance of false alarms. Visible-light cameras, thermal imaging technology, radar, and other detection sensors are increasingly taking on different tasks within a shared system. A radar can detect a moving object independent of lighting conditions and determine its position, while an optical or thermal camera is used for verification and classification. AI-based video analysis then distinguishes between people, vehicles, or animals and reports only relevant events to a control center.
However, complex scenes remain particularly challenging. According to Müller, automatic video analysis under uncontrolled conditions continues to face considerable challenges. Different movements, changing backgrounds, occlusions, and many objects appearing simultaneously make robust interpretation difficult. Future development will therefore not move exclusively in the direction of AI – hybrid approaches that combine classic signal and image processing with neural methods can deliver better results. A key bottleneck remains the availability of suitable and sufficiently extensive training data.
New Sensor Principles
In addition to intelligent evaluation, new sensor principles could expand video surveillance. One example is range-gated imaging, or gated viewing: the scene is illuminated with very short laser pulses, and the camera is opened only for a precisely defined time window. Since the travel time of the light is known, the system can selectively capture reflected light from a specific distance range. Light that is scattered back by fog, rain, or snow largely does not enter the relevant image. In multisensor systems, this technology is combined with day/night and thermal imaging cameras. Applications range from maritime surveillance and border protection to the detection and tracking of objects and drones.
Fraunhofer IOSB is also investigating gated viewing for poor visibility conditions. By selecting returning photons based on timing, disruptive reflections from the foreground and background can be suppressed, while distance information can also be obtained. Another complementary technology is millimeter-wave radar, which can detect objects even in darkness, smoke, or fog and determine their distance and speed when optical systems reach their limits.
Context
For future security systems, this points less toward the dominance of a single sensor and more toward a combination of different physical measurement principles. The next stage in the development of video surveillance will therefore not be determined by a single technology: better optical and thermal sensors, range-gated methods, radar, edge computing, and AI are all developing in parallel. In addition, more powerful methods for the semantic interpretation of images are emerging – Fraunhofer IOSB is now also working with foundation models and large vision-language models that can not only classify image content but place it within a broader semantic context.
This development is fundamentally changing the task of a video system. In the future, what matters is less about generating as many images as possible and more about reliably filtering out, from existing sensor data, the information needed for a specific security task. For operators of security technology – whether in facility protection, securing hazard points such as an offertory box or a deposit safe, or as part of larger perimeter solutions – this means that the quality of a system is increasingly measured by its ability to assess situations, no longer by resolution or image quality alone.
Practical Tips
- When purchasing new equipment, don't focus solely on resolution, but also on multisensor and edge-processing capabilities – these largely determine the reliability of automatic detection.
- For perimeter protection projects, it is worth examining combined solutions involving radar, thermal imaging, and optical cameras to reduce false alarms and improve detection under changing weather conditions.
- In environments with frequently limited visibility (fog, rain, snow), operators should evaluate the suitability of range-gated methods or gated-viewing technology.
- Since complex scenes with many objects remain a challenge for AI systems, humans remain an indispensable decision-making authority – systems should be designed as support tools, not as fully automatic replacements for operators.
- For privacy-sensitive applications, methods such as skeleton- or keypoint-based behavior recognition can be an option, as they do not require identifying the observed person.
Outlook
The camera is increasingly becoming part of a networked sensor system that recognizes and evaluates situations and specifically alerts people to relevant events. Research institutions such as Fraunhofer IOSB are actively driving this development through work on hybrid AI approaches, foundation models, and new sensor principles. For the security industry, this means ongoing transformation: moving away from pure image recording toward integrated systems that combine different physical measurement principles and automatically filter out relevant information – while preserving human decision-making authority in safety-critical situations.
