After the transition from analog to IP video surveillance over the last few years, the industry is currently in a phase where multi megapixel (MMP) cameras receive all the buzz at security shows. This tendency created the market for a new surveillance camera type: panoramic cameras. A single camera covers an entire area continuously without leaving any blind spot. Panoramic cameras can be installed on a wall to give a 180° panoramic view or on a ceiling to give a 360° view to cover the full area below that ceiling.
Panoramic cameras transformed from a product with great potentials into a key component of video surveillance systems. The extremely wide field of view together with an extremely high resolution brings a larger coverage than conventional cameras. Two panoramic styles: Multi imager vs. panoramic lens
Panoramic security cameras are basically available in two types:
(1) Panoramic cameras that use a single extreme wide angle lens of up to 360°.
(2) Multi imager surveillance cameras that stitch the separate video streams from each sensor together in one image. These cameras usually have three or more lenses/sensors.
Fisheye lenses can form the image of the full hemispheric region located in front of the lens. Such lenses form a circular field of view of 180° horizontally and 180 ° vertically. If the sensor is large enough to capture this image, the full hemispheric region in front of the camera is captured. This is called a 360° camera. The circular image fits on the sensor and has a field of view of 180° horizontally and 180 ° vertically. If the circular image fits within the length of the sensor, but is bigger than its width, then we have a 180° camera. The camera has a field of view of 180° horizontally, but less than 180 ° vertically.
Panomorph lenses are special type of fisheye lenses which forms an image ellipse instead of an image circle. The advantage is that such lenses can make use of more pixels of the image sensor compared to a traditional fisheye lens.
360° cameras are expected to be used as a supplement to conventional security cameras. The panoramic camera is used to provide an overview of a certain area and conventional cameras are used to provide details in a narrower scene.
‐ Wider field of view
‐ No blind spots
‐ Tracking of persons or moving object through an entire room or area
‐ Lower installation costs (and thus TCO) since only one camera can cover an entire area.
Conventional fixed camera
‐ Focus on specific area with full resolution (e.g frontal identification). In essence, fixed cameras give more useful pixels per image area than the panoramic cameras made with the same sensor.
‐ No image distortion
Panoramic cameras can be used as digital PTZ cameras that monitor an entire area. The operator can pan tilt and zoom digitally to a certain part of the scene without losing sight on other regions of interest. A panoramic camera beats a PTZ camera by monitoring and recording all events and activities in a large area simultaneously. Where a PTZ camera focuses on a specific part of an area, can a panoramic camera provide a recording of the full area.
On the other hand, a typical scenario is to use a 360° camera for monitoring one area and a PTZ camera is used to zoom in on a certain event in that area. However, panoramic camera has added value with conventional cameras; panoramic camera has a tactic role while PTZ camera has an operational role:
‐ The size of a panoramic camera is a fraction of the size and weight of a PTZ camera
‐ Panoramic monitor and records full area all the time. This means live and retrospective access to a fully covered area. A PTZ camera can only show and record one part of the scene at once.
‐ Panoramic cameras can monitor and record multiple regions of interest (ROI) at once, whereas a PTZ camera can only monitor one ROI at a time.
‐ PTZ cameras have an OPTICAL zoom that provides a larger range. In certain cases, such as rooftop surveillance, this is better than the DIGITAL zoom on panoramic cameras.
‐ A clean and stylish aesthetics design is a key driver for buying a panoramic camera. Compare with a big heavy PTZ camera, a panoramic camera can be installed at any location, whereas a PTZ is more intrusive and can affect aesthetics of the building.
Resolution is the number of pixels that can be displayed. If the number of pixels on the sensor is low, the image resolution will be low, and details are not visible. Low resolution leads either to pixilated or to blurred images in places where panoramic lens distortion correction algorithms need to expand images. On the other hand, if a sensor with more pixels is used it will require a significantly higher computing power. The introduction of megapixel security cameras has improved general image quality and improves the quality of panoramic cameras. This takes away some arguments to use multi imager cameras, because with the current sensors and lenses available in the market, security companies can provide sovereign performance.
The resolution of a security camera using a fisheye or panomorphic lens is not as straightforward as conventional security cameras. A panoramic camera using a 12MP sensor will not provide a full 12MP resolution. The optical circle of the lens is smaller than the active image area of the sensor (as was seen in the drawings in section about fisheye lenses) and therefore does not cover all active pixels. The effective resolution of the image is the amount of pixels that fit within the optical circle. An example: let’s assume a 12MP sensor with 4000x3000 pixels and a size of 6.20x4.65mm. The 360° optical circle of the lens used has a diameter of 4.1mm. This would result in square that has a 7.0MP resolution with an active area of the circular image of 5.49MP. This often results in resolution game that security camera manufactures like to play. Is the panoramic camera really providing a 5MP image or this is just the sensor’s native resolution?
Besides details, the sensor in a security camera is also responsible for the dynamic range and low-light sensitivity. The usability of a panoramic image is highly dependent on the dynamic range of the sensor. With a panoramic camera the likeliness to have bright and dark areas in the field of view is much higher. A low dynamic range – either too dark or too bright images - will not be able to show details in all areas of the scene. A high dynamic range enables the user to see details in both bright and dark areas. Single exposure wide dynamic range is applied to reveal all details in dark and both bright parts of the scene.
Cameras equipped with a fisheye or panomorphic lens produce video that has a circular shape with a scene that is optically distorted. Image correction is needed to create an optimized useful image without distortion; this is called dewarping. The dewarping algorithm actually remaps the pixels in the scene to optimize the image and remove distortion. The algorithm can offer various image display possibilities, also could a dewarping library. Some of those options are a panorama view, a surround view or even real time PTZ viewing. The various video viewing modes allow the user to control the 180° or 360° images from a panoramic surveillance camera and guarantee a distortion-free video. Additionally it provides the freedom to define custom definable Regions of Interests (ROI) by adding presets.
Together with a multi megapixel resolution, dewarping requires a significantly higher computing power and higher resource consumption than ordinary cameras. The computing power determines the time needed to process the image and its transmission speed (latency). Dewarping can be done via a special dewarping algorithm on the camera (edge dewarping) or on the pc-side via a separate video client. The 4 main elements need to be carefully balanced are Resolution, Frame rate, De-warping quality and De-warping mode possibilities.
Edge dewarping means that camera itself can dewarp the images before sending them to Video Management Systems (VMS). Users just need to select the dewarped video viewing mode, and the edge dewarping can be adjusted automatically. Normally, a special Video Client or VMS will be needed to do the image correction when the edge dewarping is not available. This involves hardware and maintenance costs.
Edge dewarping is cheaper. Dewarping requires a high computing power. This is generally provided by expensive pc’s in an expensive system. With edge dewarping, dewarping is done in the camera rather than using the computer's CPU. This significantly reduces the rocessing power needed on a pc. It is not needed to buy more or more expensive equipment to process the image and all information that comes with it. This additional equipment would put pressure on the customers’ budget when buying panoramic cameras.
Easier integration. Most panoramic cameras require a special Video SDK for dewarping, this makes integration to VMS challenging. The VMS will need to support the special Video SDK. This locks down the systems to a particular dewarping enabled VMS and causes bad or no
integration with 3rd party software. This limits the wide implementation and use of panoramic cameras. Edge dewarping allows the user to see a dewarped directly from the camera which eases the work and installer or integrator need to do.
Virtual cameras. Having edge dewarping allows sending the dewarped images as separate streams. By doing this can lower the bandwidth and make the system to be more flexible. It can offer a different field of view to all users with access to the security surveillance system. The security guard could have access to all streams, but it needs to confront people entering a store with only stream with the virtual camera facing the entrance.
A very strong feature that makes panoramic surveillance camera even more useful is by applying Intelligent Video Analysis (IVA) on the video. IVA enables detecting, tracking, following and analyzing all movements on a full 360° blind spot free scene, the objects and persons in the entire space of store or warehouse could completely be included in the scene. The other benefit by using panoramic lens with IVA is no need to combine the analysis done on multiple fixed cameras.
IVA tracks multiple actions within an entire area, whereas a PTZ follow one movement and miss the others not in the current FOV. The operator determines the FOV, whereas a panoramic camera has the entire overview continuously. IVA helps surveillance operators focus on attention by identified areas. Also observing an entire area by single person pushes concentration to the limit – after only 20 minutes, an operator can miss as much as 90% of the activity in a scene. Real-time IVA processing identifies alert conditions, giving the information it needs to react swiftly and take action. As an example on the use of IVA and panoramic surveillance, airport is a suitable location to execute the function. Panoramic surveillance cameras are installed on a strategic location where it monitors the customs officers’ desk, people enter, queuing and leaving this area. Imaginary lines are added to the scene to define restricted areas. When somebody passes this line, it alerts the operator by sending an alarm. The operator can focus on that area and follow the person through the entire area without losing sight of this persons or having to switch to other cameras. Simultaneously, the other areas in the scene are being monitored without losing information.
What is the FlexiDome 7000?
Check out the "low profile" Flexidome 700 IC in this video at the 23 second mark
Dewarping Panoramic Cameras in Bosch Video Management System (BVMS)
A Bit More Technical...