A 1.8 gigapixel video surveillance camera named ARGUS was built jointly by DARPA (Defense Advanced Research Projects Agency) and the US Army. It is capable to pick out a sleeping dog in the Earth from the altitude of 20,000 feet (6 km). In other words, it can resolve details as close to six inches. Quite amazing! The ARGUS can be attached to drone (unmanned aeroplane remotely controlled) and taken to a height of 20,000 feet to observe 25 square kilometres at any instant. Thus entire New York City can be brought into surveillance by two ARGUS attached drones to hover over the city. The entire Manhattan is under observation 24x7.
Like ARGUS, AWARE-2 is another gigapixel camera. It was used to study the behaviour of tundra swans present in Pungo Lake, USA [1]. With AWARE-2 camera gigapixel snapshot was taken [1] and found 656 swans swimming in the lake and 27 flying above the lake at that instant. Scientists can use the snapshot to track individual swan or study the swans' flock (group) behaviour. Existing scanning panoramic camera cannot achieve this feat. Use of gigapixel is not confined to ecology alone. It can be applied in fields like urban planning, traffic control, forestry, archaeology and so on.
 |
| Gigapixel snapshot of Budapest city, Hungary. Inset: two landmarks building in the city. Zooming operation on gigapixel image helps us find the buildings. Image courtesy www.photographyblog.com [7] |
Gigapixel images – Opportunities and Challenges
Gigapixel images are wide angled, with large spatial resolution and high dynamic range. In a conventional image all information in the picture can be picked up in a glance. In gigapixel images viewers can spend hours to discover objects or subject of interest. The nature of interaction provides a feeling of exploration. Thus users have an option to upgrade themselves from viewers to explorers. Apart from the new 'experience' which provides amazement, helps to build annotated images for scientific studies.
The 1.8 gigapixel ARGUS operating at 12 fps (frames per second) generate around 75 GB per second. This huge data has to be stored, transferred and processed and valuable information has to be extracted from the data in real time. The gigapixel camera construction and image acquisition is one aspect and processing of huge data generated by the camera is another aspect of interest. We will go through one by one.
Gigapixel images using conventional camera
Gigapixel images are extremely big photos. Wide lens cameras are required to capture a large area like Pungo Lake. Wide lens cameras are large in size and aberrations proportionally increase with size. Thus users are left with only two choices; either bear the cost or bear the blur due to aberrations. Quality of the picture is highly dependent on the lens that is used to focus the light into the image sensor array. Professional quality lenses produce crisp pictures but they are very costly. Professional quality lenses use high quality raw materials, very high quality control and importantly hand crafted by humans. Consumer grade lenses are produced in batches by machines, process requires less human intervention and not so strict quality control [2].
Field of View (FOV) of camera is determined by focal length of the lens and image sensor's size. Professional camera lenses are made up of several 'lens elements' to minimize aberrations. Following Web links will help you to understand the basics of FOV, focal length, types of lenses and aberrations [3], [4], [5]. This will help to understand gigapixel camera articles.
Construction of gigapixel camera
Unlike conventional camera, gigapixel cameras are constructed using array of micro-cameras. Each micro-camera is a conventional camera that uses more than 10 megapixel image sensor. As the lens size is relatively small corresponding aberrations is also quite low. Micro-cameras are triggered to capture images at the same instant. Thus precise control of camera is very essential. For example, ARGUS uses 368 micro-cameras, each with five megapixel image sensor. AWARE-2 uses 98 micro-cameras, each with 14 megapixel image sensor.
Assembling gigapixel image
As stated earlier construction of gigapixel of camera is one phase and assembling of gigapixel image is another phase. Captured images have to be stitched together to form the gigapixel image. This acquisition of gigapixel image involves more than 10^9 detectors. The image resolution is very high for conventional monitors to handle. So, gigapixel images have to be downsampled and projected on the screen. The real power or uniqueness of gigapixel image lies in exploration of image. Viewer application is required to help users to explore. Viewer application has to take huge data as input and process in near real-time and provide output.
Stitching images from micro-cameras involves following stages; production of radiance images, geometric alignment of images; production low-resolution images for display. The raw data from camera is taken and converted into RGB values. This process is called demosaicing (in conventional camera it will be carried out in the camera itself). Correction for lens aberration is performed. This is called de-vignette. Among the micro-cameras minor difference in shutter speed and illumination is expected. Thus few more image processing operations are performed to produce radiance images. The first step in geometric alignment is finding radial distortion. Next step is to match image with eight overlapping (neighbouring) images. Even after this errors arise due to moving objects. These are corrected by using appropriate algorithms [6].
Gigapixel viewer
The sizes of gigapixel images are very huge to transfer like conventional JPEG images. Next, panning and zooming operations are compulsorily used in gigapixel image viewing. These two requirements bring in following constraints. The gigapixel image is stored in pyramidal order and wide-angled and low-resolution image is shown first to the user. As expected, user may perform zooming operation on select portion of image. The image FOV has to be changed accordingly without creating a visual displeasure to user. As the image is zoomed the average luminance has to be adjusted to view the contents clearly. In gigapixel images, world coordinate systems are used. Generally cylindrical and perspective projections are used. This helps to decouple the way gigapixel image is stored and displayed to the user [6].
Gigapixel camera opens up new vistas for business and entertainment. People can have virtual tour on remote location and 360 degree view of products can be carried out by the click of the mouse. This obviates the need for costly show rooms. As gigapixel camera process and transfers huge data, Gigabit network will become need of the hour. On the other hand 24x7 surveillance possibilities create privacy concerns.
Sources
- D. J. Brady, M. E. Gehm, R. A. Stack, D. L. Marks, D. S. Kittle, D. R. Golish, E. M. Vera and S. D. Feller, “Multiscale gigapixel photography,” Nature, vol. 486, pp. 386–389, June, 2012
- Why Are Some Lenses So Expensive? | http://photographylife.com/why-are-some-lenses-so-expensive
- Understanding Camera Lenses | http://www.cambridgeincolour.com/tutorials/camera-lenses.htm
- Photography Basics - Focal length - Discover Digital Photography | http://www.discoverdigitalphotography.com/2011/photography-basics-focal-length/
- Digital Photography Essentials #001 @Digital Outback Photo | http://www.outbackphoto.com/dp_essentials/dp_essentials_01/essay.html
- Johannes Kopf, Matt Uyttendaele, Oliver Deussen, Michael F. Cohen, Capturing and Viewing Gigapixel Images, ACM Transactions on Graphics (TOG), vol. 26, no. 3, article no. 93, July 2007.
- 71-Gigapixel Photo Sets New Record – PhotographyBLOG | http://www.photographyblog.com/news/71-gigapixel_photo_sets_new_record/
