Video Analytics – I

    Today every nook and corner of public place is under surveillance camera. They work round the clock and for 365 days. Thus huge amount of video data is generated. Monitoring a video for more than 20 minutes is tedious to human beings and security officers mostly fail to detect abnormal activities. Thus there is a requirement for “machine assistance” to security officers. Watching video via Internet has become a norm. Popular video servers like YouTube, Dailymotion, and Metacafe provide video free of charge. Netflix is subscription based video server. Sixty hours of video are uploaded every minute in YouTube video servers alone [1]. Thus one can imagine the quantum of video content available in Internet. In the years to come more than half of Internet traffic will be due to video. We encounter problem choosing 'right' or cherry-pick the video from the huge pile of video scattered in Internet. Here to we require some form of “machine assistance” to ordinary viewer.

Video is made up of consecutive sequences of frames or images. Each image contain large amount of pixel information. Images offer very little prior structure to work with [2]. Earlier video databases were small and manual annotation was a possible solution. Today it is not a feasible solution. Present day computing power will manage huge size of data. But automatic analysis of video requires artificial intelligence. Video analytics is the baby step in that direction. Video analytics deals with extraction of information from video with the aid of machine assistance. Video processing means performing some image processing (like resampling or colour correction) on the video content. Most of the time extracted information is overlaid on the video for better human interpretation. Big data analytics is latest buzz word in technical world. Video analytics is considered as a subset of big data analytics.

Eighty years of entertainment video

Let us begin the post with a piece of information. In the Internet more than 50 percentage of traffic is due to video transmission. Colour movies emerged in early 1930s and it was the only source of video at that time. TV emerged in 1950s and added huge collection video to the world. The evolution of video from 1934 to 2014 (80 years) is to be discussed in this post. 

Video Classification

For a general public, a movie stored in video cassette or Video Compact Disc (VCD) is considered as a video. But a graduate in electronics engineering will quantify video as “sequence of still pictures (frames) that are displayed on screen in a very small time interval.” Video can be classified into three major categories namely; entertainment video, Industrial video and Surveillance video. As name implies entertainment video encompasses movies and TV programmes. As per the definition Black and White (B&W) silent movies come under the category of video.  But in this post it is restricted to colour talkies (colour movie with soundtrack). In manufacturing industries video cameras are extensively used to capture the production processes. Videos are analysed and features are extracted. These features are used as feedback in production processes. Machines fitted with camera replace human beings. They work 24x7, seven days a week without a holiday. As they don't require any pay rise, there is no strike in factories. The video content generated by machine vision systems can be branded as Industrial video. Most of the public places are monitored by a surveillance system. This cost effective method reduces the requirement of Beat Officers (police who are on patrol), aftermath of crime recorded video acts as clues to nail the culprits and acts as prosecution evidence in the court of law. Video content is increasing leaps and bounds. 

Gigapixel Images

A 1.8 gigapixel video surveillance camera named ARGUS was built jointly by DARPA (Defense Advanced Research Projects Agency) and the US Army. It is capable to pick out a sleeping dog in the Earth from the altitude of 20,000 feet (6 km).  In other words, it can resolve details as close to six inches. Quite amazing! The ARGUS can be attached to drone (unmanned aeroplane remotely controlled) and taken to a height of 20,000 feet to observe 25 square kilometres at any instant. Thus entire New York City can be brought into surveillance by two ARGUS attached drones to hover over the city.  The entire Manhattan is under observation 24x7.  

Like ARGUS, AWARE-2 is another gigapixel camera. It was used to study the behaviour of tundra swans present in Pungo Lake, USA [1]. With AWARE-2 camera gigapixel snapshot was taken [1] and found 656 swans swimming in the lake and 27 flying above the lake at that instant. Scientists can use the snapshot to track individual swan or study the swans' flock (group) behaviour.  Existing scanning panoramic camera cannot achieve this feat. Use of gigapixel is not confined to ecology alone. It can be applied in fields like urban planning, traffic control, forestry, archaeology and so on.

Gigapixel snapshot of Budapest city, Hungary. Inset: two landmarks building in the city. Zooming operation  on gigapixel image helps us find the buildings. Image courtesy www.photographyblog.com [7]

Digital Society

Today we are living in digital society.  Due to rise of digital technology basic fabric of society is almost changed.  For example, in India twenty years back, cellular phones were bulky and were a status symbol. Users seldom revealed their cell phone numbers, as incoming calls were also metered or charged. Now sleek cell phones are the norm and each Indian possesses a cell phone. The word “Privacy” is unknown to Indians earlier, now they practice it. Thus digital technology has started shaping the society. Earlier posts like tribal, agrarian and industrial society discussed length and breadth of technology-society association. In this post emphasis will on the digital technology.

Technology has so advanced that by click of a button we get any information we want. Is this means we are far superior than our ancestors? The answer is Yes and No. Internet has become giant virtual library. Internet contains lot of text information and videos which are structured, static and suited for one way communication (author to reader or viewers). We have honed our skills to surf and sift information from Internet. By this reason, yes is the answer.  We lack the ability to communicate, leave alone extracting information from fellow humans. This is because human to human interaction is much unstructured, dynamic and two way (author to listener, and listener to author). Communication is complex and lot of training is required to excel [1].  We had fishermen, who observed the cloud and predicted the weather and other vital information required for fishing. In agrarian society humans acted as an information repository.  

Today society banks more on man-machine interaction rather than human-human interaction. This may result in assorted individuals (Man, woman, boy and girl) instead of a family (Father, mother, son and daughter) staying in a house. Thus our challenge is to make the digital devices to assist us in improving human-human interaction.

Digital Devices

We come across devices like digital clock, mp3 player, audio compact disc player, Video Compact Disc (VCD) player, Digital Video Disc (DVD) player, computer, cellular phone, Tablet and so on. These diverse devices can be brought under umbrella of digital devices. Analog devices operate on analog signals which are continuous in time as well as in value. Digital signals are discrete in time and can hold only two values i.e. ones and zeros. Dominance of digital devices over analog counter parts is due to the axis of IC fabrication, Computer and Internet.

Figure 1. Digital Clock | Image courtesy Wikipedia

Pre-Digital Society

In earlier posts tribal, agrarian and industrial society were discussed. In this post, post-industrial society will be discussed.  One may wonder about the title and they may have an urge rename it as “information society.” Lot of definitions exists for information society [1]. Industrialization of a nation is gauged by amount of petroleum or energy usage. Likewise two metrics can be used for information society. First is extent of information usage. Next is economy of a country should hang on information production (for example movies, music videos, books, information equipments etc). So it is better to leave the discussion on information society to economists and social scientists. The present digital society may lead us to information society. 

       This is one part of series of articles dealing with evolution of societies. In the present society we encounter lot of displays. These articles try to find out the answer for “rise of display and smart devices.” I firmly believe engineering, technology, economics and end user behaviour has to be studied as a whole and not in isolation.  That is why article like this appear in this blog which is dedicated to Digital Image Processing. 

The post-industrial society can be broadly classified into pre-digital and digital society. In Industrial society electricity was used a form energy like coal and petroleum. In digital society, electricity is used to transport information. In digital society, information production, distribution and consumption happens in binary form i.e. ones and zeros. In pre-digital society analog signals were used and optional use of electronic components.