Closed-circuit video surveillance systems usually involve nothing more than security personnel watching a bank of monitors that display realtime videos taken from cameras positioned elsewhere that are watching gamblers at casino tables, travelers winding through endless lines at airport security checks, or the commotion at Occupy protests.

It's a tedious, monotonous job. Most operators have an attention span of less than 30 minutes. Then their minds wonder; their eyes may be open but their mind is at the ball game or the movies or already at lunch. There's a reasonable chance they won't notice the satchel left unattended by the propane tanks or a suspected terrorist trying to blend in with the crowd.

What if the video camera was smarter than the tired, bored operator, in the sense that it could stitch the images from a dozen cameras to one monitor instead of 12, and also detect aberrant behavior or unusual, threatening activity? What if the camera was smart enough to know that there should never be an unattended satchel left near the propane tanks and that when it 'sees' a package or suitcase where it logically shouldn't be it would automatically notify the operator through some alerting mechanism?

There's a smart camera like that at Logan International Airport right now, bolted to the ceiling of Terminal A.

Big Brother's new eyes

Smart cameras, based on video analytic technology, are emerging in various security venues. One smart camera security system has been watching passengers at Logan International for the past year or so. It's a test to see if the system performs as designed.

The new video security surveillance system, named Imaging System for Immersive Surveillance, or ISIS, was conceived at the Lincoln Laboratory at the Massachusetts Institute of Technology and launched by the Pacific Northwest National Laboratory for the Department of Homeland Security's Science and Technology Directorate. The version installed at Logan, called Spiral 1, sees the entire concourse like a large fisheye lens, but that's where the similarity with most conventional surveillance systems ends.

This cutting edge technology is years ahead of the current commercial offerings, according to PNNL.

Whereas a typical fisheye lens distorts the image and can only provide limited resolution, ISIS video is perfectly detailed, from edge-to-edge. That's because the video is made from a series of individual cameras stitched into a single, live view, like a high-res video quilt.

"Wide-area views generated by typical closed-circuit television systems are often chopped up into different monitors, so the security professional has to look a bunch a different screens to get overall perspective," said PNNL's Doug MacDonald, ISIS project manager. "With ISIS, everything is now displayed on one large monitor."

Coverage this sweeping, with detail this fine, requires an extremely high pixel count. ISIS has a resolution capability of 100 megapixels. That's as detailed as 50 full-HDTV movies playing at once, with optical detail to spare. ISIS also has a zooming capability for getting up close and personal without losing clarity.

Each operator works independently with their own monitor and have access to any part of the 360-degree coverage. Further, each operator can review realtime video or access archive video taken moments or days before without affecting the operational performance of the system. Operators are able to use monitors at their desks, typically 24" flat-screens, but at the same time have access to clear, high resolution video never before provided. ISIS can see clearly up to 160 yards away.

When officials need to identify a suspect from among the throngs of passengers mulling about the airport, ISIS provides a clear enough image that operation of the entire terminal would not have to be interrupted.

In the event that a terrorist attack has occurred, forensic investigators can pore over the most recent video, using pan, zoom, and tilt controls to reconstruct who did what and when. Because these controls are virtual, different regions of a crime scene can feasibly be studied by separate investigative teams simultaneously.

A stitch in real time

The stitching technology isn't exactly cutting-edge magic. Creative photographers have for years used low-cost stitching software to create interesting high-res panoramic images, such as the famous image of the National Mall taken on Inauguration Day 2009. But those are still images, created after the scene was shot with several cameras. ISIS quilts video in realtime, while at the same time allowing operators to maintain the full field of view, even while a focal point of choice is magnified.

MacDonald said in the commercial world there are many software systems that can automatically identity suspicious activity, but they are proficient only in scarcely populated or unpopulated areas, where it's easy to spot a lone person leaving a parcel unattended.

"If someone comes and set a bag down and walks away, these systems will pick that up, but our goal is to do that in highly crowded environments," he said.

Other ISIS features, many of which are commercially available, are provided by a suite of software applications called video analytics. One app lets operators define restricted areas, for which ISIS provides an alert the moment the area is breached. ISIS is designed for use in any environment where surveillance of large open areas is required.

Another feature lets operators select a specific target, such as a suspicious person, package, or pickup. The system then tags the subject and follows it, automatically panning and tilting as needed. Video analytics at high resolution across the universal field of view, coupled with the ability to follow objects against a cluttered background, provides high-quality situational awareness as incidents unfold.

"What we've found in the past is the typical security professional watching a bank of monitors has an attention span of about 20 minutes," MacDonald said. "What we wanted to do is give these professionals a tool that can continuously monitor an area, and then provide operator alerts to abnormal events or locate intruders in off-limits areas."

MacDonald said he sees ISIS being used in any crowded environment or any location where homeland protection demands continuous monitoring for abnormal events.

Lights, camera, controversy

Many of these surveillance advances are troubling to those as interested in preserving freedom and civil liberties as they are in combating crime or detecting terrorists. They believe government dragnet surveillance activities are is not directed solely at suspected terrorists and criminals, they are instead directed at everyone, and therefore tend to erode the right to privacy and the freedoms of speech, association, and religion.

The ACLU website states, "Increasingly, the government is engaged in warrantless surveillance that vacuums up sensitive information about innocent people. And this surveillance takes place in secret, with little or no oversight by the courts, by Congress, or by the public."

Steven Surfaro. Axis Communications' industry liaison, said "the benefits of public surveillance in crime reduction have been studied and documented extensively and outweigh privacy concerns, as long as best practices are adhered to."

Surfaro said any erosion of privacy or civil liberty in video surveillance are overcome when a system is designed and used according to established best practices.

ISIS spiral 2

The ISIS Spiral 1 system installed at Logan International has worked well enough to be exhibited to visiting representatives of local law enforcement officials, branches of the military, as well as state and federal homeland security agencies. Several commercial entities with a stake in protecting the nation's infrastructure have also shown interest.

A second generation prototype of ISIS, called Spiral 2, that more than doubles the resolution to 200 megapixels is currently undergoing operational testing. The results will be used to refine the technology before Spiral 2 is installed at Logan, probably sometime in Spring 2012.

The partnership of Logan International and DHS began in December 2009, allowing potential Homeland Security end users the opportunity to evaluate the ISIS technology. Beyond the potential for enhancing security at the nation's airports, if successful, the current testing at Logan could pave the way for the eventual deployment of ISIS to protect other critical venues.

ISIS Spiral 2 will use custom sensors and video boards, longer range cameras, higher resolution, a more efficient video format, and a discreet, chandelier-like frame no bigger than a basketball. Eventually, DHS plans to develop a version of ISIS that will use infrared cameras to detect events that occur at night.

"We're continuing to advance the video analytic capabilities of the software package, and incorporate features as requested by potential users that have been part of the demonstrations," MacDonald said. "The response to this system and the technology so far has been extremely favorable by all."

Many of the ISIS capabilities originated in technology previously developed at MIT's Lincoln Laboratory for military applications. The current ISIS system was built with commercial off-the-shelf cameras, computers, image processing boards, and software.

ISIS was nominated for the Christopher Columbus Homeland Security Award for cutting edge technology that could be applied to the protection of the nation's assets.