CIE Components in Electronics
Founded in 2005, MotionDSP started with licensed technology from the University of California and spun that into a tool for enhancing video image quality through a process called computational super-resolution reconstruction — rebuilding each frame of video in a sequence by referencing information in the other frames around it. While the results were impressive, the limitations of processing capabilities at the time translated to time-consuming processing. According to the CEO Sean Varah, “When we ran the original algorithms, it took more than a day to get a single frame of standard definition video reconstructed. But before we came along, there really wasn’t software to improve video resolution in an automated fashion.”
MotionDSP’s Ikena software solution caught on as an off-line forensics tool in law enforcement, with customers including the US Secret Service and Scotland Yard. Video evidence — whether it originates from a witness’s cell phone or a closed-circuit video camera — frequently suffers from quality issues like low resolution, shake, poor lighting, and noise artifacts that can obscure forensically-valuable information.
“Like on a C.S.I.-type TV show where the bad guy robs a bank and there’s only a low-quality surveillance video of the license plate when he drives away , so the police use our software to reconstruct the letters on the license plate,” says Varah.
Ikena’s capabilities are also being used in the Intelligence Community, and MotionDSP secured investment from In-Q-Tel, a non-profit organisation that invests on behalf of the US Intelligence Community back in 2007. Advances in graphics processing hardware soon allowed MotionDSP to push Ikena’s capabilities further. “We’re now able to process faster than 30 frames a second on even high-definition video sources. The only way we could get to that kind of performance was with the GPU,” says Varah.
Real-time video enhancement
Beyond graphics processing, Ikena’s capabilities also benefit from increased CPU throughput. Varah explains, “We want to find the fastest processing – the most cores at the highest clock speed for the lowest power. That’s what we look for.”
With added compute power from advanced processing and graphics technologies, Ikena has been drawing interest from the US Department of Defense (DoD) as part of its ongoing investment in new technologies for Intelligence, Surveillance, and Reconnaissance (ISR) in Iraq and Afghanistan.
In the modern era, automated video recording is featured on virtually all military vehicles — including manned vessels like fighter jets, trucks, and tanks, as well as unmanned craft such as Predator UAVs (unmanned aerial vehicles). “In 2009 more than 24 years worth of video was recorded in Iraq and Afghanistan,” says Varah. “The amount of video that is being recorded in theatre is absolutely staggering.”
Because of this growing dependence on recorded imagery, the US government has started to write general requirements for real-time image processing to improve the quality of the video they capture. “Analysts need better video so they can see what’s going on – the military calls it ‘better situational awareness’.” says Varah.
“Our differentiation is our ability to deliver real-time super-resolution image processing on full motion video, and that’s thanks to the GPU. Real-time processing is a critical requirement, because without real-time it’s pretty much useless to the analyst who has to make real-time decisions.”
A tough customer
In a situation where the clarity of a live video stream can help save lives, there’s a high bar set in terms of technology standards and requirements. As Varah explains, “The DoD is a very hard customer to satisfy. All of our image processing has to be capable of doing 1080p video resolution, which is huge. 1080p is six times the resolution of standard definition (SD) video. To give you a sense, we estimate that a SD video stream takes 150 to 200 gigaflops of compute to process. A 1080p stream requires something in the order of 1.2 teraflops to process.
“On the highest end AMD graphics processors, we’re able to do 1080p video in real-time, and two streams of 720P video. And we’re not able to do this on any other GPU,”says Varah.
MotionDSP also recently adapted Ikena for OpenCL, an open standard that ensures that the software will be supported by multiple hardware vendors. “The U.S. government customers are not crazy about anything that is sole source. Whenever they do procurement, they’re looking for something where they’ve got competition. And they look for open standards. They have literally had the experience where vendors give them systems that are so vertical, and so closed, that people have died because they can’t get these systems to talk to each other,” says Varah.
“Porting to OpenCL got us an open platform that will be a standard for years to come. But in particular, working closely with AMD to tune our algorithms specifically to take advantage of AMD’s architecture was also highly valuable because we were able to extract a level of performance that we hadn’t gotten from competing hardware. More performance means we can deliver more capabilities to our DoD customers: higher-resolutions, better processing quality, and more simultaneous streams.”
New installations
In addition to processing throughput and graphics horsepower, MotionDSP found that AMD’s interest in energy efficient technologies could help them address challenging new use cases where Ikena could be of benefit.
“There are really two advantages to AMD. One is on the high-end, where AMD is getting us easily 30 to 60% better performance over a competitor’s products. On the low end, AMD lets us do sophisticated image processing with significantly less power — 30 to 50% less. And that’s really important when you want to embed our capability directly on a UAV, which has strict requirements for heat, weight, and power,” says Varah.
Stiletto: a high-speed, floating data center
In June 2011, MotionDSP demonstrated its capability on a DoD experimental boat — an 88-foot, carbon fibre catamaran featuring four 1500HP diesel engines, eight channels of live HD video, and two AMD-powered Colfax CX1250-N4 rackmount servers running Ikena software.
“We installed a mini-cloud of AMD hardware. We had two AMD-powered 1U servers, each with dual AMD Opteron 6100 Series processors and dual ATI FirePro V8800 professional graphics cards. Through those two machines we piped about 8 channels of video, so each of the AMD-based servers was doing real-time processing on up to 4 channels of video at any one time,” says Varah.
“The experiment was really to test how much better we could make the existing military-grade sensors. We took live feeds from the ship’s SeaFlir and GyroCam cameras and processed them in real-time. And what was amazing was we proved we could make them substantially better — an Air Force analyst commented that our software improved the video by up to 2 video NIIRS levels.“
The results of Ikena running on AMD-powered servers in an extremely challenging instance aboard a high-speed, experimental stealth craft impressed the audience. “We had more than 6 different DoD agencies come to our Stiletto demonstration. It was so successful that we now have a number of deployments, or “transitions” as the DoD calls them, happening now – customers who are choosing to integrate our software into a variety of different platforms, from Full Motion Video analyst’s workstations to large enterprise server installations.” says Varah.
“And as a result of our success improving sensors, the Stiletto program has invited us to come back to install our capabilities as a permanent capability.” The Assistant Secretary of Defense for Research & Engineering, Rapid Reaction Technology Office (RRTO) sponsors Stiletto for technology demonstrations.
A technology partnership
With a vast and ever-increasing quantity of video-hours captured-due to the expanding fleet of military vehicles deployed and the multiplying number of cameras mounted per vehicle — there will be plenty of new opportunities and challenges for MotionDSP to tackle going forward.
For example, technology that could perform automated tactical analysis of video imagery would greatly reduce the burden on humans facing insurmountable hours of footage. “The next problem that we’re working on is really doing more to reduce what we call the Operator Workload, to help the DoD do more with fewer people,” says Varah. “This sort of advanced computer vision will require even more compute power than we’re using now.”
There’s also a growing demand for Ikena at home, in the public safety sector. Avwatch, an aerial surveillance provider, supports disaster recovery ground teams with real-time video from above. Recent projects include Hurricane Irene and Montana forest fires.
As Avwatch President Chris Kluckhuhn explains, “We’re using Ikena to stabilize and enhance video. When we’re streaming live broadcasts, all the Ikena improvements to the video are being done in real-time, before they’re distributed live.” In the case of the Montana forest fires, Ikena’s Mosaicking feature was able to instantly stitch together several minutes of video to create a wide area “satellite view” of the forest fires.
“MotionDSP is at the leading edge of using computing technology to take postproduction Enhancements to a real-time environment and give capabilities to first-responders that we’ve never had before,” says Kluckhuhn.
Whether it’s assisting emergency crews with real-time tactical information, helping to prosecute a criminal case in court, or keeping a watchful eye over the troops abroad — MotionDSP is focused on developing new technologies that put safety first, according to Varah.
“It’s very difficult to have leading-edge performance and total stability. You rarely have those two together. AMD does a very good job about being aggressive in terms of rolling out improvements and optimizations, but at the same time providing a driver that’s robust. And that’s really essential.”
