The stuff of science fiction is coming to life in the work of computer scientists studying human gait patterns.
They are working on the hypothesis that each person has a unique gait so that one day our so-called signature motion will be as valuable as a fingerprint in charting identity.
Facial-recognition systems are in use and are comparatively well-advanced. Extending the idea to include the whole body, though, raises a host of problems that the mathematical formulas behind the relevant software have yet to solve.
The challenges are immense because they involve capturing subtle changes in a person’s walk that can be compromised by shadows, different walking surfaces and what a person carries. However, if successful, the systems would be valuable in security surveillance as well as important diagnostic tools for physicians who deal with movement disorders.
“At the time we started doing this work, homeland security wasn’t an issue; the project only really started as a project for computer animation and for medical applications,” says Alex Vasilescu, a research scientist at New York University and a computer science doctoral candidate at the University of Toronto.
Her work on facial recognition — known in tech speak as “TensorFaces: Multilinear Tensor Decomposition of Image Ensembles” — led Massachusetts Institute of Technology’s Technology Review last year to name her as one of the top 100 young scientists in the country.
An NYU Web site explains that the purpose of facial-recognition software is “to enhance computers’ ability to match multiple characteristics of a face in ways that overcome vagaries of shading, angle or expression.” Gait patterns, formally called “human motion signatures” by researchers, are based on being able similarly to extract and analyze characteristics of movement patterns.
“I wanted to extract an individual’s personal style and the explicit manner in which they move in a way that translates across different motions and is consistent,” she explains in a telephone interview. Her initial interest grew out of a common observation that friends often recognize one another at a distance by shape or movement alone before it’s possible to see the face. She wondered, she says, if she could teach a computer to perform in the same way.
“At the time, I was thinking of all these actors who have unique ways of moving, and I wondered how to translate that into a signature gait or style. You can capture this and sit there and play back the motion. If you take that particular motion, you will ask, ‘What is the signature here, and how does that change when he does anything else, like running up- or downhill — a signature motion that would reveal other traits?’”
Next, Ms. Vasilescu says, the researcher has to find how close a signature motion is to normal so that a person who has gone for physical therapy, for example, can have his progress charted using this system.
“People usually say that they feel better after therapy, but that is a fuzzy word. Using this, you can see improvement and a way of quantifying different treatment methods,” she says.
She knew she easily could recognize someone familiar, but what happens, she wondered, if you can’t see the person’s face in the dark and he or she is not looking into a camera? “What is another modality to use based on the way they move?”
Some work along these lines had been explored by psychologists as far back as the 1960s, she says. The magazine Scientific American, she recalls, had written about how reflective markers placed on different joints allowed an observer to trace the way dots were moving through a dark space. That prompted the question of how a person could deduce what was happening in that particular scene.
David Herrington of the Technical Support Working Group, an interagency organization that funds anti-terrorism research through the Department of Defense, praises Ms. Vasilescu’s work on solving basic mathematical problems involved in such research but calls the technology immature. He says it might take 10 years for human gait studies to prove themselves.
“Ten years in this business is an eternity,” notes Larry Davis, chairman of the department of computer science at the University of Maryland, which has received research money in the past from the Department of Defense’s Advanced Research Projects Agency (DARPA) for the Human Identification at a Distance project. The study involved five universities around the country interested in developing computer algorithms that could identify people by how they walk.
Progress was measured by how well a program could compare a person’s gait against all the gaits in its database.
The project ended last fall because it was in the same office as the Total Information Awareness project, which was shut down, he says. “It would have ended anyway,” he adds. “There wasn’t really enough work done to come to any firm conclusion. It wasn’t a technology ready for incorporation into any surveillance [method].”
When gait research is perfected, its uses for anti-terrorism surveillance will be invaluable, says a former program manager at DARPA who asked that his name not be used. He mentions the possibility of having software that would detect whether a stranger walking into a facility is a frightened woman or a terrorist with hidden explosives.
Entrepreneurs who have developed computer software sophisticated enough to separate objects of interest from the background for security surveillance purposes — a step along the path to individual-recognition systems — include ObjectVideo of Reston. The company’s VEW, for video early warning, software can be programmed specifically to send an alert only after determining whether a movement or object constitutes a danger, according to Alan Lipton, VideoObject’s chief technology officer.
It, too, can be traced back to DARPA-funded work on computer vision technology for defense purposes. A customer can set the rules verbally by giving the system what Mr. Lipton calls policy statements on what to watch out for. It isn’t motion or gait detection per se, but a way of detecting when people are doing unusual things. When instructed, the system can send an alarm, for instance, about a bag left unattended for a long time.