Can a Computer Teach Itself Common Sense?

[Pedro's Klinefelter]

In the not too distant past, suggesting that a computer could demonstrate “common sense” would be considered an oxymoron. But supercomputers such as IBM’s Watson may portend new roles for computers.

 

Currently, researchers at Carnegie Mellon University are running a computer program 24/7 in an effort to capture data and teach itself common sense on a massive scale.

 

The program is called the Never Ending Image Learner (NEIL) as the software searches the Web for images, doing its best to understand them on its own and, as it builds a growing visual database, gathering common sense on a massive scale.

 

 

NEIL leverages recent advances in computer vision that enable computer programs to identify and label objects in images, to characterize scenes and to recognize attributes, such as colors, lighting and materials, all with a minimum of human supervision.

 

In turn, the data it generates will further enhance the ability of computers to understand the visual world.

NEIL has significant advances over earlier robotic devices as it can makes associations between things to obtain common sense information. Information that people know almost intuitively — that cars often are found on roads, that buildings tend to be vertical and that ducks look sort of like geese.

 

Based on text references, it might seem that the color associated with sheep is black, but people — and now NEIL — nevertheless know that sheep typically are white.

“Images are the best way to learn visual properties,” said Abhinav Gupta, Ph.D., assistant research professor in Carnegie Mellon’s Robotics Institute.

“Images also include a lot of common sense information about the world. People learn this by themselves and, with NEIL, we hope that computers will do so as well.”

A computer cluster has been running the NEIL program since late July and already has analyzed three million images, identifying 1,500 types of objects in half a million images and 1,200 types of scenes in hundreds of thousands of images.

It has connected the dots to learn 2,500 associations from thousands of instances.

One motivation for the NEIL project is to create the world’s largest visual structured knowledge base, where objects, scenes, actions, attributes and contextual relationships are labeled and cataloged.

What we have learned in the last 5-10 years of computer vision research is that the more data you have, the better computer vision becomes,” Gupta said.

 

Some projects, such as ImageNet and Visipedia, have tried to compile this structured data with human assistance.

 

But the scale of the Internet is so vast — Facebook alone holds more than 200 billion images — that the only hope to analyze it all is to teach computers to do it largely by themselves.

 

People also tell NEIL what categories of objects, scenes, etc., to search and analyze. But sometimes, what NEIL finds can surprise even the researchers.

 

It can be anticipated, for instance, that a search for “apple” might return images of fruit as well as laptop computers. But Gupta and his team, landlubbers all, had no idea that a search for F-18 would identify not only images of a fighter jet, but also F18-class catamarans.

 

As its search proceeds, NEIL develops subcategories of objects – tricycles can be for kids, for adults and can be motorized, or cars come in a variety of brands and models.

 

And it begins to notice associations – that zebras tend to be found in savannahs, for instance, and that stock trading floors are typically crowded.

 

Before NEIL can become a household name, scale will need to be reduced as NEIL is computationally intensive, with the program running on two clusters of computers that include 200 processing cores.