Artificial intelligence now a reality as intuition algorithm beats humans in data test

Computers could soon replace human intuition in many areas - and a new system developed at MIT has shown it can outperform even the smar...

Computers could soon replace human intuition in many areas - and a new system developed at MIT has shown it can outperform even the smartest of people. 

MIT researchers have designed a big-data analysis system which aims to replace human intuition in the search for buried patterns.

The system, called the Data Science Machine, competed against human teams in three data science competitions, and outperformed them.

In the three competitions, the Data Science Machine made predictions that were 94, 96, and 87 percent as accurate as the winning submissions, performing better than 615 of the 906 participating teams.

'We view the Data Science Machine as a natural complement to human intelligence,' Max Kanter, whose master's thesis provides the basis for the Data Science Machine, told MIT News.

'There's so much data out there to be analyzed. 

'And right now it's just sitting there not doing anything,' he said. 

'So maybe we can come up with a solution that will at least get us started on it, at least get us moving.' 

The Data Science Machine completed its prediction algorithms at 'inhuman' speed, taking between two and 12 hours for each submission. 

Human teams worked on their algorithms for months.

Kanter works alongside Kalyan Veeramachaneni, his thesis advisor and a research scientist at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL).

While human intuition is often required in choosing the particular features of data to be analyzed, the two researchers have developed ways for the machine to make the determination.

'What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering,' Veeramachaneni said to MIT News.

The Data Science Machine uses numerical indicators to track correlations in structural relationships that are built within database designs. 

As these indicators stack up across the database, the Data Science Machine is able to apply operations and find averages within the numbers.

Along with this process, the machine looks for categorical data, which comes across in a limited range of values, like days of the week.

Through his work with the Anyscale Leaning for All group at CSAIL, Veeramachaneni and other researchers apply machine-learning techniques to practical problems, like predicting which students may drop out of an online course. 

Manufacturing the feature, he says, is critical.

'The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas.' 

'The Data Science Machine is one of those unbelievable projects where applying cutting-edge research to solve practical problems opens an entirely new way of looking at the problem,' said Margo Seltzer, a professor of computer science at Harvard University who was not involved in the work.

'I think what they've done is going to become the standard quickly — very quickly.'