Resources >‏ In the Media >‏ Latest News Details

Latest News Details

Reading a neural network’s mind

Publication Date:
12/12/2017
Category:
In the Media
Neural networks MIT.jpg
Neural networks, which learn to perform computational tasks by analyzing huge sets of training data, have been responsible for the most impressive recent advances in artificial intelligence, including speech-recognition and automatic-translation systems.

During training, however, a neural net continually adjusts its internal settings in ways that even its creators can’t interpret. Much recent work in computer science has focused on clever techniques for determining just how neural nets do what they do.

In several recent papers, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Qatar Computing Research Institute have used a recently developed interpretive technique, which had been applied in other areas, to analyze neural networks trained to do machine translation and speech recognition.

They find empirical support for some common intuitions about how the networks probably work. For example, the systems seem to concentrate on lower-level tasks, such as sound recognition or part-of-speech recognition, before moving on to higher-level tasks, such as transcription or semantic interpretation.

But the researchers also find a surprising omission in the type of data the translation network considers, and they show that correcting that omission improves the network’s performance. The improvement is modest, but it points toward the possibility that analysis of neural networks could help improve the accuracy of artificial intelligence systems.

“In machine translation, historically, there was sort of a pyramid with different layers,” says Jim Glass, a CSAIL senior research scientist who worked on the project with Yonatan Belinkov, an MIT graduate student in electrical engineering and computer science. “At the lowest level there was the word, the surface forms, and the top of the pyramid was some kind of interlingual representation, and you’d have different layers where you were doing syntax, semantics. This was a very abstract notion, but the idea was the higher up you went in the pyramid, the easier it would be to translate to a new language, and then you’d go down again. So part of what Yonatan is doing is trying to figure out what aspects of this notion are being encoded in the network.”

The work on machine translation was presented recently in two papers at the International Joint Conference on Natural Language Processing. On one, Belinkov is first author, and Glass is senior author, and on the other, Belinkov is a co-author. On both, they’re joined by researchers from the Qatar Computing Research Institute (QCRI), including Lluís Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and Stephan Vogel. Belinkov and Glass are sole authors on the paper analyzing speech recognition systems, which Belinkov presented at the Neural Information Processing Symposium last week.

To read more click here.



Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

Economist story pic.JPG

Improving disaster response efforts through data

08/02/2018

Extreme weather events put the most vulnerable communities at high risk. How can data analytics strengthen early warning systems and and support relief efforts for communities in need? The size and ...

Read More

Yazan Wired story pic.jpg

Your sloppy bitcoin drug deals will haunt you for years

28/01/2018

Perhaps you bought some illegal narcotics on the Silk Road half a decade ago, back when that digital black market for every contraband imaginable was still online and bustling. You might already ...

Read More

Luis Luque El Correo.jpg

Entrevista con Luis Fernández Luque, cofundador de Salumedia e investigador del Qatar Computing Research Institute

08/01/2018

Si quiere buscar un ejemplo de ciudadano del mundo, de los que al cabo del año vive y trabaja desde numerosos países, y a través de internet, esté donde esté, desarrolla en remoto actividades para ...

Read More

Events

2018

Eman interns pic 2017.jpg

QCRI Summer Internship Program

Download ICS File 06/05/2018  - 05/07/2018 , Hamad Bin Khalifa Research Complex

Each year, Qatar Computing Research Institute organizes a summer internship program for undergraduate students studying computer science, computer engineering and other disciplines. The internship is unpaid, and QCRI does not provide any visa support.

Read More

Past Events

Regina

Public Talk by Prof. Regina Barzilay "Artificial Intelligence for Oncology: Learning to Cure Cancer from Images and Text"

Download ICS File 27/03/2018 ,

Artificial Intelligence for Oncology: Learning to Cure Cancer from Images and Text A talk by Professor Regina Barzilay, MIT CSAIL Winner of 2017 MacArthur ‘genius grant’ At Education City Student ...

Read More

Slide1.JPG

QCRI & MIT-CSAIL Annual Project Review 2018

Download ICS File 27/03/2018 ,

Executive Overview Sessions Open to public Date:    Tuesday, March 27, 2018 Time:    9:00AM – 3:00PM Venue:  HBKU Research Complex Multipurpose Room To view full agenda, please click here . To RSVP, ...

Read More

News

Issa Khalil cyber.jpg

QCRI and Turkish scientists to build pre-emptive cyber security platform

10/04/2018

Project to build defensive platform to detect emerging cyberattacks awarded $1.65m grant.

Read More

Daniela Rus CSAIL (2).JPG

MIT-CSAIL researchers visit Doha for annual QCRI research meeting

25/03/2018

Meeting updates joint research projects between the two institutions and will feature 'genius grant' recipient Prof. Regina Barzilay, who uses AI to detect cancer, as keynote speaker.

Read More

Arcpic1.jpg

QCRI’s Advanced Transcription System snares ARC’18 Best Innovation Award

19/03/2018

Her Highness Sheikha Moza bint Nasser presents accolade for system that automatically converts speech to text using state-of-the-art speech recognition techniques.

Read More