My main interest is in bridging Machine Learning and Neuroscience. I am focusing on building deep neural network models of the brain’s ventral stream that are more human-like in their behavior as well as their internals.
Previous work includes research in computer vision at Harvard, and natural language processing and reinforcement learning at Salesforce. My educational background is largely in computer science. I am currently a PhD/Graduate student at MIT BCS (Brain and Cognitive Sciences) with Jim DiCarlo and collaborate with Josh Tenenbaum and Gabriel Kreiman at Harvard.

Oral presentation at NeurIPS

The work lead by Jonas Kubilius and me on “Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs” was accepted to NeurIPS as an oral! Only 36 (or 0.5%) out of the 6743 submissions were selected as an oral, so we’re very excited to present our ideas how Machine Learning and Neuroscience can interact again in the form of models of the brain.

The McGovern institute and Robin.ly have also reported about our findings.

Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?

The field of Machine Learning is doing pretty well at quantifying its goals and progress, yet Neuroscience is lagging behind in that regard — current claims are often qualitative and not rigorously compared with other models across a wider spectrum of tasks.

Brain-Score is our attempt to speed up progress in Neuroscience by providing a platform where models and data can compete against each other: https://www.biorxiv.org/content/early/2018/09/05/407007

Deep neural networks trained on ImageNet classification do the best on our current set of benchmarks and there is a lot of criticism about the mis-alignment between these networks and the primate ventral stream: mapping between the many layers and brain regions is unclear, the models are too large and are just static feed-forward processors.
We thus created a more brain-like model, “CORnet”, which does well on Brain-Score with only four areas and recurrent processing: https://www.biorxiv.org/content/early/2018/09/04/408385

EDIT: Science Magazine wrote a news piece about the use of deep neural networks as models of the brain with the final paragraphs devoted to Brain-Score: http://sciencemag.org/news/2018/09/smarter-ais-could-help-us-understand-how-our-brains-interpret-world

Recurrent computations for the recognition of occluded objects (humans + deep nets)

Finally out (in PNAS)! Our paper on recurrent computations for the recognition of occluded objects, in humans as well as models. Feed-forward alone doesn’t seem to cut it, but attractor dynamics help; similarly the brain requires recurrent processing to untangle highly occluded images.


We have some pretty visualization gifs in the github, along with the code: https://github.com/kreimanlab/occlusion-classification


EDIT: MIT News covered our work, along with a video of us giving the intuition behind it: http://news.mit.edu/2018/mit-martin-schrimpf-advancing-machine-ability-recognize-partially-seen-objects-0920

Searching for non-intuitive architectures

Summer Internship work is out in ICLR! Automatic architecture search finds non-intuitive (at least to me) architecture including sine curves and division.

I’m really glad to have worked with a fantastic team at Salesforce Research, most closely with Stephen Merity and Richard Socher.

Blog: https://einstein.ai/research/domain-specific-language-for-automated-rnn-architecture-search

Paper + Reviews: https://openreview.net/forum?id=SkOb1Fl0Z

Master’s Thesis: Brain-inspired Recurrent Neural Algorithms for Advanced Object Recognition

It’s done! I finished my Master’s Thesis which focused on the idea and implementation of recurrent neural networks in computer vision, inspired by findings in neuroscience. The two main applications of this technique shown here are the recognition of partially occluded objects and the integration of context cues.

Here’s the link: Brain-inspired Recurrent Neural Algorithms for Advanced Object Recognition – Martin Schrimpf

On the robustness of neural networks

There is a new project we are beginning to look into which analyzes today’s neural networks in terms of stability and plasticity.
More explicitly, we evaluate how well these networks can cope with changes to their weights and how well they can adapt to new information. Some preliminary results suggest that if weights in lower layers are perturbed, this has a more severe effect on performance than if higher layers are perturbed. This has a nice correlation to neuroscience where it is assumed that our hierarchically lower cortical layers in the visual cortex remain rather fixed over the years.

Update: we just uploaded a version to arXiv (https://arxiv.org/abs/1703.08245) which is currently under review at ICML.

NIPS Brains&Bits Poster

Just presented our work on Recurrent Computations for Pattern Completion at the NIPS 2016 Brains & Bits Workshop!

Here’s the poster that I presented.

It was an awesome conference, lots of new work and amazing individuals.
Here’s a really short summary, but I highly recommend going through the papers and talks:

  • unsupervised learning and GANs are hot
  • learning to learn is becoming hot
  • new threshold for deep: 1202 layers

TensorFlow seminar paper on arXiv

After some requests, I have uploaded my (really short) analysis of Google’s TensorFlow to arXiv: https://arxiv.org/abs/1611.08903.

It is really just a small seminar paper, the main finding is that while using any Machine Learning framework is generally a good idea, TensorFlow has a really good chance of sticking around due to its already widespread usage within Google and research coupled with a growing community.

Page 1 Page 2