Very excited to share that I was awarded the Takeda Fellowship on research at the intersection of AI and Health!
My development setup
Since this has been helpful to many colleagues, I thought I would put together my development setup and especially include a small how-to on setting up PyCharm with remote support.
PyCharm remote setup
I have found the remote support of PyCharm to be extremely helpful. It is only available in the professional version, but you get it for free if you’re a student.
The pros are extremely appealing: You can run everything on a remote server while the development experience feels as if it were local. PyCharm takes care of uploading changes, running the remote interpreter (even in debug mode), and displaying figures directly in the IDE.
Follow these steps to setup PyCharm in the same way:
- open File > Settings, navigate to Project > Python interpreter
- On the right side of the “Python Interpreter” line, click the gear icon and click “Add…”
- select “SSH Interpreter” and create a new server configuration. Enter the host and your username, click next
- Enter your password or private key, click Next
- Enter the path to the remote interpreter. I use a local miniconda installation, so for me the path is something like /braintree/home/user/miniconda3/bin/python.
- On the same screen, click the Folder icon to the right of “Sync folders” and input where you want your local files synced. Click Finish
That’s it! You can now program as usual in your IDE and files will automatically be transferred to the remote host. When you run/debug things, they will automatically run on the remote host. (In case it doesn’t sync, make sure your remote path is entered in Settings > Deployment > Mappings.)
Advanced tips
- Running fig.show() with a matplotlib figure will show it directly in the IDE
- For multiple attached projects, you can go to Settings > Build, Execution, Deployment > Deployment > (in view) Mappings to setup the local/remote path pairs
- Make sure to use the integrated Version Control tools. If they don’t show up automatically, check that the directory is recognized under version control in Settings > Version Control
- Use auto-formatting! PyCharm can automatically reformat your code according to a specific (or default) code style
- Use hotkeys! They make it much quicker to accept suggestions, auto-format, run, step into etc. (I use the Eclipse keymap out of habit)
- To install miniconda locally, I usually run wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && bash Miniconda3-latest-Linux-x86_64.sh.
Hardware:
- SurfaceBook laptop on a stand
- 2 external 24″ monitors
- Kinesis Freestyle2 split keyboard
- wireless mouse
- external webcam
- wireless headset
- SurfaceDock to bring it all together with minimal daily cable management
Software:
- PyCharm IDE
- KiTTY ssh client
- Keep for todo lists
- OneNote for note-keeping
- PowerPoint for presentations / running plots
- KeePass password manager
- Dropbox for all documents and presentations (except code, but including the password database so that I can access it on my phone through KeePass2Android)
- Github for anything code-related
Oral presentation at NeurIPS
The work led jointly with Jonas Kubilius on “Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs” was accepted to NeurIPS as an oral! Only 36 (or 0.5%) out of the 6743 submissions were selected as an oral, so we’re very excited to present our ideas how Machine Learning and Neuroscience can interact again in the form of models of the brain.
The McGovern institute and Robin.ly have also reported about our findings.
Efficient Neural Stimulation with In-Silico Stimulation, class project
For my class project in Alan Jasanoff’s course Neurotechnology in Action, I experimented with modeling the behavioral effects of microstimulation. I used Afraz et al., Nature 2006 as a reference and started from our recently developed CORnet model (Kubilius*, Schrimpf* et al. NeurIPS 2019). By increasing the activity at parts in the model, I was able to reproduce the behavioral effects reported by Afraz et al. to a first extent. On top of that, I used the model to predict which improvements in neurotechnology hardware would best support behavioral control. I think modeling the behavioral effects of neural perturbations is an exciting direction and I’m hoping to thereby bring the models closer to clinical applications. For more details, please see the paper.
Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?
The field of Machine Learning is doing pretty well at quantifying its goals and progress, yet Neuroscience is lagging behind in that regard — current claims are often qualitative and not rigorously compared with other models across a wider spectrum of tasks.
Brain-Score is our attempt to speed up progress in Neuroscience by providing a platform where models and data can compete against each other: https://www.biorxiv.org/content/early/2018/09/05/407007
Deep neural networks trained on ImageNet classification do the best on our current set of benchmarks and there is a lot of criticism about the mis-alignment between these networks and the primate ventral stream: mapping between the many layers and brain regions is unclear, the models are too large and are just static feed-forward processors.
We thus created a more brain-like model, “CORnet”, which does well on Brain-Score with only four areas and recurrent processing: https://www.biorxiv.org/content/early/2018/09/04/408385
EDIT: Science Magazine wrote a news piece about the use of deep neural networks as models of the brain with the final paragraphs devoted to Brain-Score: http://sciencemag.org/news/2018/09/smarter-ais-could-help-us-understand-how-our-brains-interpret-world
Recurrent computations for the recognition of occluded objects (humans + deep nets)
Finally out (in PNAS)! Our paper on recurrent computations for the recognition of occluded objects, in humans as well as models. Feed-forward alone doesn’t seem to cut it, but attractor dynamics help; similarly the brain requires recurrent processing to untangle highly occluded images.
http://www.pnas.org/content/early/2018/08/07/1719397115/
We have some pretty visualization gifs in the github, along with the code: https://github.com/kreimanlab/occlusion-classification
EDIT: MIT News covered our work, along with a video of us giving the intuition behind it: http://news.mit.edu/2018/mit-martin-schrimpf-advancing-machine-ability-recognize-partially-seen-objects-0920
Finalist at the Google.org Impact Challenge Germany
Searching for non-intuitive architectures
Summer Internship work is out in ICLR! Automatic architecture search finds non-intuitive (at least to me) architecture including sine curves and division.
I’m really glad to have worked with a fantastic team at Salesforce Research, most closely with Stephen Merity and Richard Socher.
Blog: https://einstein.ai/research/domain-specific-language-for-automated-rnn-architecture-search
Paper + Reviews: https://openreview.net/forum?id=SkOb1Fl0Z
Master’s Thesis: Brain-inspired Recurrent Neural Algorithms for Advanced Object Recognition
It’s done! I finished my Master’s Thesis which focused on the idea and implementation of recurrent neural networks in computer vision, inspired by findings in neuroscience. The two main applications of this technique shown here are the recognition of partially occluded objects and the integration of context cues.
Here’s the link: Brain-inspired Recurrent Neural Algorithms for Advanced Object Recognition – Martin Schrimpf
On the robustness of neural networks
There is a new project we are beginning to look into which analyzes today’s neural networks in terms of stability and plasticity.
More explicitly, we evaluate how well these networks can cope with changes to their weights and how well they can adapt to new information. Some preliminary results suggest that if weights in lower layers are perturbed, this has a more severe effect on performance than if higher layers are perturbed. This has a nice correlation to neuroscience where it is assumed that our hierarchically lower cortical layers in the visual cortex remain rather fixed over the years.
Update: we just uploaded a version to arXiv (https://arxiv.org/abs/1703.08245) which is currently under review at ICML.
NIPS Brains&Bits Poster
Just presented our work on Recurrent Computations for Pattern Completion at the NIPS 2016 Brains & Bits Workshop!
Here’s the poster that I presented.
It was an awesome conference, lots of new work and amazing individuals.
Here’s a really short summary, but I highly recommend going through the papers and talks:
- unsupervised learning and GANs are hot
- learning to learn is becoming hot
- new threshold for deep: 1202 layers
TensorFlow seminar paper on arXiv
After some requests, I have uploaded my (really short) analysis of Google’s TensorFlow to arXiv: https://arxiv.org/abs/1611.08903.
It is really just a small seminar paper, the main finding is that while using any Machine Learning framework is generally a good idea, TensorFlow has a really good chance of sticking around due to its already widespread usage within Google and research coupled with a growing community.
Hello
I am a tenure-track assistant professor at the EPFL Neuro-X institute where I head the NeuroAI Lab; with appointments at the School of Life Sciences, and the School of Computer and Communication Sciences.
My research focuses on a computational understanding of the neural mechanisms underlying natural intelligence in vision and language. To achieve this goal, I bridge Deep Learning, Neuroscience, and Cognitive Science, building artificial neural network models that match the brain’s neural representations in their internal processing and are aligned to human behavior in their outputs.
I completed my PhD at the MIT Brain and Cognitive Sciences department with Jim DiCarlo, following Bachelor’s and Master’s degrees in computer science at TUM, LMU, and UNA. Previous work includes research in human-like vision at Harvard, natural language processing + reinforcement learning at Salesforce, as well as other projects in industry. I love translating discoveries into applications and have founded — and worked in — several startups. My work has been recognized in the news at Science magazine, MIT News, and Scientific American.
Integration Prize
We just won the Integration Prize of the Government of Swabia with Integreat!
Scalable Database Concurrency Control using Transactional Memory
Although it’s been a while, I thought I’d upload my Bachelor’s Thesis for others to read: Scalable Database Concurrency Control using Transactional Memory.pdf.
The work consists of two parts:
Part 1 analyzes the constraints of Hardware Transactional Memory (HTM) and identifies data structures that profit most of this technique.
Part 2 attempts different implementations of HTM in MySQL’s InnoDB storage component and evaluates the results.