Biologically Plausible Neural Networks

A primary goal of computational neuroscience is to simulate [parts of] the brain to better understand how the brain functions, infers, learns, remembers, etc. At very low levels of abstraction, the brain can be modeled on a per neuron basis, with neural dynamics encoded as a complex set of differential equations. Naturally, these simulations scale poorly with neuron population size, and so we must make certain assumptions that allow us to more efficiently approximate neural activity. On this other end of the spectrum are Artificial Neural Networks (ANNs) which are extremely efficient, scale well with the number of learnable parameters, and are trainable using gradient descent and backpropagation, but make many assumptions that reduce their efficacy as models of the brain. Both simulations can be used to study aspects of neural activity but fail to capture the whole picture.

In an effort to merge these two schools of thought, my recent work at the MIT McGovern Institute has been the implementation of a biologically plausible neural network package that allows researchers to easily simulate biologically faithful models while maintaining the ability to train them using traditional ML approaches and scale them to useful numbers of parameters. This package, tentatively dubbed BioPlNN, attempts to bridge this gap, starting from the ANN side of the problem. In essence, this means incorporating biological plausibility, one feature at a time, into existing ANN software, namely PyTorch. In practice, this means leveraging tools like trainable sparse matrix multiplications to implement sparsely topographically connected populations of neurons representing a cortical sheet. It also involves restricting ANNs such that their behavior mimics that of the brain (e.g. having purely excitatory/inhibitory neurons, constraining feedforward and feedback connectivity in RNN architectures, adding learnable time constants analogous to the capacitive time constants of neurons). This endeavor is still a work in progress, and you can find more frequent updates on the project's GitHub page.