Being a Better Model

I enjoy thinking about how to apply insights from our understanding of biological intelligence to improve machine intelligence. However, the insights often flow in the opposite direction as well: machine learning often recasts findings from neuroscience and psychology in a more grounded and principled way. The following is a perspective on what machine learning can teach us about ourselves.

No matter what you’re trying to do with machine learning, to perform any task, you need a model. Machine learning researchers invent new models and tweaks to existing models, each promising to solve new tasks, use less data, or be easier to train. Many new ideas have come out of this AI spring that we find ourselves in. As a result, we’re gaining a better understanding of the process of learning more generally. However, many of these ideas are applicable to biological learning as well; after all, most biological systems are also learning system.

There are two complementary ways to construct a model that is capable of performing some task: either you have strong prior assumptions on the model or you have a lot of data from which to learn the model. Both directions are essential. Without priors, i.e. knowing what modeling choices are reasonable, the space of all possible models is too expansive to possibly enumerate. And without data, we cannot adapt and shift our model away from our (often imperfect) priors. In this light, a model is only as good as its priors and the data that it’s trained on. Machine learning practitioners are often concerned with finding a reasonable set of priors for the amount of data that is available. In the ideal situation, one would only have weak priors on the model and copious amounts of data.

This general idea, highlighting the importance of data and priors for a system capable of performing some task, also applies to biological systems. We are all very complex models, derived from our environments. Much of what makes us highly capable models; our specialized organs for sensing, moving, maintaining our bodies, and importantly, the architecture of our nervous systems; is encoded in our DNA. The process of evolution has stumbled upon these priors over millennia, and due to their success, have been passed down through natural selection. In addition to these specializations, which are hardcoded, many animals, with humans as the star example, have an enormous capacity for learning. We are constantly synthesizing information from our senses, allowing us to learn how to walk, talk, and so many other amazing skills. This builds in a certain amount of flexibility, allowing us to pick up difficult skills and adapt to an uncertain environment, things which evolution would struggle to directly encode. We are each a combination of nature, from our biological priors, and nurture, from the data we encounter in our lifetimes.

Considering that your biological priors have been learned through evolution, it becomes clear that you are a direct result of the data in the environment encountered by you and your line of ancestors. Or conversely, without that environmental data, we wouldn’t be us. Your biological priors enter in the form of physical characteristics, but thankfully, it seems that much of what makes you a unique individual is learned during your lifetime. Nobody is born as a star athlete, musician, or genius. We might be biologically predisposed to excel at certain tasks, but this only represents a potential. Becoming proficient at a task, just becoming anything, requires learning on your part. And to learn, one must have the requisite experiences, access to the environmental data.

I prefer to look at this optimistically: you can be whomever you want to be, it only requires exposing yourself to the right experiences from which to learn. Unlike machine learning models, we largely have control over the data that we encounter and learn from. In machine learning parlance, this is referred to as active learning. If you want to learn a new skill, change some aspect of yourself, or achieve some personal goal, it largely comes down to seeking out those experiences and putting in the effort to learn from them through repeated exposure. Admittedly, this is often easier said than done. Stepping outside of one’s comfort zone is difficult, but the consequences are often highly rewarding in terms of personal growth.

To conclude, your brain is an incredible learning system, the envy of every machine learning researcher. But without the necessary data, i.e. experiences, that amazing capacity for learning will go squandered. If you feed your brain the same patterns day after day, you’ll never be challenged to grow as a person. So break out of your established routines! Feed your brain the data that it craves! Go explore the world! Go strike up a conversation with a stranger! Try something new! Cooking, art, a sport, volunteering, anything! Go explore the world’s data distribution; your brain will thank you.