AI for Biology Could Eclipse LLMs: Thoughts on What Stands in the Way

The impact of AI for Biology could be enormous but there are unique challenges to biological data that will require distinct breakthroughs instead of copy and pasting success with LLMs

Current state of AI for Biology

If successful the opportunity for bio AI is enormous. I would argue the impact could be larger than the breakthrough of LLM chatbots: from understanding life’s origins to curing disease, and prolonging human life. The rise of Large Language Models (LLMs) and advancements in image generators has spurred interest in this crossroads of AI and biology. After all, DNA is just another language with only four letters, right? But history reminds us, through the evolution of LLMs, that such breakthroughs demand persistence, ample resources, and a comprehensive grasp of the inherent challenges. Efforts that have both the diversity of expertise required and the long-term perspective to chip away at the difficult problems are few and far between. Its also been proposed that biologic challenges will be fixed by LLM derived “virtual biologists” (Machines of Loving Grace). While an exciting prospect, my personal view is this is a ways off still and there are more immediate impacts that AI can make on biology.

Recent articles have highlighted the “failures” in AI for biology
so I don’t aim to pile on to that topic, but a missing thread is specifically identifying these challenges so we can be more successful with future campaigns. In this post, I’ll highlight three of the primary challenges I see that stand out for making models that can further our understanding of biology compared to LLMs and potential ways to navigate around them.

I’m extremely optimistic about what we can do with AI in biology but believe we’ll only achieve that by being pragmatic about the work that lies ahead.

Notable Examples

A few notable examples of AI for biology are worth highlighting and will be referenced in the sections below:

David Baker (RoseTTAFold), Demis Hassabis and John Jumper (Alphafold) recently were granted the Nobel Prize for the work in protein structure prediction

Alphafold 3 – The most well known and arguably the most successful application of AI for a biological problem to date. AlphaFold employs deep learning techniques to predict protein structures by analyzing amino acid sequences and multiple sequence alignments (MSAs). It utilizes attention-based neural networks to model spatial relationships within proteins, iteratively refining its predictions to achieve high accuracy.

RoseTTA Fold – David Baker and the Rosetta team have been at the problem of protein structure prediction for decades. Early versions of Rosetta were relatively simple and based off free energy of the structures. RoseTTA Fold leverages deep learning to predict protein structures but uses a three-track neural network architecture, where information flows between sequence, distance, and coordinate predictions simultaneously.

Profluent – Company that develops and utilizes proseLM which integrates transformer-based protein language models with structural and functional contexts to enhance protein sequence design. The main application to date has been to develop variants of existing CRISPR systems that are not in the original training set.

Evo – Utilizes a transformer-based architectures called StripedHyena which is tailored for long biological sequences and combines attention mechanisms with convolutional operations. Evo applies it to a significantly larger data set. The application highlighted in the original preprint focuses on generating new CRISPR variants including the RNA.

EVOLVEpro – A straighforward but powerful (and often underappreciated) application of protein models. EVOLVEpro is a “few-shot active learning framework” designed to enhance protein activity by integrating protein language models (PLMs) with activity predictors. This allows protein engineering campaigns to be run much more efficiently and notably uses new data with a has a clear objective.

Note: Existing LLMs like ChatGPT also have the potential for substantial impact on the way we do science and treat patients but here I’m just focusing on new models to better understand biology. Also if looking for a more in depth review of strategies there are many good reviews that have been written.

Challenge 1: A poorly defined objective without rapid, verifiable feedback

The hype surrounding AI and biology has made it all too common to first think of the solution (AI) before considering the problem it’s solving. Frequently AI in biology tries to solve a large, complex problem like “make better drugs” rather than taking on a more basic problem that underlies this goal.

While the digital realm of AI, like that of LLMs, has clear objectives that can be rapidly verified through reinforcement learning from human feedback (RLHF), the goal of biological models is usually more ambiguous. For instance, training AI to develop drugs based solely on the few thousand approved ones is akin to solving a puzzle with most of the pieces missing. The multifaceted reasons drugs fail, many of which are beyond the actual candidate molecule, further complicate the situation.

If “better drugs” is your feedback, it may take a decade to run one cycle making it nearly impossible to build a reasonable model on human time scales. This is apparent from the recent clinical data of some of the first “AI-designed drugs” in the clinic years after the program initiation. I point this out not as a judgment on AI for biology but just to highlight the slow feedback loop. Its also important to highlight that the rate of developing new drugs is usually limited by clinical development and many companies applying AI are tackling problems earlier in the process which has made many skeptical that this will actually yield more therapeutics.

In essence, without a clear understanding of the problem at hand, even the most sophisticated AI tools risk chasing phantoms, rendering the efforts futile.

Challenge 2: Data quality. Heterogeneity, inability to link high-dimensional data, and missing metadata

At the forefront of data collection in biology is Next-Generation Sequencing (NGS), which can catalog virtually everything we know about a cell or organism’s genome and how it is expressed. While NGS has brought about a revolution in the sheer scale and depth of data acquisition, it presents challenges tied to data quality that AI has to grapple with.

One of the most salient challenges is the heterogeneous nature of biological samples. Often models are trained by pooling data together from samples all around the world that might cover decades, introducing additional variables. Add to this the variability from different sequencing methods and sample preparations, and consistency becomes elusive. In some AI applications, more data can mitigate quality issues but in biology, factors like the uniqueness of samples and sequencing costs cap data volume, amplifying the emphasis on quality.

With NGS, we delve into high-dimensional data but where each dimension is measured under different conditions. We might have genomic samples from one study, RNAseq from another, and methylation data from a third – with no clear roadmap for integration. Moreover, as biology unfolds, there are ever-emerging dimensions we’ve yet to identify or measure.

In addition, often, the metadata associated with the samples lacks the critical information needed to train the models, such as environmental factors and specific patient histories. When that metadata is available, patient privacy laws such as HIPAA often constrain how the data can be used. These incomplete and heterogeneous datasets make training large-scale models difficult without additional curation.

The AlphaFold Strategy: Have 10,000s of PhD scientists painstakingly curate your training data

AlphaFold (arguably the most successful application of AI to biology to date) progress in predicting protein 3D structures is deeply rooted in the quality of its data source, the Protein Data Bank (PDB). It is often underappreciated how unique it is to have a biology dataset of this quality and the effort that it’s taken to produce it. For decades, PDB entries have been diligently curated by PhD-level scientists who scrutinize every atom. This attention to detail has ensured accuracy that’s unparalleled in other biological data sets.

The combination of PDB’s meticulous curation and AlphaFold’s advanced techniques demonstrates the transformative power of quality data in tandem with sophisticated AI.

Challenge 3: Data quantity? Biological data is incomplete and scale is not a near-term solution

LLMs have prospered in environments rich with easily accessible and (largely) free data. To the section above on data quality, internet text data is inherently of high quality for training language models (if text is on the web it likely is communicating something understandable by a human). I previously assumed that we just have a lot less data in biology compared to the internet, which is often cited as the key bottleneck for AI for biology. That said back of the envelope estimates don’t suggest that its solely a data quantity issue for biological models. If we estimate that web text is around a 1 exabyte (1000 petabytes) of data this is not too far off from the NCBI SRA which on its own is 50 petabytes. However, biological data is much less information rich when taking into account the above two challenges (a specific query and data quality). For one thing genomic data is highly redundant, even more so than web text (human genomes are ~99.9% identical). More importantly, in biology we are often looking for a rare event (like a new disease causing mutation or a molecule that binds a target particularly well) and as a result models that are effectively trying to compress information are not the answer for all applications.

Biology is arguably dramatically more complex than language and so our measurements will always be incomplete (apart from a perfect cell simulation). In this tight data scenario, I would propose that the solution isn’t simply to scale data (eg sequence everyone on earth) but about maximizing the utility of every single data piece. It stresses the need to be innovative and efficient, extracting more insights from less (Springtail AI is an example of a group that is thinking along these lines). Using our existing understanding of biology, physics, and chemistry can supplement this data shortage.

As an example, Alphafold has benefited from protein structure prediction being fundamentally a physics problem more than a biological one. Protein structure prediction benefits in its capability to reconcile biological heterogeneities with physical constraints. This approach amplifies the worth of each data point, allowing for insightful and accurate predictions, even when the sheer data quantity might seem limiting.

TL;DR Components for successfully building biological models

Its an exciting time for biology and I truly believe that AI will have a huge impact on biology in the coming years and decades. That said I don’t think this will come by simply riding on the coattails of LLMs and is going to require fundamental research into architectures and data to support these efforts. To summarize the points above, here are some potential components to building a successful biological model:

Solve a focused problem where clear feedback can be obtained on a reasonable timescale
- Create clear success metrics before starting
Prioritize data quality – either collect data de novo or employ large-scale curation
Transformers that have worked for LLMs may not be the solution for biological data
Build hybrid teams with both ML and deep biology expertise

Lucas Harrington