MolEmb: When Multimodal AI Models Become Molecular Search Engines

Large multimodal artificial intelligence models have been presented primarily as systems that «see,» read, and generate responses. The study MolEmb examines a different role: whether the same type of model can be converted into a framework for representing molecules, generating vectors that are reused in property prediction, search, and retrieval of scientific information.

The change is not merely technical. A classic molecular encoder typically takes a representation of the molecule—such as a graph, a SMILES string, or its geometry—and returns a fixed vector. MolEmb proposes an interface where the molecule is presented as a two-dimensional visualization and canonical SMILES, while a natural-language prompt defines the scientific context. Thus, the same molecular identity can take on different useful representations when the query concerns, for example, toxicity, solubility, or crossing of the blood-brain barrier.

From a fixed vector to a family of representations

The authors define the general molecular embedding model as a function that takes two arguments: a molecular profile containing the molecule’s available faces and a semantic context. The output remains a fixed-length vector, but it is not necessarily the same for every use. For a given molecule, the set of embeddings resulting from different contexts forms a context-aware family.

This distinguishes three concepts that are often confused. The molecular identity identifies the molecule. The profile describes what observations are available—in this particular study, a 2D image and SMILES. Context specifies the scientific lens through which the representation should be organized. The goal is not for a vector to «know everything,» but to activate the right information for the right task.

For companies investing in AI infrastructure, this concept reflects a broader pattern: value lies not only in content production, but also in the creation of stable representations that power search, recommendations, matching, and retrieval-augmented workflows. The same distinction appears in multi-vector embeddings for corporate search, where the correct representation depends on the domain and the query. In chemistry, however, the cost of an incorrect generalization is high, which is why evaluation must clearly distinguish between prediction, retrieval, and context sensitivity.

How MolEmb adapts an MLLM

MolEmb retains the pretrained backbone of the multimodal large language model and adds small, trainable LoRA adapters. The input combines the two-dimensional molecular representation, the canonical SMILES, and, where necessary, a natural language prompt. The hidden state at the end-of-sequence position is used as the final embedding, without additional mean or CLS pooling.

Training aligns pairs of molecules and descriptions in the same space. The molecular side and the textual side pass through the same MLLM backbone in separate forward passes. A bidirectional contrastive objective of the InfoNCE type brings the correct pair closer together and pushes the incorrect pairs further apart. The temperature is set to 0.07, and the vectors are normalized before calculating the cosine similarity.

The researchers are testing three backbones: Intern-S1-mini 8B, Qwen3-VL-8B, and Qwen3.5-0.8B. This selection allows them to examine both a scientifically pre-trained model and, more generally, multimodal models of different sizes. Training was performed on eight NVIDIA A100 GPUs, with LoRA applied to both the language and visual components.

Alignment Data and Why Their Variety Matters

The basic molecule-text alignment uses three sources: MolTextNet, KnowMol-100k, and ChEBI-20-MM. The MolTextNet schema is based on a corpus of descriptions from a single primary source. The Mixed condition combines the three sources to add names, functional groups, ontology-type definitions, and physicochemical properties.

After the global canonical SMILES check, the Mixed dataset includes 411,709 training pairs, 4,149 validation pairs, and 4,165 test pairs. The team removed 5,041 duplicate pairs. These numbers are important not to showcase scale as an end in itself, but because the study concludes that covering scientific contexts is more important than simply adding more similar descriptions.

In property tests, Mixed alignment improves on certain endpoints, particularly ESOL and Tox21, but it does not outperform other methods across the board. MolTextNet remains stronger on other tasks. This precludes an oversimplified interpretation along the lines of «more datasets always mean a better model.» The composition and semantic diversity of the training data determine what knowledge is transferred.

Prediction of molecular properties in eight benchmarks

The evaluation covers three regression benchmarks—ESOL, Lipophilicity, and FreeSolv—and five classification benchmarks: BACE, BBBP, ClinTox, Tox21, and SIDER. For regression, RMSE is used, where a lower value is better. For classification, ROC-AUC is used, with macro averaging applied to the multi-label datasets Tox21 and SIDER.

The direct adaptation of MLLMs, without prior molecule-text alignment, already matches the performance of several pretrained graph methods, especially in regression tasks. For example, Intern-S1-mini Direct achieves an RMSE of 0.827 on ESOL and 1.839 on FreeSolv, while on ClinTox it achieves a ROC-AUC of 99.51%. These are results specific to this protocol and do not prove that an MLLM generally outperforms other methods in every setting.

The most cautious conclusion is that the multimodal interface can be optimized for molecular prediction. Removing the 2D image leads to small but consistent losses on most tasks, so the visual aspect adds information beyond SMILES. Nevertheless, prediction using a task-specific head is not sufficient to prove that we have a reusable embedding model.

The Crucial Test: Molecule-Text Retrieval

To verify whether the space is truly shared between molecules and text, the authors perform molecule-to-text and text-to-molecule retrieval on 2,970 held-out MolTextNet pairs. The random R@1 is approximately 0.034%. An off-the-shelf Qwen3-VL-Embedding-8B achieves approximately 3% R@1 in both directions, indicating that a general multimodal embedding does not, on its own, possess a reliable chemical space.

After contrastive alignment, all MolEmb backbones exceed 73% R@1 in both directions. Intern-S1-mini achieves 85.39% for molecule-to-text and 84.88% for text-to-molecule, while Qwen3.5-0.8B achieves 83,97% and 84,58%, respectively. The scientifically pre-trained Intern-S1-mini achieves the best R@1 performance among the three backbones.

The comparison shows why it is not enough to simply purchase a «general-purpose embedding API» and assume that it will work in a specialized scientific field. The backbone provides capabilities and a common interface, but a reliable domain space results from targeted alignment and validation using data that represents real-world usage. The same reasoning applies when designing Hybrid AI search using keywords, embeddings, and fallbacks.

MolCAR: When a Single Molecule Has Multiple Correct Answers

The MolCAR benchmark isolates a more challenging question: Can the model select the correct information for the same molecule when only the instruction changes? Each target document links a real experimental outcome from one of eight task families to the same molecular profile. Thus, the model cannot succeed simply by recognizing the molecule. It must discern the scientific context.

MolCAR-Structured deterministically generates cards from real labels and molecular descriptors. The evaluation set consists of 4,092 molecule-context pairs across 1,653 held-out molecules. MolCAR-Natural replaces the structured cards with scientific-style notes generated by Intern-S1 235B, but reconnects the verified observed results and the molecular profile to preserve the ground truth.

The eight metrics include BACE-1 inhibition, blood-brain barrier permeability, water solubility, hydration free energy, lipophilicity, Tox21 toxicity outcomes, clinical toxicity, and side-effect profile. The main metric, Context R@1, examines whether the correct context ranks first among the documents for the same molecule, largely eliminating the advantage of simple identity recognition.

The Failure of General Alignment and the Solution

Prior to continued alignment, task instructions have little effect on context ranking. In MolCAR-Structured, Intern-S1-mini achieves 44.4% Context R@1 with instructions and 40.4% without instructions. Qwen3.5-0.8B scores 41.11 TP3T compared to 40.41 TP3T. In other words, the general molecule-text alignment model learned to link molecules and descriptions, but it did not learn to reliably distinguish between multiple valid descriptions of the same molecule.

The authors attribute this breakdown to the structure of the data: typical descriptions are relatively narrow and templated, and rarely present the same molecule through different task lenses. MolCAR-Train adds precisely this supervision. It includes 10,844 pairs for continued alignment, after evaluation molecules based on canonical SMILES have been removed from the training set.

After continued alignment, the Intern-S1-mini achieves 99.81 TP3T Context R@1 on the structured benchmark, compared to 40.41 TP3T without task instructions. The Qwen3.5-0.8B achieves 69.41 TP3T. In the natural variant, the corresponding values are 99.21 TP3T and 72.01 TP3T. This change does not mean that the open scientific question has been resolved. It demonstrates, in a controlled benchmark of known task families, that context sensitivity can be learned when the right supervision is provided.

The Essential Data Lesson for AI Products

MolEmb’s most important finding is that context-aware embedding is primarily a property of the training data. A larger or more general backbone is not sufficient if the training corpus does not demonstrate how the relevant information changes depending on the task. Embeddings do not derive their operational significance from the model’s name, but from the contrasts they observed during training.

This principle extends well beyond chemistry. In an e-commerce retrieval system, the same product may be relevant for different purposes: compatibility, profit margin, sustainability, or fast delivery. In a B2B knowledge base, the same document may be evaluated differently for sales enablement, compliance, or technical support. This is a design analogy, not a result measured by the study. The practical lesson is to collect task-conditioned examples that correspond to actual product decisions.

For AI and marketing technology teams, this also changes how we measure performance. A single retrieval score may mask the fact that the system correctly identifies the object but ignores the intent. Tests are needed that keep the identity constant and change only the context, exactly as MolCAR does. Similar attention is needed in Multimodal image search using embeddings, where the most powerful general model is not always the best solution for that particular retrieval task.

What the research does not prove

MolEmb is considered an adaptation route rather than a fully scaled molecular foundation model. The molecular profile is limited to 2D depictions and canonical SMILES. It does not include conformers, reactions, spectra, or assay metadata, although these could be integrated into the same interface in the future.

MolCAR covers eight property-related task families and does not test transfer to completely unknown task families or open-ended scientific search. Furthermore, the Mixed corpus remains small and less diverse than industrial-scale embedding pretraining. Therefore, the 99.81 TP3T score on a controlled context benchmark should not be presented as a general measure of accuracy in drug discovery.

There is still the usual gap between benchmarks and production use: data quality, drift, scientific validation, computational cost, and governance. This paper documents a feasible architectural approach and a diagnostic tool, not a ready-to-use clinical or pharmaceutical decision-making system. In a production environment, documented risk controls and human oversight are also required, in accordance with the NIST AI RMF framework.

A Practical Strategy for Teams Building Retrieval Systems

The first step is to define the object’s views and usage contexts separately. In this study, the views are images and SMILES, while the contexts are scientific papers. In another domain, the views might be text, images, attributes, and history, while the contexts must be derived from actual business decisions.

Next, a benchmark is needed that tests difficult negative examples. The correct object with the wrong context must be considered incorrect; otherwise, the metric will reward simple identification. The evaluation must also distinguish between object retrieval, document retrieval, and within-object context ranking, so that it is clear where the system is failing.

Finally, the team needs a data loop. When context routing fails, the solution is not automatically a larger model. MolEmb shows that outcome-grounded, task-diverse supervision can substantially alter the geometry of the embedding space through parameter-efficient adaptation. The priority, therefore, is to improve the structure of the examples and repeat the evaluation without leakage.

What We Take Away from MolEmb

The study shows that multimodal language models can serve as a basis for reusable molecular embeddings, not just as generators or chemistry assistants. Simple direct adaptation performs competitively on several property benchmarks, molecule-text contrastive alignment creates a robust shared retrieval space, and targeted continued alignment adds task sensitivity.

The most well-founded conclusion is not that specialist molecular encoders have become obsolete. It is that a second path is emerging: a common multimodal interface, natural-language conditioning, and embeddings that can serve more than one downstream workflow. The success of this path will depend on datasets that capture not only what a molecule is, but also why the user is examining it.

For a business, the practical value lies in the rigor of the evaluation. The model must demonstrate that it retrieves the correct information for the intended purpose, using sufficient data, clear baselines, and operational limits. Simply choosing a larger model does not replace this work.