RiteeshGrandhi/sciclaimeval-shared-task huggingface dataset:
Baselines.
We compare FuncMol and FuncMol dec to three state-of-the-art approaches. EDM [ 4 ] and GeoLDM [ 53 ] are diffusion models operating on point clouds (the latter is a latent-space extension of the former). VoxMol [ 5 ] is a voxel-based generative model that uses neural empirical Bayes, similar to our generative approach.
All of the methods generate molecules as a set of atom types and their coordinates.
EDM and GeoLDM apply diffusion directly to point clouds, while VoxMol and FuncMol rely on an additional (cheap) post-processing step to extract atomic coordinates from voxel grids or modulation codes, respectively.
We follow previous work [ 58 , 54 , 5 , 62 , 86 ] , and use standard cheminformatics software (OpenBabel [ 87 ] ) to determine the molecule’s atomic bonds given their atomic coordinates.
The same post-processing is applied to all models for fairness of comparison.

Metrics . We consider several metrics used in previous work [ 5 ] to benchmark unconditional molecule generation for the standard QM9 and GEOM-drugs datasets (for the CREMP metrics, see Section ˜ 5.4 ): stable mol and stable atom , the percentage of stable molecules and atoms (as defined in [ 4 ] ); validity , the percentage of generated molecules that passes RDKit [ 88 ] ’s sanitization filter; uniqueness , the proportion of valid molecules that have different canonical SMILES; valency W 1 , the Wasserstein distance between the distribution of valencies in the generated and test set; atoms TV and bonds TV , the total variation between the distribution of atom types and bond types; bond length W 1 and bond angle W 1 , the Wasserstein distance between the distribution of bond and lengths.
We also report the average sampling time per molecule . In the case of our method, this time includes the MCMC “walk” steps, the denoising “jump”, the rendering, peak detection and bond inference.

To further investigate the quality of molecular conformations and other molecular properties on GEOM-drugs, we consider some additional metrics. These include: single fragment , the percentage of molecules that contains only a single fragment; median strain energy [ 89 ] , the difference between the internal energy of the generated molecule’s pose and a relaxed pose of the molecule using RDKit’s Universal Force Field [ 90 ] , computed over all molecules; ring size TV , the total variation between the empirical distribution of ring sizes (i.e. number of heavy atoms in rings) in generated and test sets; number of atoms/mol TV , the total variation between the empirical distribution of number of atoms per molecule in generated and test sets (in the case of molecules with multiple fragments, we consider only the largest fragment); QED, SA and logp , measure the drug-likeness score [ 91 ] , the synthesizability score [ 92 ] and the lipophilic efficiency, respectively (computed with RDKit).
Ablations.
In Appendix ˜ E we report a series of ablation studies for the neural field and the generative model. Section ˜ E.1 measures the reconstruction quality of the training molecules. Section ˜ E.2 illustrates the improvements due to continuous atomic coordinate refinement. Section ˜ E.3 shows that our field-based decoder is robust to noise, making it an ideal choice for generative modeling. Section ˜ E.4 ablates the impact of the number of walk steps in the WJS scheme of Section ˜ 4.2 .
Finally, Section ˜ E.5 ablates the impact of the chosen resolution when sampling codes and decoding them back to molecules. In practice, we observe that 0.25Å  provides a good trade-off between the sampling time and the quality of the generated molecules.
Results on QM9
As pointed by previous authors [ 93 , 4 ] , this dataset is not fully suited for unconditional generative models: a model that captures the training distribution will have to generate samples from training set, due to the enumeration.
However, many previous work report results on this dataset. Therefore, we also show results for completeness.

Table 1 report the metrics described in Section ˜ 5.1 .
We see that FuncMol slightly improves VoxMol and both models perform worse compared to the equivariant point-cloud based baselines.
We note that sampling time of FuncMol is an order of magnitude better than baselines.
Results on GEOM-drugs
Table 2 reports the same set of metrics as in the previous dataset.
FuncMol performs favorably over point cloud diffusion models and is close to VoxMol’s performance.
In particular, FuncMol and VoxMol generate molecules that are significantly more stable and better capture the distribution of bond angles. Table ˜ 3 shows results on additional metrics (described in Section ˜ 5.1 ).
We also include the following plots of Appendix ˜ F : Figure ˜ 9 shows the cumulative distribution function of strain energies for generated molecules and Figures ˜ 10 and 11 show the histograms of the other metrics.

The results are clear: FuncMol samples better drug-like molecules than point-cloud diffusion models .
In fact, about half the molecules of point cloud methods have multiple fragments, they have an order of magnitude higher median strain energy, the distribution of ring sizes is off and the QED, SA and logp scores are lower.
The results of FuncMol are close to VoxMol in most but not all metrics.
However, our approach is much more scalable and efficient: FuncMol generates molecules an order of magnitude faster than previous methods (see the last column of Table ˜ 2 ). Appendix ˜ H shows some molecules generated by FuncMol on GEOM-drugs.
Results on CREMP
To showcase the scalability of FuncMol, we train it on a dataset of larger molecules.
We choose the macrocylic peptides of CREMP, that contains on average 74 atoms, making it challenging to train models using point-clouds.
These molecules also pose serious limitations to voxel-based approaches as they require modeling a volume of 24^{3} cubic Angstroms.
We tried to train VoxMol on this dataset using the official implementation, but did not succeed: it takes around 10 hours per epoch on 4 A100 GPUs, while FuncMol takes less than 12 minutes per neural field epoch and 15s per denoiser epoch.
We use the same code dimension and neural field architecture as in GEOM-drugs, therefore the computational training cost of FuncMol remains unchanged.

Figure ˜ 3 shows that FuncMol captures well the underlying distribution of macrocyclic conformations. We show the distribution of bond angles ( \theta_{1} , \theta_{2} , \theta_{3} ) and dihedrals ( \phi , \psi , \omega ) of both molecules from test set and generated molecules. We also show the KL-divergence between test and generated samples. Approximately 65% of the generated molecules were valid peptides (that is, we could extract a sequence of amino acids from the SMILES strings). The Ramachandran plots [ 94 ] show that FuncMol recovers the main modes of the distribution. We note that the bond angles and dihedrals distributions are learned without having any explicit priors on the structure of these peptides. Appendix ˜ H shows some generated macrocylic peptides. Finally, our model takes around 1.5s to generate a molecule.
For reference, should VoxMol be trained successfully, it would take over a minute to sample a single molecule (assuming similar sampling parameters as in other datasets). This is a substantial speedup that showcases the potential of FuncMol to scale to even larger molecules.
Discussion
We introduce a new continuous representation of 3D molecules based on their atomic occupancy field and a score-based generative model operating on this representation.
Each molecule is assigned a code that modulates a shared neural field.
We demonstrate that we can build an all-atom generative model of 3D molecules, FuncMol, with state-of-the-art sampling time and competitive performance on challenging drug-like datasets.
We believe that this model introduces a new paradigm for all-atom 3D modeling of molecules that has many useful properties, namely scalability, expressivity, and flexibility, as it can model various molecular design problems (involving structure, electron densities, etc.) with minor architecture changes. Future research directions include exploring different neural field architectures, adapt the model for conditional generation (e.g., structure conditioning) or model the molecular bonds alongside the atomic coordinates 3 3 3 Recent work [ 54 , 95 ] show that this improves generation quality. See Appendix G to see how our method compares with a representative baseline that uses the bond information. . Moreover, the scalability of FuncMol can be a potential alternative for all-atom representations of large biomolecules.
Acknowledgements
We would like to thank the Prescient Design team for helpful discussions and Genentech’s HPC team for providing a reliable environment to train and analyze models.
E.1 Reconstruction quality
We analyze the reconstruction quality of the fields and molecules we compressed with latent codes.
Field reconstruction.
We report in Table ˜ 5 the reconstruction performance of the molecular fields using our conditional MFN architecture and learned codes.
Molecule reconstruction.
To make more sense out of these raw reconstruction metrics, we show that the 3D molecules decoded from learned training codes are valid and stable molecules.
This means that we successfully compressed the training data into low-dimensional latent vectors via our field-based representation. Table ˜ 6 reports metrics of molecules decoded from the learned training codes for GEOM-drugs and QM9.
For each dataset, we display the metrics of the training molecules, the molecules decoded from our training codes and the molecules derived from a voxelized representation of the field at a resolution of .25Å.
We observe that the molecules rendered from codes have better metrics than those derived from voxels showing the validity of our new representation.
These results are an upper bound to the results in Table ˜ 2 .
E.2 Atomic coordinate refinement
A big advantage of using neural fields is that we can represent signals, here molecules, in continuous space rather than in discrete space as in voxel representations.
The continuous refinement introduced in Section ˜ 3.3 improves the quality of the molecules by finding more precise atomic coordinates. Table ˜ 7 shows the improvement of this continuous refinement over the refinement used in [ 5 ] that operates in discrete space: by going to continuous space, we overcome the limitations of discrete grids and substantially improve the stability of the molecules and the angle between bonds.
E.3 Neural field robustness to noise
Here, we analyze how the neural field is robust to noise on the modulation code space. Figure ˜ 8 illustrates how molecular stability and the distance between the distribution of bond angles change as we increasingly add noise to the codes.
Each point consists of the average of the metric over 4000 random modulation codes from the validation set.
Interestingly, the neural field is quite robust to noise as we see that the metrics unchanged even at a reasonable amount of noise.
We believe that this code robustness to noise helps better learn the denoiser.
E.4 Number of walk steps
Table ˜ 8 shows how the quality of molecules changes as we increase the number of walk steps K in the walk-jump sampling for FuncMol on GEOM-drugs.
In this experiment, we sample 2000 samples and compute metrics w.r.t the validation set. First, we observe that some metrics improve (e.g., molecular stability) while other get worse (e.g., uniqueness). Second, we observe that the walk-jump chain is extremely stable, allowing us to perform as much as 50000 MCMC steps in the chain without breaking it. Finally, sampling time does not increase significantly: going from 500 to 50000 steps only results in a 10 \times increase in sampling time (this is because the sampling bottleneck is on finding the atomic coordinates).
E.5 Impact of resolution at decoding time
Finally, we measure the impact of resolution on sampling time and quality. As expected, sampling time becomes slower as we we have finer resolution. We also notice that finer resolution has better results than coarse ones (although the results stop improving). We chose resolution 0.25Å as it provides a good trade-off between performance and speed. The table below shows the results as a function of resolution.
Figure 1: (a) a conditional neural field encodes a molecular field v into a low dimensional latent code z . (b) using a learned score function g_{\theta} , FuncMol performs sampling in latent space via Langevin MCMC. These codes are decoded back into molecules.; (a) a conditional neural field encodes a molecular field v into a low dimensional latent code z . (b) using a learned score function g_{\theta} , FuncMol performs sampling in latent space via Langevin MCMC. These codes are decoded back into molecules.
Figure 2: Conditional neural field f_{\phi} using the multiplicative filter network architecture. (a) A latent code z and some coordinates x are given as input to the model that outputs the occupancy field at that location for the corresponding molecule, f_{\phi}(x,z) . (b) The code and coordinates are processed via FiLM layers and Hadamard products. We denote the overall operation at layer l as H^{(l)} .; Conditional neural field f_{\phi} using the multiplicative filter network architecture. (a) A latent code z and some coordinates x are given as input to the model that outputs the occupancy field at that location for the corresponding molecule, f_{\phi}(x,z) . (b) The code and coordinates are processed via FiLM layers and Hadamard products. We denote the overall operation at layer l as H^{(l)} .
Figure 3: Qualitative evaluation on CREMP following [ 84 ] . Left: Comparison of the bond angles ( \theta_{1} , \theta_{2} , \theta_{3} ) in each residue and dihedral distributions ( \phi , \psi , \omega ) for each residue from the reference test set (gray) and the generated samples (blue).
KL divergence is calculated as \text{KL}(\text{test}\mid\mid\text{sampled}) .
Right: Ramachandran plots [ 94 ] (colored by density where darker tones represent high density regions).; Qualitative evaluation on CREMP following [ 84 ] . Left: Comparison of the bond angles ( \theta_{1} , \theta_{2} , \theta_{3} ) in each residue and dihedral distributions ( \phi , \psi , \omega ) for each residue from the reference test set (gray) and the generated samples (blue).
KL divergence is calculated as \text{KL}(\text{test}\mid\mid\text{sampled}) .
Right: Ramachandran plots [ 94 ] (colored by density where darker tones represent high density regions).
Figure 4: Auto-encoding approach for neural field representation. A voxelized representation of molecule is encoded int the latent space z with a 3D CNN. This representation is then decoded with a conditional MFN for any point x in space.; Auto-encoding approach for neural field representation. A voxelized representation of molecule is encoded int the latent space z with a 3D CNN. This representation is then decoded with a conditional MFN for any point x in space.
Figure 5: Interpolation in the latent modulation space for different pairs of molecules from GEOM-drugs. Each interpolated codes is protected back to the learned manifold of molecules via a noise/denoise operation. FuncMol produces semantically meaningful patterns in the interpolated space and we observe that molecules close in latent space share similar structure.; Interpolation in the latent modulation space for different pairs of molecules from GEOM-drugs. Each interpolated codes is protected back to the learned manifold of molecules via a noise/denoise operation. FuncMol produces semantically meaningful patterns in the interpolated space and we observe that molecules close in latent space share similar structure.
Figure 6: t-SNE plots of latent modulations codes of QM9 molecules for different molecular properties. For each plot, we pick 200 molecules from validation set with high value of a property (blue) and 200 with low value (red). We show results for four properties: (a) heat capacity ( C_{v} ), (b) internal energy ( U_{0} ), (c) isotropic polarization ( \alpha ) and (d) dipole moment ( \mu ).; t-SNE plots of latent modulations codes of QM9 molecules for different molecular properties. For each plot, we pick 200 molecules from validation set with high value of a property (blue) and 200 with low value (red). We show results for four properties: (a) heat capacity ( C_{v} ), (b) internal energy ( U_{0} ), (c) isotropic polarization ( \alpha ) and (d) dipole moment ( \mu ).
Figure 7: Performance of linear regression model (a.k.a linear probing) trained on modulation codes to predict molecular properties on QM9. We show the scatter plots and Spearman correlation for four different properties: (a) heat capacity ( C_{v} ), (b) internal energy ( U_{0} ), (c) isotropic polarization ( \alpha ) and (d) dipole moment ( \mu ).; Performance of linear regression model (a.k.a linear probing) trained on modulation codes to predict molecular properties on QM9. We show the scatter plots and Spearman correlation for four different properties: (a) heat capacity ( C_{v} ), (b) internal energy ( U_{0} ), (c) isotropic polarization ( \alpha ) and (d) dipole moment ( \mu ).
Figure 8: Ablation: code robustness to noise on GEOM-drugs (blue) and QM9 (red). Stable molecule (a) and bond angle distance (b) metrics as we increasingly add noise to the codes. Metrics are computed with 4000 generated samples on validation reference set.; Ablation: code robustness to noise on GEOM-drugs (blue) and QM9 (red). Stable molecule (a) and bond angle distance (b) metrics as we increasingly add noise to the codes. Metrics are computed with 4000 generated samples on validation reference set.
Figure 9: Cumulative distribution function of strain energy of generated molecules on GEOM-drugs based on 10000 molecules.; Cumulative distribution function of strain energy of generated molecules on GEOM-drugs based on 10000 molecules.
Figure 10: Histograms (over 10000 samples) showing (first row) distribution of number of fragments, (second) distribution of ring size, and (third) distribution of number of atoms per molecule.; Histograms (over 10000 samples) showing (first row) distribution of number of fragments, (second) distribution of ring size, and (third) distribution of number of atoms per molecule.
Figure 11: Histograms (over 10000 samples) showing (top) distribution of QED, (mid) SA and (bottom) log p.; Histograms (over 10000 samples) showing (top) distribution of QED, (mid) SA and (bottom) log p.
Figure 12: Generated samples from FuncMol trained on GEOM-drugs.; Generated samples from FuncMol trained on GEOM-drugs.
Figure 13: Generated samples from FuncMol trained on CREMP.; Generated samples from FuncMol trained on CREMP.
Figure 14: Two single MCMC chains generated by FuncMol, initialized randomly with different seeds (seen from left to right, top to bottom). Molecules are generated after each 200 "walk" steps.; Two single MCMC chains generated by FuncMol, initialized randomly with different seeds (seen from left to right, top to bottom). Molecules are generated after each 200 "walk" steps.
QM9 results w.r.t. test set for 10000 samples per model. \uparrow / \downarrow indicate that higher/lower numbers are better. The row data are randomly sampled molecules from the validation set. We report 1-sigma error bars over 3 sampling runs.
GEOM-drugs results, standard metrics w.r.t. test set for 10000 samples per model. \uparrow / \downarrow indicate that higher/lower numbers are better. The row data are randomly sampled molecules from the validation set. We report 1-sigma error bars over 3 sampling runs.
GEOM-drugs results, additional metrics w.r.t. test set for 10000 samples per model. \uparrow / \downarrow indicate that higher/lower numbers are better. The row data are randomly sampled molecules from the validation set. We report 1-sigma error bars over 3 sampling runs.
Comparing walk-jump sampling and diffusion model on QM9 results (10000 samples per model). \uparrow / \downarrow indicate that higher/lower numbers are better. The row data are randomly sampled molecules from the validation set.
Ablation: field reconstruction (whole training set).
Ablation: molecule reconstruction (sample of 4k).
Ablation: continuous refinement improvement on code reconstruction performance. Metrics computed with 4000 generated samples on validation reference set.
Ablation on the number of walk steps K on GEOM-drugs. Metrics computed with 2000 generated samples on test reference set.
Ablation on the impact of resolution on sampling quality on GEOM-drugs. Metrics computed with 2000 generated samples on test reference set.
Comparison of MolDiff with H and FuncMol