SlowGuess/NeurIPS_2023_OCR huggingface dataset:
'Rephrase the text of the following prompt by adding more explanation: \n'
Figure 8: Few-shot performance on each dataset
Table 15: Pretraining ablation study on Pharmacokinetic tasks
bbbp	cyp450	Average_pha
bioactivity assay only	0.5313	0.6829	0.6071
physico-chemical only	0.5932	0.4976	0.5454
both	0.5939	0.7125	0.6532
Table 16: Pretraining ablation study on Physical-chemical tasks
esol	freesolv	lipo	Average_phy
bioactivity assay only	-	-	-
physico-chemical only	1.1822	5.2935	1.3778	2.6178
both	1.1320	5.1027	1.3450	2.5266
'Rephrase the text of the following prompt shorter: \n'
Given a task instruction, we rephrase the instruction by the prompts above. Here is an example of four types of rephrased task instruction from Toxcast:
"CEETOX_H295R_ANDR, is one of 23 assay component(s) measured or calculated from the CEETOX_H295R assay. It is designed to make measurements of hormone induction, a form of inducible reporter, as detected with absorbance signals by HPLC-MS-MS technology.Data from the assay component CEETOX_H295R_ANDR was analyzed into 2 assay endpoints. This assay endpoint, CEETOX_H295R_ANDR_dn, was analyzed in the positive fitting direction relative to DMSO as the negative control and baseline of activity. Using a type of inducible reporter, loss-of-signal activity using HPLC-MS-MS was used to understand synthesis of Androstenedione in H295R cell line at 48hr of chemical exposure. To generalize the intended target to other relatable targets, this assay endpoint is annotated to the steroid hormone intended target family, where the subfamily is androgens. Is this molecule effective to this assay?"
"The CEETOX_H295R_ANDR is a component of the CEETOX_H295R assay, which measures hormone induction through absorbance signals detected by HPLC-MS-MS technology. The data from CEETOX_H295R_ANDR has been analyzed into two endpoints, with CEETOX_H295R_ANDR_dn being analyzed in relation to DMSO as the negative control and baseline of activity. HPLC-MS-MS was used to detect loss-of-signal activity and understand the synthesis of Androstenedione in H295R cell line after 48 hours of chemical exposure. This assay endpoint is related to the steroid hormone intended target family, specifically the subfamily of androgens, and can be generalized to other similar targets. Can this assay be effectively performed using this molecule?"
"The CEETOX_H295R_ANDR assay component is just one of the 23 assay components that are measured or calculated from the CEETOX_H295R assay. The assay is specifically designed to measure hormone induction, which is a form of inducible reporter, and is detected using absorbance signals by means of HPLC-MS-MS technology. The data obtained from the CEETOX_H295R_ANDR assay component was analyzed into two assay endpoints. The CEETOX_H295R_ANDR_dn assay endpoint was analyzed in the positive fitting direction in relation to DMSO as the negative control and activity baseline. To understand the synthesis of Androstenedione in the H295R cell line after 48 hours of chemical exposure, loss-of-signal activity was
used with HPLC-MS-MS technology. This endpoint is annotated to the steroid hormone intended target family to help other related targets, where the subfamily is androgens. Can it be determined if this particular molecule exhibits desirable efficacy to be utilized in this particular assay?"
"The CEETOX_H295R_ANDR is an assay component that is one of the 23 components that are measured or calculated from the CEETOX_H295R assay. It is intended to measure hormone induction, which is a form of inducible reporter, and the measurement is done with the help of absorbance signals using HPLC-MS-MS technology. The data obtained from the measurement of assay component CEETOX_H295R_ANDR is analyzed into two assay endpoints. One of these endpoints, CEETOX_H295R_ANDR_dn, is analyzed in the positive fitting direction, relative to DMSO, which is used as the negative control and baseline for activity. The HPLC-MS-MS technology is used to detect the loss-of-signal activity, which helps in understanding the synthesis of Androstenedione in H295R cell line after 48 hours of chemical exposure. To make the intended target more comprehensive and relatable to other targets, the assay endpoint is annotated to the steroid hormone intended target family, where the subfamily is androgens. Can this molecule be used for this assay?"
"CEETOX_H295R_ANDR is one of 23 components in the CEETOX_H295R assay, measuring hormone induction detected with absorbance signals by HPLC-MS-MS. It's analyzed into 2 endpoints, with CEETOX_H295R_ANDR_dn being the positive fitting direction relative to the negative control. It analyzes the loss-of-signal activity to understand Androstenedione synthesis in H295R cell line after 48hr chemical exposure. It's annotated as a steroid hormone intended target in androgens sub-family. Is molecule suitable for assay?"
C.6 Instruction Ablation
To ablate the explanation-based instruction, we remove the explanation and only keep the assay name. The ablated instruction for the instruction above is:
"The assay name is CEETOX_H295R_ANDR. Is this molecule effective to this assay?"
C.7 Attention Visualization
We present visualizations of the attention of text tokens to molecule graphs, demonstrating how our unified transformer incorporates molecule information using various instructions. We randomly sample molecules and attention heads for visualization. To emphasize high-level features, we focus on visualizing the attention patterns of the last layer. The redder means the larger attention value.
For BACE instruction, we visualize the attention of several keywords marked in red to molecules:
For BBBP instruction:
'In general, molecules that passively diffuse across the brain blood barrier have the molecular weight less than 500, with a LogP of 2-4, and no more than five hydrogen bond donors or acceptors. Does the molecule adhere to the three rules or not?'
(a) BACE1
(b) aspartic-acid
(c) Alzheimer
(d) myelin
(e) aspartic
(f) bilodal
(g) catalytic
(h) cleft
(i) effective
Figure 9: Visualization of attention for BACE on molecule O=C(N(C)C1CCCCC1)CCc1cc2c(nc1N)cccc2
(i) effective
Figure 10: Visualization of attention for BACE on molecule O=C(NCC1CCCCC1)CCc1cc2c(nc1N)cccc2
(g) catalytic
$\mathrm{O = C1NC(CN1Cclcccc1)(Cc1cccccc1)C(=O)NC(Cclcccccc1)C(O)C[NH2 + ]Cc1cc(N(C)C)ccc1}$
(i) effective
Figure 11: Visualization of attention for BACE on molecule
(i) effective
Figure 12: Visualization of attention for BACE on molecule  $\mathrm{O = C1N(C)C(=[NH2 + ])NC1(c1cccccc1)c1cccccl}$
(i) effective
Figure 13: Visualization of attention for BACE on molecule O(c1cc2CN(CCCC(=O)N(C)C3CCCCC3)C3CCCCC3)C(=[NH+]c2cc1)N)c1cccc1
(a) brain blood barrier
(b) molecular weight
(c) 500
(d) LogP
(e) 2-4
(f) five
(g) hydrogen bond donors
(h) acceptors
(i) three rules
Figure 14: Visualization of attention for BBBP on molecule
(i) three rules
Figure 15: Visualization of attention for BBBP on molecule Cc1nccc2c1[nH]c3cccccc23
(i) three rules
Figure 16: Visualization of attention for BBBP on molecule CC1=C2NC3=CC(=O)C=CC3=C2C=CN1
(i) three rules
Figure 17: Visualization of attention for BBBP on molecule COc1cc2CCN(C)C3CC4(C=CC(=O)C=C4)c(c1O)c23
(i) three rules
Figure 18: Visualization of attention for BBBP on molecule CCC1(C)CC(=O)NC1=O