zhiminy/EvalEng huggingface dataset:
Repository: snap-stanford/ogb
Issue: A potential bug about DglNodePropPredDataset
Root cause: External Dependency Breakage

I come up with a bug when I try to use `DglNodePropPredDataset` to load dataset `ogbn-papers100M` as DGL form.
I use the script
```
from ogb.nodeproppred import DglNodePropPredDataset
graph_name = "ogbn-papers100M"
ogbg = DglNodePropPredDataset(graph_name)
```
and got the error message
```
File "examples/distributed/graphsage/partition_graph.py", line 21, in load_ogb
    data = DglNodePropPredDataset(name=name, root=root)
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/ogb/nodeproppred/dataset_dgl.py", line 69, in __init__
    self.pre_process()
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/ogb/nodeproppred/dataset_dgl.py", line 76, in pre_process
    self.graph, label_dict = load_graphs(pre_processed_file_path)
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/data/graph_serialize.py", line 195, in load_graphs
    return load_graph_v2(filename, idx_list)
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/data/graph_serialize.py", line 207, in load_graph_v2
    return [gdata.get_graph() for gdata in heterograph_list], label_dict
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/data/graph_serialize.py", line 207, in <listcomp>
    return [gdata.get_graph() for gdata in heterograph_list], label_dict
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/data/heterograph_serialize.py", line 66, in get_graph
    ndict = {
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/data/heterograph_serialize.py", line 67, in <dictcomp>
    ntensor[i]: F.zerocopy_from_dgl_ndarray(ntensor[i + 1])
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/backend/pytorch/tensor.py", line 466, in zerocopy_from_dgl_ndarray
    if data.shape == (0,):
  File "/opt/conda/envs/dgl-dev-gpu-117/lib/python3.8/site-packages/dgl-2.0-py3.8-linux-x86_64.egg/dgl/_ffi/ndarray.py", line 211, in shape
    for i in range(self.handle.contents.ndim)
AttributeError: 'NoneType' object has no attribute 'contents'
Segmentation fault (core dumped)
```
Is there a solution to this bug?

I have encountered this problem recently. Have you solved it？



i also got same problem, has anyone solved it.....
Repository: snap-stanford/ogb
Issue: The mapping from node id to paper id in dataset ogbn-arxiv
Root cause: Interface Contract Mismatch

I've found the _titleabs.tsv_ file containing paper-ids and texts, and the _nodeidx2paperid.csv_ file containing the mapping from node id to paper id. But the paper-ids in the  _nodeidx2paperid.csv_ file do not match any id in the _titleabs.tsv_ file. Did I miss something? Or is some file falling behind the latest version that leads to this issue?

Might be related: https://github.com/snap-stanford/ogb/issues/478
Repository: snap-stanford/ogb
Issue: Feature request: link to direct download of datasets -- for those of us not using Python
Root cause: Unimplemented Feature Gap

I'm running some experiments on a different ML framework, not in Python. While the downloading/dataset library is very convenient, may I suggest adding links to download the data directly as well ? 

Thanks!

ps.: Just unpack the contents of the various `master.csv` to the `README.md`, or even better somewhere in the [OGB](https://ogb.stanford.edu/) site ?
Repository: snap-stanford/ogb
Issue: Detailed sentence bert model for the MAG240M dataset

Hi,the team of ogb. Thanks for your efforts on collecting and processing these valuable datasets! You said you used a RoBERTa sentence encoder to generate a 768-dimensional vector for each paper node in the MAG240M dataset. May I ask which RoBERTa sentence encoder you used? Could you give me a specific huggingface link? Best regards!
<img width="1182" alt="image" src="https://github.com/snap-stanford/ogb/assets/56426168/9b69393b-ab4c-432d-910e-44598bfd8515">
@weihua916 @rusty1s 

https://huggingface.co/sentence-transformers/roberta-base-nli-stsb-mean-tokens
```python
from sentence_transformers import SentenceTransformer
SentenceTransformer('roberta-base-nli-stsb-mean-tokens')
```

> ```python
> mean
> ```

Thank you very much for the quick reply! Because this model is deprecated, do you consider to provide the original feature processing script for the MAG240M paper nodes? So that we can process the feature with more powerful sentence bert model🤗 Best regards!
Repository: snap-stanford/ogb
Issue: ogbn-arxiv text features don't match
Root cause: Validation Gap

I raised a concern in the DGL Github that there's a mismatch between the ogbn-arxiv text features and their graph representation. Wanting to surface it here as well.
https://github.com/dmlc/dgl/issues/7270

The DGL graph node count actually matches the number of nodes referenced here: 
![image](https://github.com/snap-stanford/ogb/assets/7418031/6fdb5738-0d02-4043-bbdd-aeb59fbf38a4)
(https://ogb.stanford.edu/docs/nodeprop/)

So, it seems that the text abstracts in https://snap.stanford.edu/ogb/data/misc/ogbn_arxiv/titleabs.tsv.gz (179,719 total) contain additional nodes that were not included in the other graph? 

How do I map between the two?

Check Issue #222 might be helpful.
Repository: snap-stanford/ogb
Issue: ogbg-molchembl dataset fails due to default pickle_protocol
Root cause: Configuration Error

**How to repro:**

`ds = GraphPropPredDataset('ogbg-molchembl', root='/tmp/ogb_datasets')` fails at the `torch.save` step of the pre_process method, because pickle "cannot serialize a string larger than 4 gb". 

**What I've done:**

I've tried setting torch.serialization.DEFAULT_PROTOCOL = 4 (which according to [this](https://docs.python.org/3/library/pickle.html) adds support for large objects) before calling above, but this did not help -- I think it should be passed as arg to torch.save.

This does not occur with the dgl variant.

Interesting. I tested the following locally, and it worked fine. Could you clarify what you mean by "This does not occur with the dgl variant."?

```python
from ogb.graphproppred import DglGraphPropPredDataset
    
d_name = 'ogbg-molchembl'
dataset = DglGraphPropPredDataset(name = d_name) 
```

Ah I mean the Dgl variant works, and it's the pure python dataset which fails to save the pickle file. I think the further processing to Dgl datastructure reduces size of the pickle enough.

Edited root comment to reflect it's `GraphPropPredDataset` that fails.

You are right, thanks for noticing this. I have resolved the issue in the master branch by using `protocol = 4` in `torch.save()`.
Repository: snap-stanford/ogb
Issue: Bad zip file error while extracting papers100M data
Root cause: External Dependency Breakage

Hi, I have tried to download ogbn-papers100M data using 
"from ogb.nodeproppred import NodePropPredDataset
dataset = NodePropPredDataset(name = 'ogbn-papers100M')" - Using this it doesn't download entire data and throws error during extraction

I have also tried to download from the link used by ogb library "http://snap.stanford.edu/ogb/data/nodeproppred/papers100M-bin.zip" using "wget --tries=inf http://snap.stanford.edu/ogb/data/nodeproppred/papers100M-bin.zip";
After many retries the zip file is downloaded, but when I extract the file manually or give the zip file in the path of NodePropPredDataset it throws CRC error

Attaching error shown during manual extraction:

Using 7-zip:
![Screenshot 2024-05-09 175025](https://github.com/snap-stanford/ogb/assets/18468722/bcd9b1cf-cb61-4f97-b33e-ae77c76b4b7d)

Using unzip command in linux:
![Screenshot 2024-05-09 175131](https://github.com/snap-stanford/ogb/assets/18468722/f09aa957-73e1-4ce8-ace0-bb74be2bf622)

Is there any way around this??
Repository: snap-stanford/ogb
Issue: Clarification on Handling ogbl-wikikg2 as Heterogeneous Dataset
Root cause: Documentation Deficiency

Hi OGB Team,

I am currently working with the ogbl-wikikg2 dataset for a knowledge graph completion task. According to the dataset description on the OGB website, ogbl-wikikg2 is a knowledge graph (KG) that contains triplet edges (head, relation, tail) with 2,500,604 entities and 535 relation types. This suggests that the dataset inherently contains heterogeneous data due to the multiple relation types.

However, I noticed that in the dataset's metadata, is_hetero is set to False. This raises a few questions and potential issues for users who wish to treat this dataset as a heterogeneous graph for KG completion tasks.

**Questions:**
Why is is_hetero set to **False** for ogbl-wikikg2?

Given the nature of the data, should this be considered a heterogeneous graph dataset?

**Impact on Model Implementation:**

For tasks such as KG completion, how should users handle relation types if the dataset is treated as homogeneous?
Can you provide guidance or best practices for users who want to leverage the heterogeneous nature of the dataset (i.e., using relation types effectively in models)?
**Evaluation Metric:**

The current evaluation setup (e.g., Mean Reciprocal Rank, MRR) seems to align with KG completion tasks. Are there specific reasons for not treating this dataset as heterogeneous?
Documentation and Examples:

Could you provide more detailed documentation or examples on how to implement models that can handle the multiple relation types in ogbl-wikikg2?
Suggested Improvements:

Provide additional examples or guidelines on handling ogbl-wikikg2 as a heterogeneous graph, specifically for KG completion tasks.
Clarify any potential implications for using the dataset in its current form vs. treating it as heterogeneous.
I believe addressing these points would help many researchers and practitioners better utilize the ogbl-wikikg2 dataset for their projects.

Thank you for your attention to this matter.
Repository: snap-stanford/ogb
Issue: Question about the load of local dataset file By DglNodePropPredDataset
Root cause: Documentation Deficiency

Hello sir, thank you for sharing these good datasets. Today I face some problems with running these codes:
`dataset = AsNodePredDataset(DglNodePropPredDataset("ogbn-products"))
    g = dataset[0]`
My network could not support the running of these codes. So I try to download the product.zip from the link of [http://snap.stanford.edu/ogb/data/nodeproppred/products.zip](url) and save it on a path like './dataset/product.zip' How can I let the code load the dataset from this path rather than download again.
Repository: snap-stanford/ogb
Issue: Submission to PCQM4Mv2
Root cause: Configuration Error

Hi OGB team,

I have made a submission 1 week ago, Aug 12. But the evaluation results still not appeared in the leaderboard yet.

May I know what's the reason?

My submission title is `GraphGPT(MLM pretrained)`. The email is james.zqf@alibaba-inc.com.

Thank you.

@weihua916 

Thanks for the reminder. Just updated.

> Thanks for the reminder. Just updated.

Hi @weihua916 , I just checked the leaderboard: https://ogb.stanford.edu/docs/lsc/leaderboards/#pcqm4mv2

My submission still not there. Could you please have a check?

Ah good catch. we are taking a look. an issue with github page

It's now up

> It's now up

Thank you for the fix. I just submit a new result with different settings, hope it will appear in the board soon. :)

Uploaded

> Uploaded

Thank you very much. 

Based on the leaderboard, our solution is the current SOTA that does not utilize 3D data. :)

> Uploaded

Hey weihua,

I submitted a new version several days ago, could you please update the leaderboard?

Thank you.

@weihua916 

> > Uploaded
> 
> Hey weihua,
> 
> I submitted a new version several days ago, could you please update the leaderboard?
> 
> Thank you.
> 
> @weihua916

Hey weihua,

It is about 1 week since our latest submission, but it still not appears in the leaderboard.

Could you please have a check and update the leaderboard?

Thank you. @weihua916 

just updated

Hi Weihua @weihua916 ,

Sorry to bother you again. I make submissions to ogbl-ppa and ogbl-citation2 leaderboard 6 days ago, but they are still not in the leaderboard yet.

Could you please have a check and update? Or shall I look for someone else to deal with this?

Thank you very much.

Hi Weihua @weihua916 ,

I make submissions to ogbl-ppa and ogbl-citation2 leaderboard several days ago, but they are still not in the leaderboard yet.

Could you please have a check and update?

Thank you very much.

Hi Weihua @weihua916,

I made a submission to PCQM4Mv2 leaderboard, could you please have a check and update?

Thank you very much.

Hi Weihua @weihua916,

I made a submission to PCQM4Mv2 leaderboard about 1 week ago, could you please have a check and update?

Thank you very much.
Repository: snap-stanford/ogb
Issue: question on batch size in GraphSAINT
Root cause: Documentation Deficiency

Hi, 
I am trying to run the graphSAINT implementation in OGB (https://github.com/snap-stanford/ogb/blob/master/examples/nodeproppred/products/graph_saint.py) on the ogbn-products dataset and I wanted to clarify something about the batch size in this implementation. 

In the code, the default batch size is 20000 and the function below uses this batch size (I do not pass any batch size in the command line). 

**loader = GraphSAINTRandomWalkSampler(sampler_data, batch_size=args.batch_size, walk_length=args.walk_length,
                                         num_steps=args.num_steps, sample_coverage=0, save_dir=dataset.processed_dir)**

However, in the **_train(model, loader, optimizer, device)_** function, in the for loop (**for data in loader:**), I am printing the information for each **data** and the output is attached below. It looks like each mini-batch contains ~75K nodes (not 20K mentioned in the batch size argument passed into the GraphSAINTRandomWalkSampler function). Could you please help me understand these two different values (75K and 20K). Does the batch_size argument in GraphSAINTRandomWalkSampler function not refer to how many nodes should be contained in a mini-batch? If so, what does it refer to? 

![image](https://user-images.githubusercontent.com/18584733/88843354-d5788880-d195-11ea-914f-4c21db43f4ea.png)


Hi and thanks for your interest. In the example, we are using the random walk sampler of the GraphSAINT paper, and batch size refers to the number of nodes in a batch for which we start random walks. Hence, the resulting amount of nodes sampled is usually larger than the batch size (in the worst case its walk length * batch size). I hope this helps!

I see. Thank you for the prompt clarification!
Repository: snap-stanford/ogb
Issue: Submission to ogbl-collab
Root cause: Others

Hello OGB Team,

Last week I filled out the Leaderboard Submission Form and reported the results of my method `TopoLink` on ogbl-collab. However, I found that my results have not been updated on Leaderboard yet. Could you please update it? My email is [liulizhi1996@gmail.com](mailto:liulizhi1996@gmail.com). Thank you!

@weihua916 

updated

Thank you!
Repository: snap-stanford/ogb
Issue: RuntimeError: Stopped downloading due to interruption when downloading ogbl-wikikg2
Root cause: Configuration Error

Hi!

I encountered this error: _RuntimeError: Stopped downloading due to interruption_ when I tried to download ogbl-wikikg2. The process stops at 7/4232 for a while then triggers this error. This error only happens when I try on ogbl-wikikg2, and other datasets (even datasets from large-scale challenges) are downloaded smoothly without any problems.

Any suggestion on how I can fix this?


After investigating, the problem is in the links to download the datasets. If the link starts with `https`, the dataset will be downloaded smoothly. Changing links in `master.csv` from `http` to `https` will solve this issue.

I also has this issue, is it possible to resolve it by installing previous version? 


I didn't try the previous version, but I resolved this by tracing the code to see where download links are defined and changing them from `http` to `https`. This error is typically the firewall of your network, you can try to switch it off (if applicable) or try to make changes as I suggested above.
Repository: snap-stanford/ogb
Issue: Version Check may lead to deadlock of threadings
Root cause: Resource Mishandling

In version.py, you start a new thread to check whether it is the latest version.
```py
import os
import logging
from threading import Thread

__version__ = '1.3.6'

try:
    os.environ['OUTDATED_IGNORE'] = '1'
    from outdated import check_outdated  # noqa
except ImportError:
    check_outdated = None


def check():
    try:
        is_outdated, latest = check_outdated('ogb', __version__)
        if is_outdated:
            logging.warning(
                f'The OGB package is out of date. Your version is '
                f'{__version__}, while the latest version is {latest}.')
    except Exception:
        pass


if check_outdated is not None:
    thread = Thread(target=check)
    thread.start()
 ```
If some network errors occur in the request of version check , this thread remains alive , when the python program is going to quit , this thread may lead to deadlock in multi-threads.
Repository: snap-stanford/ogb
Issue: PCQM4Mv2 invalid SMILES
Root cause: Validation Gap

Dear OGB team,

I have detected that the smiles included in the lsc-pcqm4mv2: 'O[Si]123O[Si]3(O1)(O2)O' (position 51128) in the dataset is an invalid smiles string according to the newest versions of RDKit (concretely, version 2024.09.5). This error can be reproduced in the following way:

```
from ogb.lsc import PygPCQM4Mv2Dataset
from ogb.utils import smiles2graph

def debug_smiles2graph(smiles_string):
    try:
        return smiles2graph(smiles_string)
    except Exception as e:
        print(f"Exception occurred in smiles: {smiles_string}")
        print(e)
        raise e

mol_ds = PygPCQM4Mv2Dataset(root='../data/pcqm4mv2_invariants', smiles2graph=debug_smiles2graph)
```

As you can observe when executing, the previous snippet produces the following output:

```
Processing...
Converting SMILES strings into graphs...
  1%|▏         | 50930/3746620 [00:18<21:40, 2842.28it/s][11:26:20] Explicit valence for atom # 1 Si, 5, is greater than permitted
  1%|▏         | 51128/3746620 [00:18<22:08, 2781.28it/s]
Exception occurred in smiles: O[Si]123O[Si]3(O1)(O2)O
'NoneType' object has no attribute 'GetAtoms'
```

If we now try to convert this smiles using RDKit without the ogb package:

```
from rdkit import Chem
problematic_smiles = 'O[Si]123O[Si]3(O1)(O2)O'
mol_generated = Chem.MolFromSmiles(problematic_smiles)
print(mol_generated is None)
```

we get a `True` output, making the smiles string invalid.
Repository: snap-stanford/ogb
Issue: full-batch vs sampling on products dataset

Hi, 
For the full batch training (gnn.py), I notice that _products_ dataset is used with **T.ToSparseTensor()**. In contrast, the sampling-based techniques (ClusterGCN and GraphSaint) do not involve converting _products_ to sparse tensor format. Could you please help me understand this difference? Thank you!

The `SparseTensor` format is a newly-released feature in PyG 1.6.0, which introduces memory-efficient aggregations for operating on large graphs (see [here](https://pytorch-geometric.readthedocs.io/en/latest/notes/sparse_tensor.html) for the tutorial). Since sampling-based techniques only operate on a small subgraph, the risk of running out-of-memory is rather low. However, you can also convert your data to a `SparseTensor` for the sampling-based approaches, if that is more convenient to you.

Thank you!
Repository: snap-stanford/ogb
Issue: PytorchStreamReader Problem when loading paper_to_paper_symmetric.pt from mag240m with torch.load
Root cause: Environment Incompatibility

I came into a problem when repeating ogb/examples/lsc/mag240m/gnn.py. There is 'PytorchStreamReader failed reading zip archive: failed finding central directory' when loading  loading paper_to_paper_symmetric.pt from mag240m with torch.load. I tried various versions of pytorch but the same error came out. 

I've found that disk space is limited when preprocessing. lol
Repository: snap-stanford/ogb
Issue: Feature Request: Option to Skip Download & User Interaction for Automated/SLURM Environments
Root cause: Unimplemented Feature Gap

Hi,

I’m working with OBG in an environment where jobs are submitted and executed non-interactively,specifically in SLURM-based HPC clusters and other automated pipelines. In these scenarios, user prompts (e.g., for downloading data or requiring manual confirmation) can interrupt the execution flow and prevent successful runs.

### Feature Suggestions

It would be helpful if OBG supported a flag or configuration option to:

- Skip user interaction entirely, including prompts for data download.
- Automatically proceed with defaults or raise a clear exception, if required resources are missing.
- Optionally pre-check requirements without initiating downloads (useful for staging jobs).

This would allow OBG to be better integrated into automated pipelines.


### Proposed Options

- A global flag like --no-interaction or --headless
- An environment variable
- Programmatic access via a function parameter or config object

Thanks for your work, happy to provide a PR or further feedback if this seems feasible.

Best, 
Lorenzo
Repository: snap-stanford/ogb
Issue: Problem with OGB papers download
Root cause: External Dependency Breakage

Hello, I am facing an issue to download the  OGB-papers dataset since it always stucks at 50Gb, I have checked some previous issues and I can see that this issue was closed. However I can see that the problem still persists and clearly it is not a storage issue but rather a network issue related to ogb servers.
PS. I am using the latest version of ogb 1.3.6

python3 examples/ogbn_train.py --dataset ogbn-papers100M
/usr/local/lib/python3.12/dist-packages/outdated/__init__.py:36: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import parse_version
Training ogbn-papers100M with sgformer model.
The root is:  ./data/ogbn-papers100M
This will download 56.17GB. Will you proceed? (y/N)
y
Downloading http://snap.stanford.edu/ogb/data/nodeproppred/papers100M-bin.zip
Downloaded 56.17 GB: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 57519/57519 [20:04<00:00, 47.74it/s]
Extracting ./data/ogbn-papers100M/papers100M-bin.zip
Processing...
Loading necessary files...
This might take a while.
Processing graphs...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 14563.56it/s]
Converting graphs into PyG objects...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1472.20it/s]
Saving...
Done!
Repository: snap-stanford/ogb
Issue: Naming inconsistensies

"ogbg-mol-tox21" should read "ogbg-moltox21" in this readme.

https://github.com/snap-stanford/ogb/tree/003cc8a443fbb8b9ba7518ed8e6787e0347e6ee3/examples/graphproppred/mol#dataset

You are viewing old version. The issue is fixed in the newest README
https://github.com/snap-stanford/ogb/tree/master/examples/graphproppred/mol

Sorry, it was my bad.