adbX/reproscreener_manual_evaluations huggingface dataset:
Manual reproducibility evaluation of arXiv paper 2110.02343
Notes: Theory paper
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 4.1.."
Dataset described: False. 
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: False.
Manual reproducibility evaluation of arXiv paper 2110.02474
Notes: No code or data released
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: False. 
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2110.03135
Notes: No code or data released
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. summary, our findings and contributions are: 1) we show that the labeling of adversarial examples in adversarial training practice introduces label noise implicitly; 2)
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. Standard datasets - CIFAR, MNIST, Fashion
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiment setup section
Manual reproducibility evaluation of arXiv paper 2110.05169
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: False
Papers with Code link notes: Yes, same repo but links to the base directory (package) and not to the examples in the paper
Result replication code available: True
Is a package: True
Has wrapper scripts: False ()
Software dependencies stated: True (requirements.txt, setup.py)
Code language: python
Likely to reproduce: True (Partially)
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: False. 
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. EXPERIMENTS section
Manual reproducibility evaluation of arXiv paper 2110.08255
Notes: Good! Has main script to get data and run experiments
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Official Code)
Result replication code available: True
Has wrapper scripts: True (main_yformer.py)
Software dependencies stated: True (requirements.txt)
Code language: python
Likely to reproduce: True ()
Problem stated: True. Problem Formulation section
Objective stated: False. 
Research method stated: True. Methodology section
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: True. three real-world public datasets. ETTh1 and ETTh2 (Electricity Transformer Temperature3): These realworld datasets for the electric power deployment introduced by [35] combine
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2110.08432
Notes: Their code and data not released, only links to methods they compared
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: True. To address this problem, we propose Adjoint MAML (A-MAML).
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: True. Third, we examined our approach in three real-world applications of collaborative filtering. To this end, we used the following datasets. (1) Jester-1(https://goldberg.berkeley.edu/ jester-data/) (Goldberg et al., 2001), which are about joke ratings
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2110.09902
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Community Code)
Result replication code available: True
Has wrapper scripts: True (MAKEFILE)
Software dependencies stated: False (Listed in readme)
Code language: python and C++
Likely to reproduce: True ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. In summary, the main contribution of this paper include:  We proposed two methods to infer the proxy kernels.
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. MNIST
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: False.
Manual reproducibility evaluation of arXiv paper 2110.11688
Code available in article: True
Papers with Code link available: False
Papers with Code link matches: False
Papers with Code link notes: No
Result replication code available: True
Has wrapper scripts: True (experiments.py)
Software dependencies stated: True (CMakeLists.txt)
Code language: Python and C++
Likely to reproduce: True ()
Problem stated: True. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: True. In this section, we assess the practical performance of DP-CD against (proximal) DP-SGD on LASSO4 and `2-regularized logistic regression5. On the latter problem, we also consider the dual private coordinate descent algorithm of Damaskinos et al. (2021) (DP-SCD). For LASSO, we use the California dataset (Kelley Pace and Barry, 1997), with n = 20; 640 records and p = 8 features as well as a synthetic dataset
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Numerical Experiments section
Manual reproducibility evaluation of arXiv paper 2110.14241
Notes: Their code and data not released, only links to repos of references they used
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: True. Since our objective is for the agents to learn robust protocols that
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: True. compute the referential accuracy using images from the Flickr8k dataset [17]
Hypothesis stated: True. We hypothesize that a potential bottleneck of this and related approaches lies in the use of static populations
Prediction stated: False
Experiment setup described: True. Experiments and Results section
Manual reproducibility evaluation of arXiv paper 2111.02997
Notes: Code for the specific paper is in a different branch. Linked repo is contains all of his research
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Official Code)
Result replication code available: True
Is a package: False
Has wrapper scripts: True (examples.py, Dockerfile)
Software dependencies stated: True (requirements.txt)
Code language: python
Likely to reproduce: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: True. In particular, we are interested in the following three questions
Pseudocode present: False. 
Dataset described: False. 
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2111.03664
Notes: Their code and data not released, only GH links to methods they used
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. As training datasets, we used the two most popular datasets: MJSynth [56] and SynthText [57].
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. EXPERIMENTAL SETTINGS section
Manual reproducibility evaluation of arXiv paper 2111.08356
Notes: Their code and data not released, only links to methods implemented
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: True. We propose a novel approach to this problem,
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. Called "Algorithm 1, 2.."
Dataset described: True. Standard datasets - CIFAR
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2111.12673
Notes: Good! Scripts to reproduce experiments with seeds
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Official Code)
Result replication code available: True
Has wrapper scripts: True (run_experiment.sh)
Software dependencies stated: False (pip install command with versions in readme pip install gym==0.17.2 mujoco-py==1.50.1.68 numpy==1.19.1 torch==1.6.0 torchvision==0.7.0)
Code language: python
Likely to reproduce: True ()
Problem stated: True. To overcome these problems we propose a principled
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. PSEUDOCODE section
Dataset described: False. 
Hypothesis stated: True. we hypothesize there
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2111.15449
Notes: Readme in chinese, no instructions. Seems easy to reproduce though
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Official Code)
Result replication code available: True
Has wrapper scripts: True (train.py)
Software dependencies stated: False ()
Code language: python
Likely to reproduce: True ()
Problem stated: False. 
Objective stated: False. 
Research method stated: True. Method section
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. Standard datasets - CIFAR, ImageNet
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments and Results section
Manual reproducibility evaluation of arXiv paper 2112.00016
Notes: Should be very easy to reproduce
Code available in article: True
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo (Official Code)
Result replication code available: True
Has wrapper scripts: True (learning.py)
Software dependencies stated: False ()
Code language: python
Likely to reproduce: True ()
Problem stated: False. 
Objective stated: False. The purpose of
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. In zip file inside repo
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Experiments section
Manual reproducibility evaluation of arXiv paper 2112.03432
Notes: No code or data released
Code available in article: False
Papers with Code link available: False
Papers with Code link matches: False
Has wrapper scripts: False ()
Software dependencies stated: False ()
Problem stated: False. 
Objective stated: False. 
Research method stated: False. 
Research questions stated: False. 
Pseudocode present: True. 
Dataset described: False. 
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: False.
Manual reproducibility evaluation of arXiv paper 2112.04871
Notes: Commands to reproduce experiments in readme.
Code available in article: False
Papers with Code link available: True
Papers with Code link matches: True
Papers with Code link notes: Yes, same repo but also has another link to an older version of their model (Official Code)
Result replication code available: True
Has wrapper scripts: True (learn.py, process_datasets.py)
Software dependencies stated: False (Listed in readme)
Code language: python
Likely to reproduce: True ()
Problem stated: True. To address this problem, we propose
Objective stated: False. 
Research method stated: True. Our Method section
Research questions stated: False. 
Pseudocode present: False. 
Dataset described: True. Inside repo
Hypothesis stated: False. 
Prediction stated: False
Experiment setup described: True. Exeprimental Setting section (mispelled)