Orionnnn/hub-stats huggingface dataset:
Robust Distortion-free Watermarks for Language Models
Authors: name: Rohith Kuditipudi, name: John Thickstun, name: Tatsunori Hashimoto, name: Percy Liang
Published: 2023-07-28 14:52:08

We propose a methodology for planting watermarks in text from an
autoregressive language model that are robust to perturbations without changing
the distribution over text up to a certain maximum generation budget. We
generate watermarked text by mapping a sequence of random numbers -- which we
compute using a randomized watermark key -- to a sample from the language
model. To detect watermarked text, any party who knows the key can align the
text to the random number sequence. We instantiate our watermark methodology
with two sampling schemes: inverse transform sampling and exponential minimum
sampling. We apply these watermarks to three language models -- OPT-1.3B,
LLaMA-7B and Alpaca-7B -- to experimentally validate their statistical power
and robustness to various paraphrasing attacks. Notably, for both the OPT-1.3B
and LLaMA-7B models, we find we can reliably detect watermarked text (p leq
0.01) from 35 tokens even after corrupting between 40-50\% of the tokens
via random edits (i.e., substitutions, insertions or deletions). For the
Alpaca-7B model, we conduct a case study on the feasibility of watermarking
responses to typical user instructions. Due to the lower entropy of the
responses, detection is more difficult: around 25% of the responses -- whose
median length is around 100 tokens -- are detectable with p leq 0.01, and
the watermark is also less robust to certain automated paraphrasing attacks we
implement.
Exploring Format Consistency for Instruction Tuning
Authors: name: Shihao Liang, name: Kunlun Zhu, name: Runchu Tian, name: Yujia Qin, name: Huadong Wang, name: Xin Cong, name: Zhiyuan Liu, name: Xiaojiang Liu, name: Maosong Sun
Published: 2023-07-28 12:00:13

Instruction tuning has emerged as a promising approach to enhancing large
language models in following human instructions. It is shown that increasing
the diversity and number of instructions in the training data can consistently
enhance generalization performance, which facilitates a recent endeavor to
collect various instructions and integrate existing instruction tuning datasets
into larger collections. However, different users have their unique ways of
expressing instructions, and there often exist variations across different
datasets in the instruction styles and formats, i.e., format inconsistency. In
this work, we study how format inconsistency may impact the performance of
instruction tuning. We propose a framework called "Unified Instruction Tuning"
(UIT), which calls OpenAI APIs for automatic format transfer among different
instruction tuning datasets. We show that UIT successfully improves the
generalization performance on unseen instructions, which highlights the
importance of format consistency for instruction tuning. To make the UIT
framework more practical, we further propose a novel perplexity-based denoising
method to reduce the noise of automatic format transfer. We also train a
smaller offline model that achieves comparable format transfer capability than
OpenAI APIs to reduce costs in practice.
Open Problems and Fundamental Limitations of Reinforcement Learning from
  Human Feedback
Authors: name: Stephen Casper, name: Xander Davies, name: Claudia Shi, name: Thomas Krendl Gilbert, name: Jérémy Scheurer, name: Javier Rando, name: Rachel Freedman, name: Tomasz Korbak, name: David Lindner, name: Pedro Freire, name: Tony Wang, name: Samuel Marks, name: Charbel-Raphaël Segerie, name: Micah Carroll, name: Andi Peng, name: Phillip Christoffersen, name: Mehul Damani, name: Stewart Slocum, name: Usman Anwar, name: Anand Siththaranjan, name: Max Nadeau, name: Eric J. Michaud, name: Jacob Pfau, name: Dmitrii Krasheninnikov, name: Xin Chen, name: Lauro Langosco, name: Peter Hase, name: Erdem Bıyık, name: Anca Dragan, name: David Krueger, name: Dorsa Sadigh, name: Dylan Hadfield-Menell
Published: 2023-07-27 22:29:25

Reinforcement learning from human feedback (RLHF) is a technique for training
AI systems to align with human goals. RLHF has emerged as the central method
used to finetune state-of-the-art large language models (LLMs). Despite this
popularity, there has been relatively little public work systematizing its
flaws. In this paper, we (1) survey open problems and fundamental limitations
of RLHF and related methods; (2) overview techniques to understand, improve,
and complement RLHF in practice; and (3) propose auditing and disclosure
standards to improve societal oversight of RLHF systems. Our work emphasizes
the limitations of RLHF and highlights the importance of a multi-faceted
approach to the development of safer AI systems.
Med-Flamingo: a Multimodal Medical Few-shot Learner
Authors: name: Michael Moor, name: Qian Huang, name: Shirley Wu, name: Michihiro Yasunaga, name: Cyril Zakka, name: Yash Dalmia, name: Eduardo Pontes Reis, name: Pranav Rajpurkar, name: Jure Leskovec
Published: 2023-07-27 20:36:02

Medicine, by its nature, is a multifaceted domain that requires the synthesis
of information across various modalities. Medical generative vision-language
models (VLMs) make a first step in this direction and promise many exciting
clinical applications. However, existing models typically have to be fine-tuned
on sizeable down-stream datasets, which poses a significant limitation as in
many medical applications data is scarce, necessitating models that are capable
of learning from few examples in real-time. Here we propose Med-Flamingo, a
multimodal few-shot learner adapted to the medical domain. Based on
OpenFlamingo-9B, we continue pre-training on paired and interleaved medical
image-text data from publications and textbooks. Med-Flamingo unlocks few-shot
generative medical visual question answering (VQA) abilities, which we evaluate
on several datasets including a novel challenging open-ended VQA dataset of
visual USMLE-style problems. Furthermore, we conduct the first human evaluation
for generative medical VQA where physicians review the problems and blinded
generations in an interactive app. Med-Flamingo improves performance in
generative medical VQA by up to 20\% in clinician's rating and firstly enables
multimodal medical few-shot adaptations, such as rationale generation. We
release our model, code, and evaluation app under
https://github.com/snap-stanford/med-flamingo.
To Adapt or Not to Adapt? Real-Time Adaptation for Semantic Segmentation
Authors: name: Marc Botet Colomer, name: Pier Luigi Dovesi, name: Theodoros Panagiotakopoulos, name: Joao Frederico Carvalho, name: Linus Härenstam-Nielsen, name: Hossein Azizpour, name: Hedvig Kjellström, name: Daniel Cremers, name: Matteo Poggi
Published: 2023-07-27 17:59:59

The goal of Online Domain Adaptation for semantic segmentation is to handle
unforeseeable domain changes that occur during deployment, like sudden weather
events. However, the high computational costs associated with brute-force
adaptation make this paradigm unfeasible for real-world applications. In this
paper we propose HAMLET, a Hardware-Aware Modular Least Expensive Training
framework for real-time domain adaptation. Our approach includes a
hardware-aware back-propagation orchestration agent (HAMT) and a dedicated
domain-shift detector that enables active control over when and how the model
is adapted (LT). Thanks to these advancements, our approach is capable of
performing semantic segmentation while simultaneously adapting at more than
29FPS on a single consumer-grade GPU. Our framework's encouraging accuracy and
speed trade-off is demonstrated on OnDA and SHIFT benchmarks through
experimental results.
How to Scale Your EMA
Authors: name: Dan Busbridge, name: Jason Ramapuram, name: Pierre Ablin, name: Tatiana Likhomanenko, name: Eeshan Gunesh Dhekane, name: Xavier Suau, name: Russ Webb
Published: 2023-07-25 20:33:48

Preserving training dynamics across batch sizes is an important tool for
practical machine learning as it enables the trade-off between batch size and
wall-clock time. This trade-off is typically enabled by a scaling rule, for
example, in stochastic gradient descent, one should scale the learning rate
linearly with the batch size. Another important tool for practical machine
learning is the model Exponential Moving Average (EMA), which is a model copy
that does not receive gradient information, but instead follows its target
model with some momentum. This model EMA can improve the robustness and
generalization properties of supervised learning, stabilize pseudo-labeling,
and provide a learning signal for Self-Supervised Learning (SSL). Prior works
have treated the model EMA separately from optimization, leading to different
training dynamics across batch sizes and lower model performance. In this work,
we provide a scaling rule for optimization in the presence of model EMAs and
demonstrate its validity across a range of architectures, optimizers, and data
modalities. We also show the rule's validity where the model EMA contributes to
the optimization of the target model, enabling us to train EMA-based
pseudo-labeling and SSL methods at small and large batch sizes. For SSL, we
enable training of BYOL up to batch size 24,576 without sacrificing
performance, optimally a 6times wall-clock time reduction.
NeRF-Det: Learning Geometry-Aware Volumetric Representation for
  Multi-View 3D Object Detection
Authors: name: Chenfeng Xu, name: Bichen Wu, name: Ji Hou, name: Sam Tsai, name: Ruilong Li, name: Jialiang Wang, name: Wei Zhan, name: Zijian He, name: Peter Vajda, name: Kurt Keutzer, name: Masayoshi Tomizuka
Published: 2023-07-27 04:36:16

We present NeRF-Det, a novel method for indoor 3D detection with posed RGB
images as input. Unlike existing indoor 3D detection methods that struggle to
model scene geometry, our method makes novel use of NeRF in an end-to-end
manner to explicitly estimate 3D geometry, thereby improving 3D detection
performance. Specifically, to avoid the significant extra latency associated
with per-scene optimization of NeRF, we introduce sufficient geometry priors to
enhance the generalizability of NeRF-MLP. Furthermore, we subtly connect the
detection and NeRF branches through a shared MLP, enabling an efficient
adaptation of NeRF to detection and yielding geometry-aware volumetric
representations for 3D detection. Our method outperforms state-of-the-arts by
3.9 mAP and 3.1 mAP on the ScanNet and ARKITScenes benchmarks, respectively. We
provide extensive analysis to shed light on how NeRF-Det works. As a result of
our joint-training design, NeRF-Det is able to generalize well to unseen scenes
for object detection, view synthesis, and depth estimation tasks without
requiring per-scene optimization. Code is available at
https://github.com/facebookresearch/NeRF-Det.
MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
Authors: name: Reiner Birkl, name: Diana Wofk, name: Matthias Müller
Published: 2023-07-26 19:01:49

We release MiDaS v3.1 for monocular depth estimation, offering a variety of
new models based on different encoder backbones. This release is motivated by
the success of transformers in computer vision, with a large variety of
pretrained vision transformers now available. We explore how using the most
promising vision transformers as image encoders impacts depth estimation
quality and runtime of the MiDaS architecture. Our investigation also includes
recent convolutional approaches that achieve comparable quality to vision
transformers in image classification tasks. While the previous release MiDaS
v3.0 solely leverages the vanilla vision transformer ViT, MiDaS v3.1 offers
additional models based on BEiT, Swin, SwinV2, Next-ViT and LeViT. These models
offer different performance-runtime tradeoffs. The best model improves the
depth estimation quality by 28% while efficient models enable downstream tasks
requiring high frame rates. We also describe the general process for
integrating new backbones. A video summarizing the work can be found at
https://youtu.be/UjaeNNFf9sE and the code is available at
https://github.com/isl-org/MiDaS.
TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis
Authors: name: Zihan Zhang, name: Richard Liu, name: Kfir Aberman, name: Rana Hanocka
Published: 2023-07-27 17:48:44

The gradual nature of a diffusion process that synthesizes samples in small
increments constitutes a key ingredient of Denoising Diffusion Probabilistic
Models (DDPM), which have presented unprecedented quality in image synthesis
and been recently explored in the motion domain. In this work, we propose to
adapt the gradual diffusion concept (operating along a diffusion time-axis)
into the temporal-axis of the motion sequence. Our key idea is to extend the
DDPM framework to support temporally varying denoising, thereby entangling the
two axes. Using our special formulation, we iteratively denoise a motion buffer
that contains a set of increasingly-noised poses, which auto-regressively
produces an arbitrarily long stream of frames. With a stationary diffusion
time-axis, in each diffusion step we increment only the temporal-axis of the
motion such that the framework produces a new, clean frame which is removed
from the beginning of the buffer, followed by a newly drawn noise vector that
is appended to it. This new mechanism paves the way towards a new framework for
long-term motion synthesis with applications to character animation and other
domains.
Scaling TransNormer to 175 Billion Parameters
Authors: name: Zhen Qin, name: Dong Li, name: Weigao Sun, name: Weixuan Sun, name: Xuyang Shen, name: Xiaodong Han, name: Yunshen Wei, name: Baohong Lv, name: Fei Yuan, name: Xiao Luo, name: Yu Qiao, name: Yiran Zhong
Published: 2023-07-27 16:45:33

We present TransNormerLLM, the first linear attention-based Large Language
Model (LLM) that outperforms conventional softmax attention-based models in
terms of both accuracy and efficiency. TransNormerLLM evolves from the previous
linear attention architecture TransNormer by making advanced modifications that
include positional embedding, linear attention acceleration, gating mechanism,
tensor normalization, inference acceleration and stabilization. Specifically,
we use LRPE together with an exponential decay to avoid attention dilution
issues while allowing the model to retain global interactions between tokens.
Additionally, we propose Lightning Attention, a cutting-edge technique that
accelerates linear attention by more than twice in runtime and reduces memory
usage by a remarkable four times. To further enhance the performance of
TransNormer, we leverage a gating mechanism to smooth training and a new tensor
normalization scheme to accelerate the model, resulting in an impressive
acceleration of over 20%. Furthermore, we have developed a robust inference
algorithm that ensures numerical stability and consistent inference speed,
regardless of the sequence length, showcasing superior efficiency during both
training and inference stages. Scalability is at the heart of our model's
design, enabling seamless deployment on large-scale clusters and facilitating
expansion to even more extensive models, all while maintaining outstanding
performance metrics. Rigorous validation of our model design is achieved
through a series of comprehensive experiments on our self-collected corpus,
boasting a size exceeding 6TB and containing over 2 trillion tokens. To ensure
data quality and relevance, we implement a new self-cleaning strategy to filter
our collected data. Our pre-trained models will be released to foster community
advancements in efficient LLMs.
PanGu-Coder2: Boosting Large Language Models for Code with Ranking
  Feedback
Authors: name: Bo Shen, name: Jiaxin Zhang, name: Taihong Chen, name: Daoguang Zan, name: Bing Geng, name: An Fu, name: Muhan Zeng, name: Ailun Yu, name: Jichuan Ji, name: Jingyang Zhao, name: Yuenan Guo, name: Qianxiang Wang
Published: 2023-07-27 15:28:29

Large Language Models for Code (Code LLM) are flourishing. New and powerful
models are released on a weekly basis, demonstrating remarkable performance on
the code generation task. Various approaches have been proposed to boost the
code generation performance of pre-trained Code LLMs, such as supervised
fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we
propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework,
which can effectively and efficiently boost pre-trained large language models
for code generation. Under this framework, we present PanGu-Coder2, which
achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through
an extensive evaluation on CoderEval and LeetCode benchmarks, we show that
PanGu-Coder2 consistently outperforms all previous Code LLMs.
Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
Authors: name: Huy Ha, name: Pete Florence, name: Shuran Song
Published: 2023-07-26 22:52:43

We present a framework for robot skill acquisition, which 1) efficiently
scale up data generation of language-labelled robot data and 2) effectively
distills this data down into a robust multi-task language-conditioned
visuo-motor policy. For (1), we use a large language model (LLM) to guide
high-level planning, and sampling-based robot planners (e.g. motion or grasp
samplers) for generating diverse and rich manipulation trajectories. To
robustify this data-collection process, the LLM also infers a code-snippet for
the success condition of each task, simultaneously enabling the data-collection
process to detect failure and retry as well as the automatic labeling of
trajectories with success/failure. For (2), we extend the diffusion policy
single-task behavior-cloning approach to multi-task settings with language
conditioning. Finally, we propose a new multi-task benchmark with 18 tasks
across five domains to test long-horizon behavior, common-sense reasoning,
tool-use, and intuitive physics. We find that our distilled policy successfully
learned the robust retrying behavior in its data collection policy, while
improving absolute success rates by 34.8% on average across five domains. The
benchmark, code, and qualitative results are on our website
https://www.cs.columbia.edu/~huy/scalingup/
Leveraging Implicit Feedback from Deployment Data in Dialogue
Authors: name: Richard Yuanzhe Pang, name: Stephen Roller, name: Kyunghyun Cho, name: He He, name: Jason Weston
Published: 2023-07-26 11:34:53

We study improving social conversational agents by learning from natural
dialogue between users and a deployed model, without extra annotations. To
implicitly measure the quality of a machine-generated utterance, we leverage
signals like user response length, sentiment and reaction of the future human
utterances in the collected dialogue episodes. Our experiments use the publicly
released deployment data from BlenderBot (Xu et al., 2023). Human evaluation
indicates improvements in our new models over baseline responses; however, we
find that some proxy signals can lead to more generations with undesirable
properties as well. For example, optimizing for conversation length can lead to
more controversial or unfriendly generations compared to the baseline, whereas
optimizing for positive sentiment or reaction can decrease these behaviors.
Towards Generalist Biomedical AI
Authors: name: Tao Tu, name: Shekoofeh Azizi, name: Danny Driess, name: Mike Schaekermann, name: Mohamed Amin, name: Pi-Chuan Chang, name: Andrew Carroll, name: Chuck Lau, name: Ryutaro Tanno, name: Ira Ktena, name: Basil Mustafa, name: Aakanksha Chowdhery, name: Yun Liu, name: Simon Kornblith, name: David Fleet, name: Philip Mansfield, name: Sushant Prakash, name: Renee Wong, name: Sunny Virmani, name: Christopher Semturs, name: S Sara Mahdavi, name: Bradley Green, name: Ewa Dominowska, name: Blaise Aguera y Arcas, name: Joelle Barral, name: Dale Webster, name: Greg S. Corrado, name: Yossi Matias, name: Karan Singhal, name: Pete Florence, name: Alan Karthikesalingam, name: Vivek Natarajan
Published: 2023-07-26 17:52:22

Medicine is inherently multimodal, with rich data modalities spanning text,
imaging, genomics, and more. Generalist biomedical artificial intelligence (AI)
systems that flexibly encode, integrate, and interpret this data at scale can
potentially enable impactful applications ranging from scientific discovery to
care delivery. To enable the development of these models, we first curate
MultiMedBench, a new multimodal biomedical benchmark. MultiMedBench encompasses
14 diverse tasks such as medical question answering, mammography and
dermatology image interpretation, radiology report generation and
summarization, and genomic variant calling. We then introduce Med-PaLM
Multimodal (Med-PaLM M), our proof of concept for a generalist biomedical AI
system. Med-PaLM M is a large multimodal generative model that flexibly encodes
and interprets biomedical data including clinical language, imaging, and
genomics with the same set of model weights. Med-PaLM M reaches performance
competitive with or exceeding the state of the art on all MultiMedBench tasks,
often surpassing specialist models by a wide margin. We also report examples of
zero-shot generalization to novel medical concepts and tasks, positive transfer
learning across tasks, and emergent zero-shot medical reasoning. To further
probe the capabilities and limitations of Med-PaLM M, we conduct a radiologist
evaluation of model-generated (and human) chest X-ray reports and observe
encouraging performance across model scales. In a side-by-side ranking on 246
retrospective chest X-rays, clinicians express a pairwise preference for
Med-PaLM M reports over those produced by radiologists in up to 40.50% of
cases, suggesting potential clinical utility. While considerable work is needed
to validate these models in real-world use cases, our results represent a
milestone towards the development of generalist biomedical AI systems.
Tracking Anything in High Quality
Authors: name: Jiawen Zhu, name: Zhenyu Chen, name: Zeqi Hao, name: Shijie Chang, name: Lu Zhang, name: Dong Wang, name: Huchuan Lu, name: Bin Luo, name: Jun-Yan He, name: Jin-Peng Lan, name: Hanyuan Chen, name: Chenyang Li
Published: 2023-07-26 06:19:46

Visual object tracking is a fundamental video task in computer vision.
Recently, the notably increasing power of perception algorithms allows the
unification of single/multiobject and box/mask-based tracking. Among them, the
Segment Anything Model (SAM) attracts much attention. In this report, we
propose HQTrack, a framework for High Quality Tracking anything in videos.
HQTrack mainly consists of a video multi-object segmenter (VMOS) and a mask
refiner (MR). Given the object to be tracked in the initial frame of a video,
VMOS propagates the object masks to the current frame. The mask results at this
stage are not accurate enough since VMOS is trained on several closeset video
object segmentation (VOS) datasets, which has limited ability to generalize to
complex and corner scenes. To further improve the quality of tracking masks, a
pretrained MR model is employed to refine the tracking results. As a compelling
testament to the effectiveness of our paradigm, without employing any tricks
such as test-time data augmentations and model ensemble, HQTrack ranks the 2nd
place in the Visual Object Tracking and Segmentation (VOTS2023) challenge. Code
and models are available at https://github.com/jiawen-zhu/HQTrack.