Hugging Face's logo
Technology Innovation Institute's profile picture

Technology Innovation Institute

company
https://www.tii.ae/
TIIuae
tiiuae

AI & ML interests

Large language models

Organization Card
About org cards

Do you believe in a better tomorrow? We do. Our team of expert researchers live the dream and work to build it every day.

🔥 Falcon-180B is now available in open-access! Try it now in our chat demo!

News

Falcon LLM

Falcon LLM is TII's flagship series of large language models, built from scratch using a custom data pipeline and distributed training library. Papers coming soon 😊.

To promote collaborations and drive innovation, we have open-sourced a number of artefacts:

  • The Falcon-180B pretrained and chat models, under the Falcon-180B TII license. Falcon-180B is the largest and most powerful open-access model available.
  • The Falcon-7/40B pretrained and instruct models, under the Apache 2.0 software license . Falcon-7B/40B models are state-of-the-art for their size, outperforming other open-source models on NLP benchmarks.
  • The RefinedWeb dataset, a massive web dataset with stringent filtering and large-scale deduplication, enabling models trained on web data alone to match or outperform models trained on curated corpora. See 📓 the paper for more information. RefinedWeb is licensed under ODC-By 1.0.

See below for a detailed list of artefacts in the Falcon LLM family:

Artefact Link Type Details
🥇 Falcon-180B Here pretrained model 180B parameters trained on 3,500 billion tokens.
Falcon-180B-Chat Here chat model Falcon-180B finetuned on a mixture of Ultrachat, Platypus and Airoboros.
🥈 Falcon-40B Here pretrained model 40B parameters trained on 1,000 billion tokens.
Falcon-40B-Instruct Here instruction/chat model Falcon-40B finetuned on the Baize dataset.
🥉 Falcon-7B Here pretrained model 6.7B parameters trained on 1,500 billion tokens.
Falcon-7B-Instruct Here instruction/chat model Falcon-7B finetuned on the Baize, GPT4All, and GPTeacher datasets.
📀 RefinedWeb Here pretraining web dataset ~600 billion "high-quality" tokens.
Falcon-RW-1B Here pretrained model 1.3B parameters trained on 350 billion tokens.
Falcon-RW-7B Here pretrained model 7.5B parameters trained on 350 billion tokens.

About us

The Technology Innovation Institute (TII) is a leading global research center dedicated to pushing the frontiers of knowledge. Our teams of scientists, researchers and engineers work in an open, flexible and agile environment to deliver discovery science and transformative technologies. Our work means we will not only prepare for the future; we will create it. Working together, we are committed to inspiring innovation for a better tomorrow.

We are part of Abu Dhabi Government’s Advanced Technology Research Council, which oversees technology research in the emirate. As a disruptor in science, we are setting new standards and serve as a catalyst for change.

Faced with a future of limitless possibilities and supported by strategically funded investments, we are encouraging a culture of discovery. Our work reinforces Abu Dhabi and the UAE’s status as an R&D hub and a global leader in breakthrough technologies.

Collections 2

spaces 2

models 8