sander-wood commited on
Commit
d75a880
1 Parent(s): 65fe965

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -9,6 +9,8 @@ bGPT supports generative modelling via next byte prediction on any type of data
9
 
10
  You can check out the [demo page](https://byte-gpt.github.io/), which includes examples generated by the bGPT model.
11
 
 
 
12
  ## Model Description
13
  Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. Inspired by the success of next token prediction in natural language processing, we introduce bGPT, a model with next byte prediction to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new possibilities for predicting, simulating, and diagnosing algorithm or hardware behaviour. It has almost flawlessly replicated the process of converting symbolic music data, achieving a low error rate of 0.0011 bits per byte in converting ABC notation to MIDI format. In addition, bGPT demonstrates exceptional capabilities in simulating CPU behaviour, with an accuracy exceeding 99.99% in executing various operations. Leveraging next byte prediction, models like bGPT can directly learn from vast binary data, effectively simulating the intricate patterns of the digital world.
14
 
@@ -22,8 +24,6 @@ We provide five weights of bGPT on [Hugging Face](https://huggingface.co/sander-
22
 
23
  The core components of bGPT include a 12-layer patch-level decoder, a 3-layer byte-level decoder, with a hidden size of 768, totaling 110 million parameters.
24
 
25
- The code for bGPT is available on [GitHub ](https://github.com/sanderwood/bgpt).
26
-
27
  ## Installation
28
 
29
  To set up the bGPT environment and install the necessary dependencies, follow these steps:
 
9
 
10
  You can check out the [demo page](https://byte-gpt.github.io/), which includes examples generated by the bGPT model.
11
 
12
+ The code for bGPT is available on [GitHub ](https://github.com/sanderwood/bgpt).
13
+
14
  ## Model Description
15
  Traditional deep learning often overlooks bytes, the basic units of the digital world, where all forms of information and operations are encoded and manipulated in binary format. Inspired by the success of next token prediction in natural language processing, we introduce bGPT, a model with next byte prediction to simulate the digital world. bGPT matches specialized models in performance across various modalities, including text, audio, and images, and offers new possibilities for predicting, simulating, and diagnosing algorithm or hardware behaviour. It has almost flawlessly replicated the process of converting symbolic music data, achieving a low error rate of 0.0011 bits per byte in converting ABC notation to MIDI format. In addition, bGPT demonstrates exceptional capabilities in simulating CPU behaviour, with an accuracy exceeding 99.99% in executing various operations. Leveraging next byte prediction, models like bGPT can directly learn from vast binary data, effectively simulating the intricate patterns of the digital world.
16
 
 
24
 
25
  The core components of bGPT include a 12-layer patch-level decoder, a 3-layer byte-level decoder, with a hidden size of 768, totaling 110 million parameters.
26
 
 
 
27
  ## Installation
28
 
29
  To set up the bGPT environment and install the necessary dependencies, follow these steps: