Google Colab for Falcon 40B and 7B with Live Response Streaming
#55
by
gaodrew
- opened
You can run both the models and get the tokens one by one.
Beam Search is also implemented.
Please see the Github repo: https://github.com/andrewgcodes/FalconStreaming
You will need Colab Pro+ using the A100 GPU for 40B.
For 7B, you can likely get away with Colab FREE if you can snag one of the GPUs. Otherwise, you can upgrade to the $9.99/mo options.
I think there is also a new Pay as You Go option.
Qual é a capital da Italia?