tiiuae

/

falcon-40b

like 2.4k

I would like to do some research and apply a single prompt to each entry in a text column in a dataset and then collect the results into a new column.

I played with the 7B model in Databricks and entered a list of 80 prompts as my first test. It took a few hours to finish. I'm thinking that running the 40B model on a larger cluster is only going to get slower and slower.

What is the recommended optimal procedure here? What kind of time/results can I expect in the best case?