Spaces:

zero-gpu-explorers

/

README

like 30

Running

@djstrong why do you have https://huggingface.co/spaces/speakleash/Bielik-7B-Instruct-v0.1/blob/main/app.py#L132 in your code in the GPU function? Try removing any stateful operation that doesn't require the GPU outside the function that is decorated with @spaces.GPU

@codelion Thank you! How to escape from the @spaces.GPU function? Run it in a thread or it still will be inside GPU scope? We need to save answers from the model.

So, the main function predictinvoke without @spaces.GPU and inside call model with @spaces.GPU.

@spaces.GPU # the only function with the decorator
def generate_response():
  ...

def predict(message, history, system_prompt, temperature, max_new_tokens, top_k, repetition_penalty, top_p):
  prepare_data()
  yield from generate_response()
  save_results()

@codelion Thank you! I have optimized the code.

@djstrong May I check what was at L132 that caused the problem? I saw the print statement and wonder if we cannot print anything in the GPU function.

No you can print stuff it will be routed to the logs

@djstrong May I check what was at L132 that caused the problem? I saw the print statement and wonder if we cannot print anything in the GPU function.

There was sending logs to Discord and a repository. However, I don't think it was causing the main problem and @codelion comment was "by the way".