Experience with LASER
Hi there,
Just wanted to know how you experienced the process of making this.
Was it a big hassle?
How computational intensive is it to apply the LASER Technique?
Do you think it's possible on a consumer grade gpu/ gaming pc? Or does it require renting something like a A100?
Hi @Venkman42 , it wasn't too difficult but it requires a lot of VRAM so I had to use an A100. I tried playing with randomized SVD (https://github.com/mlabonne/laserRMT/tree/main) to lower the VRAM requirements, but the perplexity calculation is still too expensive to run on a T4. It requires more work to work on a consumer-grade GPU but I'm pretty sure it's possible. @fernandofernandes already lowered the bar compared to the original implementation of LASER.
Hi @Venkman42 , it wasn't too difficult but it requires a lot of VRAM so I had to use an A100. I tried playing with randomized SVD (https://github.com/mlabonne/laserRMT/tree/main) to lower the VRAM requirements, but the perplexity calculation is still too expensive to run on a T4. It requires more work to work on a consumer-grade GPU but I'm pretty sure it's possible. @fernandofernandes already lowered the bar compared to the original implementation of LASER.
Thanks for the insight :) looks like I'll have to have some patience and wait for other people to come out with laser versions ^^ would love to see a phi-2 dpo laser version ^^
what is the prompt template for this?