Repository: huggingface/text-generation-inference

Issue:
Local Install: No rule to make target 'install-flash-attention-v2'
### System Info

**Release:** v1.3.4, main branch
**Target**: x86_64-unknown-linux-gnu
**Cargo version**: 1.70.0
**Commit sha**: 630800eed37b15c4b0c9eb8e6ab47212026720f7
**Docker label**: N/A
**nvidia-smi**: Wed Jan  3 18:13:50 2024
  
```
Mon Jan  8 15:13:03 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4090        On  | 00000000:01:00.0 Off |                  Off |
|  0%   40C    P8              26W / 450W |      3MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce RTX 4090        On  | 00000000:02:00.0 Off |                  Off |
|  0%   46C    P8              26W / 450W |      3MiB / 24564MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
```

### Information

- [ ] Docker
- [X] The CLI directly

### Tasks

- [X] An officially supported command
- [ ] My own modifications

### Reproduction

Followed steps found within the local install instructions (https://github.com/huggingface/text-generation-inference#local-install), in addition to a couple undocumented make steps 

1. Fork & pull from main branch, v1.3.4
2. Install and test Rust: 1.70.0 (90c541806 2023-05-31)
3. Install & test Conda: 23.11.0
5. Install and test libprotoc: 3.21.12
6. Execute `export BUILD_EXTENSIONS=True`
7. Execute `make install`
8. Execute `cd server && make install-vllm-cuda && cd ..`
9. Update source code for paged attention [as described here](https://github.com/huggingface/text-generation-inference/pull/1386/commits/ad7f8396732c12368c7961e15fe2287f1aa42db3)
10. Execute `cd server && make install-flash-attention && cd ..`
11. Execute `make run-falcon-7b-instruct`, which yields the following message:

_Flash Attention V2 is not installed.
Use the official Docker image (ghcr.io/huggingface/text-generation-inference:latest) or install flash attention v2 with `cd server && make install install-flash-attention-v2`_

12. Execute `cd server && make install install-flash-attention-v2`, which yields the following error message:

_make: *** No rule to make target 'install-flash-attention-v2'.  Stop._

### Expected behavior

Make target should be found and executed. Solution below.

**SOLUTION**: 

The file that issues the warning, `server/text_generation_server/utils/flash_attn.py`, is missing the proper make target suffix. It should be: 

`make install install-flash-attention-v2-cuda` or `install-flash-attention-v2-rocm`


Discussion:

Resolving patch:
diff --git a/server/text_generation_server/utils/flash_attn.py b/server/text_generation_server/utils/flash_attn.py
--- a/server/text_generation_server/utils/flash_attn.py
+++ b/server/text_generation_server/utils/flash_attn.py
@@ -23,10 +23,15 @@
     try:
         import flash_attn_2_cuda
     except ImportError:
+        architecture_suffix = ""
+        if IS_CUDA_SYSTEM:
+            architecture_suffix = "-cuda"
+        elif IS_ROCM_SYSTEM:
+            architecture_suffix = "-rocm"
         raise ImportError(
             "Flash Attention V2 is not installed.\n"
             "Use the official Docker image (ghcr.io/huggingface/text-generation-inference:latest) "
-            "or install flash attention v2 with `cd server && make install install-flash-attention-v2`"
+            f"or install flash attention v2 with `cd server && make install install-flash-attention-v2{architecture_suffix}`"
         )
     if not (is_sm8x or is_sm90):
         raise ImportError(


Test patch: