logs/l40-4gpu-8.txt

Submitting job: /common/home/users/d/dh.huang.2023/code/llm-qa-eval/scripts/tune_rp.sh
Current Directory: /common/home/users/d/dh.huang.2023/code/llm-qa-eval
Sun Aug 25 16:31:23 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA L40                     On  |   00000000:01:00.0 Off |                    0 |
| N/A   38C    P8             36W /  300W |       1MiB /  46068MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
Linux lagoon 4.18.0-553.5.1.el8_10.x86_64 #1 SMP Thu Jun 6 09:41:19 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
NAME="Rocky Linux"
VERSION="8.10 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.10"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.10 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.10"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.10"
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              128
On-line CPU(s) list: 0-127
Thread(s) per core:  2
Core(s) per socket:  64
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          25
Model:               1
Model name:          AMD EPYC 7763 64-Core Processor
Stepping:            1
CPU MHz:             2450.000
CPU max MHz:         3529.0520
CPU min MHz:         1500.0000
BogoMIPS:            4891.15
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            32768K
NUMA node0 CPU(s):   0-127
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin brs arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm
MemTotal:       527669148 kB
Current Directory: /common/home/users/d/dh.huang.2023/code/llm-qa-eval
QA_WITH_MS_MACRO=
RESULT_FILENAME_PREFIX_BASE=Phi-3.5-mini-instruct_wd
Testing microsoft/Phi-3.5-mini-instruct with ./data/datasets/WebQSP.test.wikidata.json
RESULT_FILENAME_PREFIX=Phi-3.5-mini-instruct_wd_true
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
A new version of the following files was downloaded from https://huggingface.co/microsoft/Phi-3.5-mini-instruct:
- configuration_phi3.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/microsoft/Phi-3.5-mini-instruct:
- modeling_phi3.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
2024-08-25 16:32:14,372 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 16:32:14,372 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:15<00:15, 15.44s/it]
Downloading shards: 100%|██████████| 2/2 [00:23<00:00, 11.28s/it]
Downloading shards: 100%|██████████| 2/2 [00:23<00:00, 11.91s/it]

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:26<00:26, 26.33s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:40<00:00, 18.91s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:40<00:00, 20.02s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 16:33:20,419 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 17:24:42,292 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 17:24:42,293 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.79s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.92s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 18.95s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 17:25:20,903 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 18:18:32,005 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 18:18:32,005 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.34s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.67s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 18.67s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 18:19:10,173 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 19:14:22,403 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 19:14:22,403 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:25<00:25, 25.10s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 18.81s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:39<00:00, 19.75s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 19:15:02,620 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 20:13:44,756 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 20:13:44,756 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.71s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.88s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 18.90s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 20:14:23,253 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 21:17:23,392 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 21:17:23,392 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.83s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.97s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 19.00s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 21:18:02,190 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-25 22:37:47,514 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-25 22:37:47,514 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.55s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.88s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 18.88s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-25 22:38:26,016 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-26 00:55:39,714 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-26 00:55:39,714 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:25<00:25, 25.45s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:38<00:00, 18.41s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:38<00:00, 19.47s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-26 00:56:19,334 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-26 04:39:03,923 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-26 04:39:03,923 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:24<00:24, 24.57s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 17.82s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:37<00:00, 18.83s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-26 04:39:42,466 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-26 09:24:10,693 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-26 09:24:10,693 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:25<00:25, 25.51s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:38<00:00, 18.14s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:38<00:00, 19.25s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-26 09:24:50,135 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-26 14:30:12,616 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-26 14:30:12,616 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:23<00:23, 23.59s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:36<00:00, 17.02s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:36<00:00, 18.00s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-26 14:30:49,269 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
[nltk_data] Downloading package punkt to
[nltk_data]     /common/home/users/d/dh.huang.2023/nltk_data...
[nltk_data]   Package punkt is already up-to-date!
2024-08-26 19:44:35,757 [WARNING] [modeling_phi3.py:62] `flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.
2024-08-26 19:44:35,757 [WARNING] [modeling_phi3.py:66] Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:23<00:23, 23.03s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:35<00:00, 16.64s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:35<00:00, 17.59s/it]
The `seen_tokens` attribute is deprecated and will be removed in v4.41. Use the `cache_position` model input instead.
2024-08-26 19:45:11,738 [WARNING] [logging.py:328] You are not running the flash-attention implementation, expect numerical differences.
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset