Repository: ggml-org/llama.cpp
Pull request: gguf-py : do not align the data start offset

The safetensors format doesn't require alignment. Fixes: #18282 (which was a regression caused by #15667).

I assumed wrong since GGUF does align its data offset, and the writer for safetensors aligns to 8 bytes (see <https://github.com/huggingface/safetensors/blob/806426784adb43631e9a1102d4621126bb589347/safetensors/src/tensor.rs#L256-L258>), and also because the data offset alignment was implemented in the same way in #12820. But apparently some models aren't aligned.

It seems like PyTorch and Numpy *can* handle unaligned tensors, but I'm not completely sure (is it only for shape transformations, or does it also support arithmetic on unaligned tensors? (would need an unaligned model which has some arithmetic in its `modify_tensors` transformations to test this)). Copying the tensor (with e.g. `data.copy()`) wouldn't necessarily always be sufficient, because that doesn't seem to align to 8 bytes when the dtype is `np.uint8`. I'll try to figure out how to make an aligned copy. But if it's not really necessary in practice, then this is ready.

EDIT: I've looked at the `.data_ptr()` addresses when using the `safetensors` library with an unaligned model, and it doesn't make an aligned copy (at least when using `get_slice` like since #8482). So the new behavior is pretty much the same as with the `safetensors` library.

Tested on <https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B>

Thanks @fairydreaming for finding this problem! (and finding the rationale behind why unaligned safetensors exist)

*Make sure to read the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md) before submitting a PR*


Changed files:
filename: gguf-py/gguf/utility.py
patch: @@ -110,7 +110,6 @@ class SafetensorRemote:
     """
 
     BASE_DOMAIN = "https://huggingface.co"
-    ALIGNMENT = 8 # bytes
 
     @classmethod
     def get_list_tensors_hf_model(cls, model_id: str) -> dict[str, RemoteTensor]:
@@ -204,9 +203,6 @@ def get_metadata(cls, url: str) -> tuple[dict, int]:
 
         # Calculate the data start offset
         data_start_offset = 8 + metadata_length
-        alignment = SafetensorRemote.ALIGNMENT
-        if data_start_offset % alignment != 0:
-            data_start_offset += alignment - (data_start_offset % alignment)
 
         # Check if we have enough data to read the metadata
         if len(raw_data) < 8 + metadata_length:
@@ -298,7 +294,6 @@ class SafetensorsLocal:
         Custom parsing gives a bit more control over the memory usage.
         The official safetensors library doesn't expose file ranges.
     """
-    ALIGNMENT = 8  # bytes
 
     tensors: dict[str, LocalTensor]
 
@@ -316,9 +311,6 @@ def __init__(self, filename: Path):
                 raise ValueError(f"Failed to parse safetensors metadata as JSON: {e}")
 
             data_start_offset = f.tell()
-            alignment = self.ALIGNMENT
-            if data_start_offset % alignment != 0:
-                data_start_offset += alignment - (data_start_offset % alignment)
 
             tensors: dict[str, LocalTensor] = {}
             for name, meta in metadata.items():