Some memory allocations could be optimized away with:

```python
    for i in range(n_samples):
        np.dot(_ravel(PQd[i]), X_embedded[i] - X_embedded, out=grad[i])
```

although that probably won't change the timings by much.