Advanced NumPy: Master stride tricks with 25 illustrated exercises | by Raimi Karim | Towards Data Science

Best viewed with a Chrome browser on desktop
Update 6 Jan 2022: Updated Problem 19 based on comments from Sirouan Nouriddine.Update 5 Jan 2022: Fix typos and improve clarity.Update 30 Dec 2021: As of NumPy 1.20.0, there is a sliding_window_view API which can also serve as a higher-level API on top of the as_strided API. See here for more.
(Jump to the exercises here)
The stride tricks API can be seen as an extension of the usual way of accessing and manipulating NumPyndarray’s, giving users more flexibility to control the resulting NumPy view. While it is an esoteric feature, one particularly practical usage is when sliding windows or rolling statistics are concerned. In this article, we will go through 25 different exercises that use this API (and compare to how we would’ve done it normally).
For this article, it is recommended that the reader have mid-level knowledge of Python, NumPy, numpy.dtype, numpy.ndarray.strides, and numpy.ndarray.itemsize. For a quick introduction to NumPy arrays and strides, see the section on 💡 A bit of background below.
The exercises, which are arranged with increasing difficulty, are formatted in the following order:
The question, illustrated as a diagram with NumPy array input and the expected output, a NumPy view. Assume contiguity in the manner that you read the elements. The itemsize varies for every question.
The answer — the strides and shape to be used as parameters in numpy.lib.stride_tricks.as_strided to achieve the final NumPy view.
The explanation
The code, where as_strided is the namespace imported from this import statement:from numpy.lib.stride_tricks import as_strided
💡 A bit of background
How would you access a specific item from a block of fixed-size elements which have been (i) placed contiguously, and (ii) organised into nested sub-groups? Answer: strides. What I just briefly described is a NumPy N-dimensional array (ndarray) data structure, and we use an algorithm called the strided indexing scheme together with strides to traverse it.
Here are 4 quick bites you should know about NumPy arrays.
1) Elements in NumPy arrays occupy memory. Every element in a NumPy array uniformly occupies n bytes. For example, every element in an array with data type np.int64 occupies 8 bytes. To find out the item size, np.ndarray.itemsize.
2) Elements in NumPy arrays are stored contiguously in memory. This means they are stored side-by-side (unlike elements in Python lists).
3) There is a piece of information called shape that you probably already know, that defines how large this array is for every dimension. To access this information, np.ndarray.shape.
4) There is another piece of information called strides that indicate the number of bytes to jump to reach the next value in the dimension. To access this information, np.ndarray.strides.
With these 4 pieces of information, the memory location of an element can be found by a linear combination of the dimension with the strides as the coefficients. For a more in-depth explanation, refer to my references below.
Slice first 3 elementsSlice first 8 elementsFlatten a 2D arraySkip every other elementSlice first columnSlice a diagonalRepeat the first elementSimple 2D slicingSlice a zigzagSparse slicingTranspose a 2D arrayRepeat the first column 4 timesReshape 1D array to 2D arraySlide a 1D windowSlide a 2D window then flattenCollapse an axis from a 3D array2 cornersStaggered slicingRepeat a 2D array3D transposeSlide a 2D windowReshape 1D array to 3D arraySlide a 2D receptive field for convolutionsRepeat a 3D tensorReshape 1D array to 4D array
Slice first 3 elements
Slice first 8 elements
Flatten a 2D array
Skip every other element
Slice first column
Slice a diagonal
Repeat the first element
Simple 2D slicing
Slice a zigzag
Sparse slicing
Transpose a 2D array
Repeat the first column 4 times
Reshape 1D array to 2D array
Slide a 1D window
Slide a 2D window then flatten
Collapse an axis from a 3D array
2 corners
Staggered slicing
Repeat a 2D array
3D transpose
Slide a 2D window
Reshape 1D array to 3D array
Slide a 2D receptive field for convolutions
Repeat a 3D tensor
Reshape 1D array to 4D array
Answer
strides = (1,)  shape = (3,)
💡 ExplanationAdjacent elements in the output (i.e. 1 → 2, 2 → 3) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(3,), strides=(1,))array([1, 2, 3], dtype=int8)
Similar to
>>> x[0,:3]array([1, 2, 3], dtype=int8)
Answer
strides = (1,)  shape = (8,)
💡 ExplanationAdjacent elements in the output (eg. 1 → 2, 2 → 3, 6 → 7) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 8.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(8,), strides=(1,))array([1, 2, 3, 4, 5, 6, 7, 8], dtype=int8)
Similar to
>>> x[0,:8]array([1, 2, 3, 4, 5, 6, 7, 8], dtype=int8)
Answer
strides = (2,)  shape = (25,)
💡 ExplanationAdjacent elements in the output (eg. 1 → 2, 2 → 3, 23 → 24, 24 → 25) were originally 2 bytes apart (=1 element away × 2 bytes) in the input. The shape of this dimension is 25.
Code
>>> x = np.asarray(range(1,26), np.int16).reshape(5,5)>>> as_strided(x, shape=(25,), strides=(2,))array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10,       11, 12, 13, 14, 15, 16, 17, 18, 19, 20,       21, 22, 23, 24, 25], dtype=int16)
Similar to
>>> x.ravel()array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10,       11, 12, 13, 14, 15, 16, 17, 18, 19, 20,       21, 22, 23, 24, 25], dtype=int16)
Answer
strides = (2,)  shape = (3,)
💡 ExplanationAdjacent elements in the output (i.e. 1 → 3, 3 → 5) were originally 2 bytes apart (=2 elements away × 1 byte) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(3,), strides=(2,))array([1, 3, 5], dtype=int8)
Similar to
>>> x[0,::2]array([1, 3, 5], dtype=int8)
Answer
strides = (40,)  shape = (4,)
💡 ExplanationAdjacent elements in the output (i.e. 1 → 6, 6 → 11, 11 → 16) were originally 40 bytes apart (=5 elements away × 8 bytes) in the input. The shape of this dimension is 4.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(4,), strides=(40,))array([ 1,  6, 11, 16])
Similar to
>>> x[:4,0]array([ 1,  6, 11, 16])
Answer
strides = (48,)  shape = ( 5,)
💡 ExplanationAdjacent elements in the output (i.e. 1 → 7, 7 → 13, 13 → 19, 19 → 25) were originally 48 bytes apart (=6 elements away × 8 bytes) in the input. The shape of this dimension is 5.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(5,), strides=(48,))array([ 1,  7, 13, 19, 25])
Similar to
>>> x.diagonal()array([ 1,  7, 13, 19, 25])
Answer
strides = (0,)  shape = (5,)
💡 ExplanationAdjacent elements in the output (i.e. 1 → 1) were originally 0 bytes apart (=0 elements away × 8 bytes) in the input. The shape of this dimension is 5.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(5,), strides=(0,))array([ 1, 1, 1, 1, 1])
Similar to
>>> np.broadcast_to(x[0,0], (5,))array([1, 1, 1, 1, 1])
Answer
strides = (40,8)  shape = ( 3,4)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 8 → 9) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. The shape of this dimension is 4.
Output’s left-to-right dimension (axis=-2):Adjacent elements in the output (eg. 1 → 6, 2 → 7, 9 → 14) were originally 40 bytes apart (=5 elements away × 8 bytes) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(3,4), strides=(40,8))array([[ 1,  2,  3,  4],       [ 6,  7,  8,  9],       [11, 12, 13, 14]])
Similar to
>>> x[:3,:4]array([[ 1,  2,  3,  4],       [ 6,  7,  8,  9],       [11, 12, 13, 14]])
Answer
strides = (48,8)  shape = ( 4,2)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 7 → 8, 13 → 14, 19 → 20) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s left-to-right dimension (axis=-2):Adjacent elements in the output (eg. 1 → 7, 2 → 8, 13 → 19) were originally 48 bytes apart (=6 elements away × 8 bytes) in the input. The shape of this dimension is 4.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(4,2), strides=(48,8))array([[ 1,  2],       [ 7,  8],       [13, 14],       [19, 20]])
Similar to
>>> # this may not be achieved concisely
Answer
strides = (80,16)  shape = ( 3, 3)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 3, 21 → 23, 13 → 15) were originally 16 bytes apart (=2 elements away × 8 bytes) in the input. The shape of this dimension is 3.
Output’s left-to-right dimension (axis=-2):Adjacent elements in the output (eg. 1 → 11, 13 → 23, 15 → 25) were originally 80 bytes apart (=10 elements away × 8 bytes) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int64).reshape(5,5)>>> as_strided(x, shape=(3,3), strides=(80,16))array([[ 1,  3,  5],       [11, 13, 15],       [21, 23, 25]])
Similar to
>>> x[::2,::2]array([[ 1,  3,  5],       [11, 13, 15],       [21, 23, 25]])
Answer
strides = (1,5)  shape = (3,3)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 6, 7 → 12, 3 → 8) were originally 5 bytes apart (=5 elements away × 1 byte) in the input. The shape of this dimension is 3.
Output’s top-to-right dimension (axis=-2):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 11 → 12) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(3,3), strides=(1,5))array([[ 1,  6, 11],       [ 2,  7, 12],       [ 3,  8, 13]], dtype=int8)
Similar to
>>> x[:3,:3].Tarray([[ 1,  6, 11],       [ 2,  7, 12],       [ 3,  8, 13]], dtype=int8)
Answer
strides = (20,0)  shape = ( 5,4)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 1, 6 → 6, 16 → 16) were originally 0 bytes apart (=0 elements away × 4 bytes) in the input. The shape of this dimension is 4.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 6, 6 → 11, 11 → 16) were originally 20 bytes apart (=5 elements away × 4 bytes) in the input. The shape of this dimension is 5.
Code
>>> x = np.asarray(range(1,26), np.int32).reshape(5,5)>>> as_strided(x, shape=(5,4), strides=(20,0))array([[ 1,  1,  1,  1],       [ 6,  6,  6,  6],       [11, 11, 11, 11],       [16, 16, 16, 16],       [21, 21, 21, 21]], dtype=int32)
Similar to
>>> np.broadcast_to(x[:,0,None], (5,4))array([[ 1,  1,  1,  1],       [ 6,  6,  6,  6],       [11, 11, 11, 11],       [16, 16, 16, 16],       [21, 21, 21, 21]], dtype=int32)
Answer
strides = (24,8)  shape = ( 4,3)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 7 → 8, 11 → 12) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. The shape of this dimension is 3.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 4, 4 → 7, 7 → 10) were originally 24 bytes apart (=3 elements away × 8 bytes) in the input. The shape of this dimension is 4.
Code
>>> x = np.asarray(range(1,13), np.int64)>>> as_strided(x, shape=(4,3), strides=(24,8))array([[ 1,  2,  3],       [ 4,  5,  6],       [ 7,  8,  9],       [10, 11, 12]])
Similar to
>>> x.reshape(4,3)array([[ 1,  2,  3],       [ 4,  5,  6],       [ 7,  8,  9],       [10, 11, 12]])
Adapted from a StackOverflow post [1]. Similar to [2] and [3].
Answer
strides = (1,1)  shape = (8,3)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 3 → 4, 4 → 5) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 3.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (i.e. 1 → 2, 2 → 3, 4 → 5, ..., 7 → 8) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 8.
Code
>>> x = np.asarray(range(1,26), np.int8)>>> as_strided(x, shape=(8,3), strides=(1,1))array([[ 1,  2,  3],       [ 2,  3,  4],       [ 3,  4,  5],       [ 4,  5,  6],       [ 5,  6,  7],       [ 6,  7,  8],       [ 7,  8,  9],       [ 8,  9, 10]], dtype=int8)
Similar to
>>> # this may not be achieved concisely
Question taken from a StackOverflow post [4].
Answer
strides = (2,1)  shape = (4,6)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 0 → 1, 1 → 10, 31 → 40) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 6.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 0 → 10, 10 → 20, 41 → 51) were originally 2 bytes apart (=2 elements away × 1 byte) in the input. The shape of this dimension is 4.
Code
>>> x = np.asarray(...         [0,1,10,11,20,21,30,31,40,41,50,51],...         np.int8).reshape(6,2)>>> as_strided(x, shape=(4,6), strides=(2,1))array([[ 0,  1, 10, 11, 20, 21],       [10, 11, 20, 21, 30, 31],       [20, 21, 30, 31, 40, 41],       [30, 31, 40, 41, 50, 51]], dtype=int8)
Similar to
>>> # this may not be achieved concisely
Answer
strides = (4,1)  shape = (3,4)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 3 → 4, 4 → 5) were originally 1 byte apart (=1 element away × 1 byte) in the input. The shape of this dimension is 4.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (i.e. 1 → 5, 5 → 9, 2 → 6, 6 → 10) were originally 4 bytes apart (=4 elements away × 1 byte) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,13), np.int8).reshape(3,2,2)>>> as_strided(x, shape=(3,4), strides=(4,1))array([[ 1,  2,  3,  4],       [ 5,  6,  7,  8],       [ 9, 10, 11, 12]], dtype=int8)
Similar to
>>> x.reshape(3,4)array([[ 1,  2,  3,  4],       [ 5,  6,  7,  8],       [ 9, 10, 11, 12]], dtype=int8)
Answer
strides = (30,10, 2)  shape = ( 2, 2, 2)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (i.e. 1 → 2, 6 → 7, 16 → 17, 21 → 22) were originally 2 bytes apart (=1 element away × 2 bytes) in the input. The shape of this dimension is 2.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (i.e. 1 → 6, 2 → 7, 16 → 21, 17 → 22) were originally 10 bytes apart (=5 elements away × 2 bytes) in the input. The shape of this dimension is 2.
Output’s box-to-box dimension (axis=-3):Adjacent elements in the output (i.e. 1 → 16, 2 → 17, 6 → 21, 7 → 22) were originally 30 bytes apart (=15 elements away × 2 bytes) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,26), np.int16).reshape(5,5)>>> as_strided(x, shape=(2,2,2), strides=(30,10,2))array([[[ 1,  2],        [ 6,  7]],       [[16, 17],        [21, 22]]], dtype=int16)
Similar to
>>> # this may not be achieved concisely
Answer
strides = (10,6,1)  shape = ( 2,2,3)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 17 → 18) were originally 1 byte apart (=1 element away × 1 byte) in the input. Shape of this dimension is 3.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 7, 11 → 17, 12 → 18) were originally 6 bytes apart (=6 elements away × 1 byte) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 11, 8 → 18, 9 → 19) were originally 10 bytes apart (=10 elements away × 1 byte) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(2,2,3), strides=(10,6,1))array([[[ 1,  2,  3],        [ 7,  8,  9]],       [[11, 12, 13],        [17, 18, 19]]], dtype=int8)
Similar to
>>> # this may not be achieved concisely
This question is taken from a StackOverflow post here [5].
Answer
strides = (0,10,2)  shape = (3, 2,4)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 6 → 7) were originally 2 bytes apart (=1 element away × 2 bytes) in the input. Shape of this dimension is 4.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 6, 2 → 7, 3 → 8) were originally 10 bytes apart (=5 elements away × 2 bytes) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 1, 3 → 3, 7 → 7) were originally 0 bytes apart (=0 elements away × 2 bytes) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,26), np.int16).reshape(5,5)>>> as_strided(x, shape=(3,2,4), strides=(0,10,2))array([[[1, 2, 3, 4],        [6, 7, 8, 9]],       [[1, 2, 3, 4],        [6, 7, 8, 9]],       [[1, 2, 3, 4],        [6, 7, 8, 9]]], dtype=int16)
Similar to
>>> np.broadcast_to(x[0:2, 0:-1], (3, 2, 4))
(Thank you, Sirouan Nouriddine for sharing this!)
Answer
strides = (16,4,8)  shape = ( 3,2,2)
💡 Explanation
Output’s left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 3, 2 → 4, 10 → 12) were originally 8 bytes apart (=2 elements away × 4 bytes) in the input. The shape of this dimension is 2.
Output’s top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 2, 3 → 4, 9 → 10, 11 → 12) were originally 4 bytes apart (=1 element away × 4 bytes) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 5, 5 → 9) were originally 16 bytes apart (=4 elements away × 4 bytes) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,13), np.int32).reshape(3,2,2)>>> as_strided(x, shape=(3,2,2), strides=(16,4,8))array([[[ 1,  3],        [ 2,  4]],       [[ 5,  7],        [ 6,  8]],       [[ 9, 11],        [10, 12]]], dtype=int32)
Similar to
>>> np.swapaxes(x,1,2)array([[[ 1,  3],        [ 2,  4]],       [[ 5,  7],        [ 6,  8]],       [[ 9, 11],        [10, 12]]], dtype=int32)
Question adapted from SciPy 2008 conference [6].
Answer
strides = (40,40,8)  shape = ( 3, 2,5)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 12 → 13, 16 → 17) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. Shape of this dimension is 5.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 6, 8 → 13, 11 → 16) were originally 40 bytes apart (=5 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 9 → 14, 14 → 19) were originally 40 bytes apart (=5 elements away × 8 bytes) in the input. The shape of this dimension is 3.
Code
>>> x = np.asarray(range(1,21), np.int64).reshape(4,5)>>> as_strided(x, shape=(3,2,5), strides=(40,40,8))array([[[ 1,  2,  3,  4,  5],        [ 6,  7,  8,  9, 10]],       [[ 6,  7,  8,  9, 10],        [11, 12, 13, 14, 15]],       [[11, 12, 13, 14, 15],        [16, 17, 18, 19, 20]]])
Similar to
>>> # this may not be achieved concisely
Answer
strides = (6,3,1)  shape = (2,2,3)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 5 → 6, 7 → 8, 10 → 11) were originally 1 byte apart (=1 element away × 1 byte) in the input. Shape of this dimension is 3.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 4, 2 → 5, 8 → 11) were originally 3 bytes apart (=3 elements away × 1 byte) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 7, 2 → 8, 3 → 9) were originally 6 bytes apart (=6 elements away × 1 byte) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,13), np.int8)>>> as_strided(x, shape=(2,2,3), strides=(6,3,1))array([[[ 1,  2,  3],        [ 4,  5,  6]],       [[ 7,  8,  9],        [10, 11, 12]]], dtype=int8)
Similar to
>>> x.reshape(2,2,3)array([[[ 1,  2,  3],        [ 4,  5,  6]],       [[ 7,  8,  9],        [10, 11, 12]]], dtype=int8)
Adapted from a StackOverflow post on 2D convolution here [7].
Answer
strides = (10,2,5,1)  shape = ( 2,2,3,3)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 17 → 18, 24 → 25) were originally 1 byte apart (=1 element away × 1 byte) in the input. Shape of this dimension is 3.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 6, 2 → 7, 3 → 8) were originally 5 bytes apart (=5 elements away × 1 byte) in the input. The shape of this dimension is 3.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 3, 6 → 8, 21 → 23, 23 → 25) were originally 2 bytes apart (=2 elements away × 1 byte) in the input. The shape of this dimension is 2.
Output’s top box-to-bottom box dimension (axis=-4):Adjacent elements in the output (eg. 1 → 11, 2 → 12, 15 → 25) were originally 10 bytes apart (=10 elements away × 1 byte) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,26), np.int8).reshape(5,5)>>> as_strided(x, shape=(2,2,3,3), strides=(10,2,5,1))array([[[[ 1,  2,  3],         [ 6,  7,  8],         [11, 12, 13]],        [[ 3,  4,  5],         [ 8,  9, 10],         [13, 14, 15]]],       [[[11, 12, 13],         [16, 17, 18],         [21, 22, 23]],        [[13, 14, 15],         [18, 19, 20],         [23, 24, 25]]]], dtype=int8)
Similar to
>>> # this may not be achieved concisely
Answer
strides = (48, 0,24, 8)  shape = ( 2, 2, 2, 3)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 4 → 5) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. Shape of this dimension is 3.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 4, 7 → 10, 8 → 11) were originally 24 bytes apart (=3 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 1, 10 → 10, 12 → 12) were originally 0 bytes apart (=0 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s top box-to-bottom box dimension (axis=-4):Adjacent elements in the output (eg. 1 → 7, 2 → 8, 3 → 9) were originally 48 bytes apart (=6 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,13), np.int64).reshape(2,2,3)>>> as_strided(x, shape=(2,2,2,3), strides=(48,0,24,8))array([[[[ 1,  2,  3],         [ 4,  5,  6]],        [[ 1,  2,  3],         [ 4,  5,  6]]],       [[[ 7,  8,  9],         [10, 11, 12]],        [[ 7,  8,  9],         [10, 11, 12]]]])
Similar to
>>> np.broadcast_to(x,(2,2,2,3)).swapaxes(0,1)array([[[[ 1,  2,  3],         [ 4,  5,  6]],        [[ 1,  2,  3],         [ 4,  5,  6]]],       [[[ 7,  8,  9],         [10, 11, 12]],        [[ 7,  8,  9],         [10, 11, 12]]]])
Answer
strides = (64,32,16, 8)  shape = ( 2, 2, 2, 2)
💡 Explanation
Output’s intrabox left-to-right dimension (axis=-1):Adjacent elements in the output (eg. 1 → 2, 2 → 3, 4 → 5) were originally 8 bytes apart (=1 element away × 8 bytes) in the input. Shape of this dimension is 2.
Output’s intrabox top-to-bottom dimension (axis=-2):Adjacent elements in the output (eg. 1 → 4, 7 → 10, 8 → 11) were originally 16 bytes apart (=2 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s left box-to-right box dimension (axis=-3):Adjacent elements in the output (eg. 1 → 1, 10 → 10, 12 → 12) were originally 32 bytes apart (=4 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Output’s top box-to-bottom box dimension (axis=-4):Adjacent elements in the output (eg. 1 → 7, 2 → 8, 3 → 9) were originally 64 bytes apart (=8 elements away × 8 bytes) in the input. The shape of this dimension is 2.
Code
>>> x = np.asarray(range(1,17), np.int64)>>> as_strided(x, shape=(2,2,2,2), strides=(64,32,16,8))array([[[[ 1,  2],         [ 3,  4]],        [[ 5,  6],         [ 7,  8]]],       [[[ 9, 10],         [11, 12]],        [[13, 14],         [15, 16]]]])
Similar to
>>> x.reshape(2,2,2,2)array([[[[ 1,  2],         [ 3,  4]],        [[ 5,  6],         [ 7,  8]]],       [[[ 9, 10],         [11, 12]],        [[13, 14],         [15, 16]]]])
⚠️ While stride tricks give you more control over the resulting NumPy view, the API is not memory-safe — things can get pretty nasty if you miscalculate the itemsize (honestly I think this API should not allow client code to have to deal with item size as I haven’t seen any benefits of exposing this) or the shape or the existing strides, returning data that is actually not the initial array you created, but from a different array altogether which you probably defined few lines back 😱. This is known as buffer overflow and it’s not hard to encounter this using the stride tricks API. What’s worse is when you decide to write to this data 😱😱. It is for this reason that the stride tricks documentation cautions the user to exercise extreme care when using it.
Understanding the stride tricks API can be challenging, and I had problems with that. However, the trick (no pun intended) is to start with smaller dimensions and visualise the output of the tensors. Have fun!
Found a mistake? Let me know in the comments! :) Shoutout to David Chong for reviewing this article.
If you like my content and haven’t already subscribed to Medium, subscribe via my referral link here! NOTE: A portion of your membership fees will be apportioned to me as referral fees.
The N-dimensional array (ndarray) (numpy.org)
Advanced NumPy (scipy-lectures.org)
An Illustrated Guide to Shape and Strides (ajcr.net)
Using stride tricks with NumPy (ipython-books.github.io)