Mastering Python’s __getitem__ and slicing
Tags: python
Published: 2019-05-20 03:29:04

In Python’s data model, when you write `foo[i]` , `foo.__getitem__(i)` is called. You can implement `__getitem__` method in your class, and define the behaviour of slicing.

Example:

```python
class Foo:
    def __getitem__(self, key):
        print(key)
        return None

foo = Foo()
foo[1] # => print(1)
foo["aaa"] # => print("aaa")
```

As you know, Python can slice more complicated object.

```python
# comma separated slicing
d = {}
d[1, "hello"] = 3

# colon separated slicing
ls = [0, 1, 2, 3]
ls[:2]
ls[3:]
ls[2:4]
ls[::2]
ls[1:4:-1]
ls[:]
ls[::]

# both comma and colon
import numpy as np
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
a[:2, 1:3]
```

Let’s inspect what kind of object is passed to `__getitem__` argument.

# Comma Separated Slicing

First, we check comma separated slicing.
To inspect, we define `Inspector` class.

``` python
class Inspector:
    def __getitem__(self, key):
        return key
```

Then, we can inspect the slicing.

```python
a = Inspector()
a[1] # => 1: int

a[1, 2] # => (1, 2): tuple

a[1, 2, 3] # => (1, 2, 3): tuple

a[] # => SyntaxError
```

This behaviour is very similar to right hand side of assignment expression.

```python
t = 1 # 1: int
t = 1, 2 # (1, 2): tuple
t = 1, 2, 3 # (1, 2, 3): tuple
t = # SyntaxError
```

More examples needed?

```python
a[(1, 2, 3)] # (1, 2, 3): tuple
assert a[(1, 2, 3)] == a[1, 2, 3]
t = (1, 2, 3)
u = 1, 2, 3
assert t == u

# tuple of one element.
a[1,] # (1,): tuple
t = 1, # (1,): tuple
a[(1,)] # (1,): tuple
t = (1,) # (1,): tuple

# empty tuple
a[()] # (): tuple
t = () # (): tuple

t = 1, 2
assert a[t] == a[1, 2]

a[(1, 2), 3] # ((1, 2), 3): tuple
t = (1, 2), 3 # ((1, 2), 3): tuple
```

However, starred item and yield expression are not allowed for slicing.

```python
t = *iter([]), 1 # (1,): tuple
a[*iter([]), 1] # SyntaxError: invalid syntax

def g():
    t = yield 1 # Valid.
    a = Inspector()
    a[yield 1] # SyntaxError: invalid syntax
```

## Conclusion of comma separated slice

As same as right hand side of assignment expression, comma separated slice is interpreted as tuple.

# Colon separated slice

Let us inspect colon separated slice.

```python
# As same as above part.
class Inspector:
    def __getitem__(self, key):
        return key

a = Inspector()
```

Colon separated slice makes a slice object.

```python
a[1:2:3] # => slice(1, 2, 3)
assert a[1:2:3] == slice(1, 2, 3)
```

Docstring of slice object is:

```
slice(stop)
slice(start, stop[, step])

Create a slice object.  This is used for extended slicing (e.g. a[0:10:2]).
```

Colon separated slice’s notifications are `[start:]`, `[start::]`, `[:stop]`, `[:stop:]`, `[::step]`, `[start:stop]`, `[start:stop:]`, `[start::step]`, `[:stop:step]` and `[start:stop:step]`.

```python
start = 1
stop = 2
step = 3

assert a[start:] == a[start::] == slice(start, None, None) == slice(start, None)

assert a[:stop] == a[:stop:] == slice(None, stop, None) == slice(stop)

assert a[start:stop] == a[start:stop:] == slice(start, stop, None) == slice(start, stop)

assert a[:stop:step] == slice(None, stop, step)

assert a[start:stop:step] == slice(start, stop, step)
```

Now, we understood relations between colon separated slicing and slice object.

Type of start, stop, step are not only integer. Any types are acceptable.

```
a[[]:{}:"abc"] # => slice([], {}, "abc")

class MyClass:
    pass

a[MyClass():MyClass()] # => slice(MyClass(), MyClass())
```

## Handling of slice object

slice object have start, stop, and step properties.

```
sl = slice(1, '2', [3])

sl.start # => 1
sl.stop # => '2'
sl.step # => [3]

sl = slice('stop')

sl.start # => None
sl.stop # => 'stop'
sl.step # => None
```

slice object have `indices(len)` method.

Docstring is:

```
S.indices(len) -> (start, stop, stride)

Assuming a sequence of length len, calculate the start and stop
indices, and the stride length of the extended slice described by
S. Out of bounds indices are clipped in a manner consistent with the
handling of normal slices.
```

indices method returns if start , stop , step properties are not None and in the range of [0, len], returns them, otherwise, returns suitable value.

It helps to implement the slicing as same as list object.

```python
li = [i for i in range(10)]
def make_list(start, stop, stride):
    t = []
    if stride > 0:
        while start < stop:
            t.append(start)
            start += stride
    else:
        while start > stop:
            t.append(start)
            start += stride
    return t

start, stop, stride = a[6:].indices(10)
# start, stop, stride = 6, 10, 1
assert li[6:] == make_list(start, stop, stride) == [6, 7, 8, 9]

start, stop, stride = a[:7:2].indices(10)
# start, stop, stride = 0, 7, 2
assert li[:7:2] == make_list(start, stop, stride) == [0, 2, 4, 6]

start, stop, stride = a[:].indices(10)
# start, stop, stride = 0, 10, 1
assert li[:] == make_list(start, stop, stride) == [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

start, stop, stride = a[15:].indices(10)
# start, stop, stride = 10, 10, 1
assert li[15:] == make_list(start, stop, stride) == []

start, stop, stride = a[-3:].indices(10)
# start, stop, stride = 7, 10, 1
assert li[-3:] == make_list(start, stop, stride) == [7, 8, 9]

start, stop, stride = a[:5:-2].indices(10)
# start, stop, stride = 9, 5, 2
assert li[:5:-2] == make_list(start, stop, stride) == [9, 7]
```

## Conclusion of colon separated slicing

Colon separated slicing returns slice object. It has 3 properties. start, stop and step. If you want to implement the slicing as same as list object’s implementation, slice.indices method is helpful.

# Both comma and colon separated slicing

In this case, it is interpreted as tuple of slice.

```python
class Inspector:
    def __getitem__(self, key):
        return key

a = Inspector()

a[1:2,3:4] # (slice(1, 2, None), slice(3, 4, None))
a[1,2:3,4] # (1, slice(2, 3, None), 4)
a[1:(2,3):4] # slice(1, (2, 3), 4)
```

# `__index__` method

`__index__` method is very important to be a slicing master.

```python
import numpy as np

i = np.int32(3)
print(i) # => '3'
print(type(i)) # => 'np.int32'
```

In this code, is `ia` integer?

Yes. It is an integer but not a Python’s builtin `int` type object.

It may cause some troubles in your `__getitem__` implementation. You may want to convert to Python’s builtin `int` type.

You may think `int(i)` returns builtin `int` type. Yes, it is commonly used in Python. However, if `i` is floating point number, you may not want to convert to integer value. In this case, `__index__` method is useful.

>    `object.__index__(self)`
>    Called to implement `operator.index()`, and whenever Python needs to losslessly convert the numeric object to an integer object (such as in slicing, or in the built-in `bin()`, `hex()` and `oct()` functions). Presence of this method indicates that the numeric object is an integer type. Must return an integer.
>
>    Note
>    In order to have a coherent integer type class, when `__index__()` is defined `__int__()` should also be defined, and both should return the same value.
> (Python documentation [Data model](https://docs.python.org/3/reference/datamodel.html#object.__index__))

numpy.int{8, 16, 32, 64}, uint{8, 16, 32, 64} implements `__index__` method and it returns Python’s builtin `int` type. `float`, numpy’s floating point numbers are not implements `__index__` method.

When you want to convert an variable to int type for slicing, you should use this method.

You can also use `operator.index` function. When `__index__` is not implemented `foo.__index__()` raises `AttributeError`, `operator.index(foo)` raises `TypeError`.

```python
import operator
import numpy as np

i = np.int32(3)
i.__index__() # Returns 3: int
operator.index(i) # Returns 3: int

# builtins int type also implements __index__ method.
i = 3
i.__index__() # Returns 3: int
operator.index(i) # Returns 3: int

# floating point type doesn't implement __index__ method.
i = 3.0
i.__index__() # AttributeError: 'float' object has no attribute '__index__'
operator.index(i) # TypeError: 'float' object cannot be interpreted as an integer
```

# `__setitem__` and `__delitem__`

`__setitem__` and `__delitem__` are similar method with `__getitem__`.

`foo.__setitem__(key, value)` is called when `foo[key] = value` is evaluated. `foo.__delitem__(key)` is called when `del foo[key]` is evaluated. You may implement these methods for your class.

# Conclusion

In this article, we learned following.

- `foo[key]` calls `foo.__getitem__(key)`
- `foo[key] = value` calls `foo.__setitem__(key, value)`
- `del foo[key]` calls `foo.__delitem__(key)`
- `foo[k1, k2]` calls `foo.__getitem__((k1, k2))`
- `foo[k1:k2:k3]` calls `foo.__getitem__(slice(k1, k2, k3))`
- Integer type implements `__index__` method

This knowledge is useful for implementation of your subscriptable class.