An image processing neural network system implemented by one or more computers, wherein the image processing neural network system is configured to receive an input image and to generate a network output for the input image, and wherein the image processing neural network system comprises: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers (204, 208, 306, 310, 314, 406, 410, 416, 422) arranged in a stack one after the other, wherein each separable convolutional neural network layer is configured to: separately apply a depthwise convolution and then a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output, wherein the depthwise convolution is a channel-wise spatial convolution that is performed independently over every channel of an input to the depthwise convolution, wherein the input to the depthwise convolution comprises a plurality of channels, and the pointwise convolution is a lxl convolution that projects the channels computed by the depthwise convolution onto a new channel space; and provide the layer output as input to the next separable convolutional neural network layer in the stack; an input subnetwork configured to receive the input image, to process the input image to generate an initial output, and to provide the initial output as an input to the separable convolution subnetwork; and an output subnetwork configured to receive a separable convolution output generated by the separable convolution subnetwork and to process the separable convolution output to generate the network output for the input image, wherein the network output comprises one or more scores for each of a set of object categories, with each score representing a likelihood that the input image contains an image of an object belonging to the category. a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers (204, 208, 306, 310, 314, 406, 410, 416, 422) arranged in a stack one after the other, wherein each separable convolutional neural network layer is configured to: separately apply a depthwise convolution and then a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output, wherein the depthwise convolution is a channel-wise spatial convolution that is performed independently over every channel of an input to the depthwise convolution, wherein the input to the depthwise convolution comprises a plurality of channels, and the pointwise convolution is a lxl convolution that projects the channels computed by the depthwise convolution onto a new channel space; and provide the layer output as input to the next separable convolutional neural network layer in the stack; separately apply a depthwise convolution and then a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output, wherein the depthwise convolution is a channel-wise spatial convolution that is performed independently over every channel of an input to the depthwise convolution, wherein the input to the depthwise convolution comprises a plurality of channels, and the pointwise convolution is a lxl convolution that projects the channels computed by the depthwise convolution onto a new channel space; and provide the layer output as input to the next separable convolutional neural network layer in the stack; an input subnetwork configured to receive the input image, to process the input image to generate an initial output, and to provide the initial output as an input to the separable convolution subnetwork; and an output subnetwork configured to receive a separable convolution output generated by the separable convolution subnetwork and to process the separable convolution output to generate the network output for the input image, wherein the network output comprises one or more scores for each of a set of object categories, with each score representing a likelihood that the input image contains an image of an object belonging to the category.