Repository: wandb/wandb

Issue:
Fastai WandbCallback with multilabel datasets
* Weights and Biases version: 0.8.9
* Python version: 3.6.9
* Operating System: Linux-4.14.133-88.112.amzn1.x86_64-x86_64-with-Ubuntu-16.04-xenial

### Description

I'm trying to see prediction samples for fastai image classification, where the images are multilabel. I'd like  it to be ran on the validation data at the end of each epoch and to see captions of ground truth/predictions.

When I use a multilabel dataset, instead of ground truth/prediction captions for each image, I get a three copies of each image side by side, where each is labeled, "Input data", "Prediction", "Ground Truth" (but not the actual categories)

### What I Did

`callback_fns += [partial(WandbCallback, input_type='images', monitor='acc_thresholded')]`




Discussion:
Issue Label Bot is not confident enough to auto-label this issue. See [dashboard](https://mlbot.net/data/wandb/client) for more details.
@borisdayma could you take a look at this please?
@austinmw do you have a code sample I can look at?
The callback tries to guess the type of problem (classification, segmentation...) based on the type of input/output seen.
This case is probably not handled and it would be good to see in which category of problems the callback is classifying it.
@borisdayma Hmm I'm not familiar with what a coffee sample is. The category of problem is a CNN classifier that is multilabel (not just multiclass). So in this case, each image can have between 0 and 4 labels, and each output value uses a sigmoid activation, so they don't sum to 1. My fastai databunch is created using `.label_from_df(label_delim=' ')`.

Here's an example of what some of the label strings (pre-delimited) might look like:
`['', 'hat', 'hat glasses', 'hat gloves', 'hat glasses gloves, '', 'gloves', 'boots glasses', 'boots hat glasses gloves']`
So, an empty string would assign an image 0 labels, and an image could have up to 4 labels in this case.

Sorry I had a typo.
What is the shape of output tensor?
Is it (batch_size, 4) with each value representing probability of a class?

A simple example may make it easier to debug if possible. Maybe you are even adapting one of fastai notebooks that I can look at?
@borisdayma Yes that's the output shape. It's difficult for me to create a minimum reproducible example, because my code is currently completely intertwined with aws sagemaker. However, a good example of an almost identical problem (fastai, cnn, multilabel) is fastai's course v3 part 1 lesson 3 example using the planets dataset: https://nbviewer.jupyter.org/github/fastai/course-v3/blob/master/nbs/dl1/lesson3-planet.ipynb I think if it works with that it should work with mine as well.
Ok the issue is that multi-classification problems are not automatically detected yet. Let me look a bit more into it.
I think this can now be handled correclty with commit https://github.com/borisdayma/client/commit/5f43f7dac4068bc0bfe4a91d13469f8a0b36649c

I just need to do some more testing to ensure it does not affect other types of problems.

Feel free to test it in the meantime.

Here is a [sample run](https://app.wandb.ai/borisd13/course-v3-nbs_dl1/runs/kabv3wiv?workspace=user-borisd13).

Resolving patch:
diff --git a/wandb/fastai/__init__.py b/wandb/fastai/__init__.py
--- a/wandb/fastai/__init__.py
+++ b/wandb/fastai/__init__.py
@@ -179,7 +179,8 @@ def _wandb_log_predictions(self):
                 raise FastaiError('Unable to run "predict" method from Learner to log prediction samples.')
 
             # scalar -> likely to be a category
-            if not pred[1].shape:
+            # tensor of dim 1 -> likely to be multicategory
+            if not pred[1].shape or pred[1].dim() == 1:
                 pred_log.append(
                     wandb.Image(
                         x.data,


Test patch: