zhiminy/EvalEng huggingface dataset:
Repository: Giskard-AI/giskard-oss
Issue: fix z-index overlay between project settings modal and Lime loading screen
GitHub labels: bug


![image](https://user-images.githubusercontent.com/7889985/206464355-d19dec29-9780-41cd-8f40-e94f33e6d7e6.png)


After a quick analysis:

- OverlayLoader's `z-index` is 1000
- Vuetify's dialog background is 201
- Vuetify's dialog contents is 202

A quick search thru Vuetify's repo doesn't appear to show any `z-index` value above these numbers, so I will be lowering OverlayLoader's `z-index` to 100

sounds good to me, it's worth checking `z-index` in the rest of the codebase to make sure that lowering the overlay doesn't break anything else

Indeed thanks for the heads-up, TestEditor.vue has a `z-index` of 100 for the class `code-editor-resize`. This class only happens when the OverlayLoader is not displayed, so it does not conflict directly.

Should we still handle that case and decide between the OverlayLoader or the editor's resizing class as to which one has priority?


I'll push with both having 100, and commit again if we decide to change it
Repository: Giskard-AI/giskard-oss
Issue: Failed to create text explanation Lime Enron project
Root cause: Validation Gap

To reproduce: 
Inspect Email data train with the logistic regression model
Inspect the second entry
In the explanation panel, select Feature: subject 
Run



![Image](https://user-images.githubusercontent.com/55790616/206508116-999d4060-bf03-45c8-859a-3750c0fd7632.png)
Repository: Giskard-AI/giskard-oss
Issue: test issue
GitHub labels: question

## Checklist

<!-- Mark with an `x` all the checkboxes that apply (like `[x]`) -->

- [ ] I've searched the project's [`issues`](https://github.com/Giskard-AI/giskard/issues?q=is%3Aissue).

## ❓ Question

<!-- What is your question -->

How can I [...]?

Is it possible to [...]?

## 📎 Additional context

<!-- Add any other context or screenshots about the feature request here. -->


test

another test
Repository: Giskard-AI/giskard-oss
Issue: Add progress information on inspection loading modal
GitHub labels: enhancement

This step is often taking over 30 seconds. With a simple spinner and no information on the progress, it leads the user to wonder if there is not a bug.

It would be a much better UX to have a % progress information

<img width="1409" alt="CleanShot 2022-12-13 at 15 04 28@2x" src="https://user-images.githubusercontent.com/4739848/207354703-e0a516a3-c67f-4a0a-8c2c-1dccc3b0f861.png">



add indefinite loading bar + text explaining that we're running scoring for the full dataset

Proposal:

![Screenshot 2022-12-19 at 17 16 16](https://user-images.githubusercontent.com/1703741/208470898-29d485b8-6cd2-425b-9b06-3f337e659168.png)

Any feedback on text or placement?

Good proposal! How about "Scoring dataset of {numRows} rows. This operation can take a while..."

And a small vertical space between the message and the loading bar.

At that point in time we do not know the number of rows in the dataset, so getting that information would require some changes to the backend too.

With vertical space:
![Screenshot 2022-12-20 at 10 07 33](https://user-images.githubusercontent.com/1703741/208627739-c99c1d8f-a3fc-4df7-847d-bd043aab2f1f.png)



Ok!

Alex Combessie
Co-founder & CEO @ *Giskard*
*Eliminate risks of AI biases*
Website: giskard.ai ( https://bit.ly/364GAGi )
*⭐️ Star us on G itHub ( https://bit.ly/37aWG1y )*
( https://bit.ly/37aWG1y )

On Tue, Dec 20 2022 at 10:08 AM, Hugo V < ***@***.*** > wrote:

> 
> 
> 
> At that point in time we do not know the number of rows in the dataset, so
> getting that information would require some changes to the backend too.
> 
> 
> 
> With vertical space:
> Screenshot 2022-12-20 at 10 07 33 (
> https://user-images.githubusercontent.com/1703741/208627739-c99c1d8f-a3fc-4df7-847d-bd043aab2f1f.png
> )
> 
> 
> 
> —
> Reply to this email directly, view it on GitHub (
> https://github.com/Giskard-AI/giskard/issues/597#issuecomment-1359041076 )
> , or unsubscribe (
> https://github.com/notifications/unsubscribe-auth/ABEFGCHSVQPUWKUOGBOG7TTWOFZQPANCNFSM6AAAAAAS5IEVCA
> ).
> You are receiving this because you authored the thread. Message ID: <Giskard-AI/giskard/issues/597/1359041076
> @ github. com>
> 
> 
>
Repository: Giskard-AI/giskard-oss
Issue: Blank screen when long loading time of tests
Root cause: Unimplemented Feature Gap

On this screen, the loading time of tests often takes several seconds, with a blank screen. Can we either speed up that loading time? Or add a spinner?

<img width="1510" alt="CleanShot 2022-12-13 at 15 10 06@2x" src="https://user-images.githubusercontent.com/4739848/207355924-276d27c5-c6b5-4b2a-9105-dd4e9e7cd3b4.png">

In general, for all screens with data tables with possible long loading times between 500ms and 5 seconds, we should think how to speed up the data retrieval, and add spinners to indicate something is going on.

To reproduce: http://prod.giskard.ai/main/projects/784/test-suites/980/

removed it from 1.4.0 as this screen will soon change in v.2
Repository: Giskard-AI/giskard-oss
Issue: Split build.yml for internal and external PRs
GitHub labels: enhancement

security

Currently `build.yml` workflow is triggered by a `pull_request_target` event which has access to the repository secrets (required to access Sonar). 

We shouldn't trigger `build.yml` automatically when an external PR is submitted, instead, it should happen once a label (e.g. "safe to test") is added to the PR by one of the repo maintainers.

More info:
https://securitylab.github.com/research/github-actions-preventing-pwn-requests/
Repository: Giskard-AI/giskard-oss
Issue: Create Bug Report template for enhanced user experience 
GitHub labels: enhancement

## 🚀 Feature Request

1. Create a template for bug report.
2. Configure this template according to [configuring-issue-templates-for-your-repository](https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository)
3. Verify how the template appears to the user

## 🔈 Motivation

For enhanced user experience
Repository: Giskard-AI/giskard-oss
Issue: Multi ML Worker support
Root cause: Unimplemented Feature Gap

## 🚀 Feature Request

Support multiple ML Worker connected at the same time

## 🔈 Motivation

To execute models trained using different python versions and different environments. To let different data scientists work with Giskard at the same time

## 🛰 Alternatives

Disconnecting an actively connected ML Worker and connecting a new one. This is a current solution but it's not great because it only allows a single active worker, so no way to run models trained in different environments at the same time.

## 📎 Additional context

In the settings admin should see all actively connected ML Workers
an active ML Worker should be assigned at a project level

## ❓Questions

Should workers have unique ids to detect if a worker that was selected as "active" for a given project isn't currently available?
Repository: Giskard-AI/giskard-oss
Issue: Overview always has a teal color even when it's not selected
GitHub labels: bug

good first issue

d:easy

![image](https://user-images.githubusercontent.com/7889985/208401580-d556c0f9-0c41-4a85-90fc-83b99ebe6085.png)

The correct behavior should be the same as for the other tabs - return to regular color once  the category isn't selected

Hi I would like to solve this issue can U please assign it to me?

We can close this issue now :)
Repository: Giskard-AI/giskard-oss
Issue: Add an example of AI Inspect in a dataset
Root cause: Unimplemented Feature Gap

## 🚀 Feature Request

create a dataset that contains some examples of AI Inspect, so that this dataset is used in AI Test to execute test. It can be a button in AI inspect like “add this example in a unit dataset”

## 🔈 Motivation

It’s very useful for NLP cases to create unit dataset (sarcastic examples dataset, double negation dataset, etc.)

## 🛰 Alternatives

We can mark these examples using the "the general feedback button"

## 📎 Additional context

<!-- Add any other context or screenshots about the feature request here. -->
Repository: Giskard-AI/giskard-oss
Issue: Change the error message: Failed to establish a connection with external ML Worker. Start it with "giskard worker start -h prod.giskard.ai in the environment that can execute the specified model"
Root cause: Documentation Deficiency


Instead of `Failed to establish a connection with external ML Worker. Start it with "giskard worker start -h XXX in the environment that can execute the specified model"`
- Write:

Failed to establish a connection with external ML Worker. Start it by running`giskard worker start -h XXX` in the terminal of the machine that execute your model. For help please refer to the doc: https://docs.giskard.ai/start/guides/installation/ml-worker

- Put the code part in markdown
Repository: Giskard-AI/giskard-oss
Issue: Remove user access management from open-source version
GitHub labels: feature


Changes to settings UI. Data removed from the User page:
<img width="1512" alt="Screenshot 2023-01-03 at 10 16 43" src="https://user-images.githubusercontent.com/1703741/210328828-32dfc6a3-59a0-4743-b86b-f4a0d259c8cf.png">

And added tabs to the settings tab, with General containing the missing info: 
<img width="1512" alt="Screenshot 2023-01-03 at 10 18 24" src="https://user-images.githubusercontent.com/1703741/210329103-df7da5f7-143d-413f-91e0-db6f1d77343f.png">
Repository: Giskard-AI/giskard-oss
Issue: Error in console when inspecting a model (doesn't impact the inspection)
GitHub labels: bug

This error occurs in multiple places but doesn't impact the utilisation of giskard. It's caused by the migration to composition API by Vue

![Image](https://user-images.githubusercontent.com/55790616/208918007-9195e88b-5c77-44b7-aa24-4a0bb206247a.png)



Strange issue, Andrey committed this: https://github.com/Giskard-AI/giskard/commit/3e2a8de6170b0f0de7c8df6881924d13b8b63d6d which should be a fix for it...
Repository: Giskard-AI/giskard-oss
Issue: Blank screen on saving test suits without any changes
Root cause: Algorithmic Error

## 🐛 Bug Report

The settings button shows a blank screen if we save the test suit settings without making any changes.

## 🔬 How To Reproduce

Steps to reproduce the behavior:

1. Create a test suit however u like.
2. Click on the settings/Edit button and save it without making any changes.
3. Now, if you click the settings button again, it'll show a blank Screen

### Screenshots
This is the output after Step-3 

![image](https://user-images.githubusercontent.com/60355338/210742485-7a01774f-33b2-49c9-87ee-52a0fd5dfe5f.png)

## 📈 Expected behavior

The settings menu should be shown when the button is clicked.

![image](https://user-images.githubusercontent.com/60355338/210742934-85cc1460-c4f1-426a-9daa-9d289ae90d6f.png)
Repository: Giskard-AI/giskard-oss
Issue: giskard worker logs
GitHub labels: enhancement

good first issue

As asked by Etam, it's good to be able to see the logs with a command like `giskard worker logs` instead of doing: `!tail -f $HOME/giskard-home/run/ml-worker.log`

Please change the doc, once it's done at: https://docs.giskard.ai/start/guides/installation/ml-worker
Repository: Giskard-AI/giskard-oss
Issue: iPython version seems not to be compatible with python3.8
Root cause: External Dependency Breakage

## 🐛 Bug Report

when running a jupyter notebook with python3.8 venv, I get the following error already since the imports:
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/codeop.py in __call__(self, source, filename, symbol)
    139 
    140     def __call__(self, source, filename, symbol):
--> 141         codeob = compile(source, filename, symbol, self.flags, 1)
    142         for feature in _features:
    143             if codeob.co_flags & feature.compiler_flag:

TypeError: required field "type_ignores" missing from Module
```

## 🔬 How To Reproduce

1. Load with jupyter-notebook the following notebook: https://github.com/Giskard-AI/giskard-examples/blob/main/Credit%20scoring%20classification%20model.ipynb

2. Execute the very first cell.

### Code sample

<!-- If applicable, attach a minimal code sample to reproduce the decried issue. -->

### Environment

* OS: [e.g. Linux / Windows / macOS]
* Python version, get it with:

Python 3.8.16
jupyter-notebook 6.5.2


closing as outdated
Repository: Giskard-AI/giskard-oss
Issue: Disable the text explanation tab when there is no textual variable
Root cause: Unimplemented Feature Gap

I’ve always wondered, do we have any way of knowing when to show the explanation “text” tab ? I’m on a model with no text and it feels a bit weird to have a tab that isn’t usable
<img width="1418" alt="image" src="https://user-images.githubusercontent.com/14276335/211539931-e647fb99-40d8-46c5-8f48-1fc4e17fa23a.png">

Idea of Hugo: disable the tab with a hover message to avoid having a single tab
Repository: Giskard-AI/giskard-oss
Issue: Export and Import Projects
GitHub labels: enhancement

p:major

## 🚀 Feature Request

Possibility to export a project into a .zip file and then to import it in any instance of Giskard running.
Every part of the project should be included such as the feedbacks, the testsuites, the models and the dataset...

## 🔈 Motivation

Easily share project between Giskard instances.
Repository: Giskard-AI/giskard-oss
Issue: Inspection progress-bar displaying `NaN`
Root cause: Algorithmic Error

        I really like this bar addition, can we just find an alternative to `NaN`?
<img width="1588" alt="Screenshot 2023-01-11 at 11 23 27" src="https://user-images.githubusercontent.com/32709181/211782323-3e24f419-edab-48e6-8fb4-d872f8407ee4.png">

_Originally posted by @rabah-khalek in https://github.com/Giskard-AI/giskard/issues/613#issuecomment-1378535910_
      

Hey Rabah, do you have any specific scenario where this happens ? I can't find a reproductible case on my end

This is currently happening in the demo project of `German Credit Scoring`, in `models > Inspect > Inspect`. The inspection window I took screenshot of, disappears very quickly (so it's not crashing or anything) it's just that it's displaying a NaN.

I understand where the issue happens in the UI, but for some reason on my end testing with every dataset I always get a number and no NaN 🤔 

Then it's a byproduct of working from https://github.com/Giskard-AI/giskard/pull/618 (I'm not on `main`). Let's put a pin in it until we merge. Thanks!

closing as outdated
Repository: Giskard-AI/giskard-oss
Issue: Checking the prediction_function signature in case of non-pandas df input
Root cause: Validation Gap

We currently have:
```
    def upload_model_and_df(
            self,
            prediction_function: Callable[[pd.DataFrame], Iterable[Union[str, float, int]]],
            etc...
```
This hint should do the following (see example below):
```
def greeting(name: str) -> str:
    return 'Hello ' + name

greeting(1.5)
>>>
Traceback (most recent call last):
File "<string>", line 7, in <module>
  File "<string>", line 5, in greeting
TypeError: can only concatenate str (not "float") to str
```
but... giving a `prediction_function` that has a list (instead of pandas) doesn't produce an error:
```
def prediction_function(input: List[int]):
    return input
    

def test_func(prediction_function: Callable[[pd.DataFrame], Iterable[Union[str, float, int]]]):
    print(prediction_function)
    
test_func(prediction_function)
```

This was observed by Etam, that had a problem with the upload due to the fact that the input of their prediction_function wasn't a pandas dataframe

it seems to me a rare case where the user is aware that he should give a pandas.dataframe to give to Giskard, but keeps their prediction_function input as xgb.matrix

It's difficult to make a check other than the one we did here:
```python
    def _validate_model_execution(self, prediction_function, df: pd.DataFrame, model_type,
                                  classification_labels=None, target=None) -> None:
        if target is not None and target in df.columns:
            df = df.drop(target, axis=1)
        try:
            prediction_function(df.head(1))
        except Exception:
            raise ValueError("Invalid prediction_function input.\n"
                             "Please make sure that prediction_function(df.head(1)) does not return an error "
                             "message before uploading in Giskard")
```
If they got here, it means that they passed to `upload_model_and_df` a `pandas.dataframe`
Repository: Giskard-AI/giskard-oss
Issue: Slicing: compute chunk size dynamically during filtering
Root cause: Unimplemented Feature Gap

When slicing, we currently send a fixed number of rows each "chunk" to the ML worker. 

We can add logic that better computes a number to send so that we reduce the number of chunks sent. One option could be to find the dataset's size and number of rows, find the average rows' size and go from there

Outdated
Repository: Giskard-AI/giskard-oss
Issue: Make it possible to disconnect an ML Worker from the UI
GitHub labels: enhancement

It should be possible to disconnect an ML Worker (or workers once we support multiple ones connected) from the UI. Currently if the connection between backend and ML Worker fails, the worker keeps trying to reconnect until it succeeds.  
In order to stop ML Worker gracefully, we could send a "stop" command that would lead to the worker process exit
Repository: Giskard-AI/giskard-oss
Issue: Feedback modal doesn't open for the second time after it's closed

After the feedback modal is closed, the url isn't reset to `/feedbacks` and is still `/feedbacks/{id}`

@Googleton could you have a look at this small issue whenever you have time?

Do you have more details for a reproduction? I can't get it to happen locally, I'm on main.
EDIT: Turns out it was with the escape button 

Duplicate of #544 

Good catch @kevinmessiaen, the first one was originally created 9 months ago 🙈
Repository: Giskard-AI/giskard-oss
Issue: make `argmax` the function by default, but give the user the option to change it
Root cause: Unimplemented Feature Gap

              shouldn't we make `argmax` the function by default, but give the user the option to change it?

_Originally posted by @rabah-khalek in https://github.com/Giskard-AI/giskard/pull/618#discussion_r1064665590_
            

related to the new API: https://github.com/Giskard-AI/giskard/pull/618

There's no case scenario I can think of where a user would pick something else than the highest probability output.
Repository: Giskard-AI/giskard-oss
Issue: can't upload to "http://dev.giskard.ai" anymore
Root cause: Interface Contract Mismatch

### Issue Type

Bug

### Source

source

### Giskard Library Version

1.7,0

### Giskard Server Version

1.3.0

### OS Platform and Distribution

dev.giskard.ai

### Python version

3.8

### Installed python packages

_No response_

### Current Behaviour?

```shell
we used to be able to update to "http://dev.giskard.ai"
```


### Standalone code OR list down the steps to reproduce the issue

```shell
try to upload any project to "http://dev.giskard.ai"
```


### Relevant log output

```shell
Model successfully uploaded to project key 'None' with ID = None. It is available at http://dev.giskard.ai 
Traceback (most recent call last):
  File "Giskard_test.py", line 75, in <module>
    test_suite_id = credit_scoring.list_test_suites()[0]['id']
  File "/Users/rak/Documents/giskard-master/python-client/giskard/client/project.py", line 117, in list_test_suites
    return [{"id": t["id"], "name": t["name"]} for t in res]
  File "/Users/rak/Documents/giskard-master/python-client/giskard/client/project.py", line 117, in <listcomp>
    return [{"id": t["id"], "name": t["name"]} for t in res]
TypeError: string indices must be integers
```
```


Closing as outdated
Repository: Giskard-AI/giskard-oss
Issue: classification_label should support int value especialy for binary classification with 1 and 0
Root cause: Validation Gap

asked by Instant Gaming

TODO: to check what doesn't work

I recently used int `classification_label` in the sst2 notebook, and it did work. 
<img width="1111" alt="Screenshot 2023-02-07 at 10 14 44" src="https://user-images.githubusercontent.com/32709181/217202228-0039c635-277c-4cc7-a167-3f5979d0d75b.png">
in inspection:
<img width="1594" alt="Screenshot 2023-02-07 at 10 24 40" src="https://user-images.githubusercontent.com/32709181/217204514-1b268df7-0c33-4de9-a0f8-1727caf9a3f9.png">

Are we sure we noted correctly the issue they were having @jmsquare?
Repository: Giskard-AI/giskard-oss
Issue: For binary classification, prediction_function should support an output of 1D array (and not necessarily 2D array)
Root cause: Documentation Deficiency


This will become irrelevant in the new API as the user won't have to provide the predictions in a certain shape. Just the clf.

Some models could still return a 1D array, pytorch or custom models for example

you're right, you're right. Ok noted. Do we want a solution for this in 1.4.0?

nope

also in the case of binary classification, if the user give us 1D array, he has to tell us (or we need to tell him) to which classification_label it belongs. The benefit of 2D array is that it follows the same order as the classification_labels.

In the documentation, we should clear mention that the first "classification_label" correspond to the 1D array that is provided
Repository: Giskard-AI/giskard-oss
Issue: ml-worker keeps occupying port even after being terminated
Root cause: Resource Mishandling

### Issue Type

Bug

### Source

source

### Giskard Library Version

1.7.0

### Giskard Server Version

1.3.0

### OS Platform and Distribution

mac

### Python version

3.8

### Installed python packages

_No response_

### Current Behaviour?

```shell
If ml-worker gets disrupted (not by a ctrl+c):

set with shape (894, 20) executed in 0:00:00.024833
2023-01-29 15:10:06,917 pid:44857 ThreadPoolExecutor-0_0 giskard.ml_worker.utils.logging INFO     Test test_disparate_impact executed in 0:00:00.917087
[1]    44857 terminated  giskard worker start -h localhost
```
re-running it, displays this:
```
iskard worker start -h localhost
2023-01-29 15:28:09,485 pid:47520 MainThread giskard.cli  INFO     Starting ML Worker client
2023-01-29 15:28:09,485 pid:47520 MainThread giskard.cli  INFO     Python: /Users/rak/Documents/giskard-master/python-client/.venv/bin/python (3.8.16)
2023-01-29 15:28:09,485 pid:47520 MainThread giskard.cli  INFO     Giskard Home: /Users/rak/giskard-home
2023-01-29 15:28:09,487 pid:47520 MainThread giskard.cli  WARNING  Another ML Worker client for localhost:40051 is already running with PID: 45508. Not starting a new one.

```

closing as outdated