Repository: Cloud-CV/EvalAI
Issue: Problem Building using docker-compose 
Root cause: Environment Incompatibility

I am trying to build EvalAI on my Macbook Air M1 and getting the below error at the end. 
<img width="763" alt="Screen Shot 2022-01-24 at 23 54 56" src="https://user-images.githubusercontent.com/38133770/150927125-6fd231ed-12cb-46ed-bc1a-f01c29091d9c.png">


Hi @Krutarth-Bhatt, 

We don't officially support EvalAI development environment for Macbook M1. Some contributors have found out a fix for this issue by adding this change in nodejs dockerfile

```
FROM --platform=linux/amd64 node:12.4.0
```

If this doesn't work you can post the issue on our gitter channel where someone can help you resolve this issue.

Can you please tell me in which while I need to add this??

Hi @Krutarth-Bhatt 

Please add above line to two files present at:
1. docker/dev/nodejs/Dockerfile
2. docker/dev/nodejs_v2/Dockerfile

If this doesn't work you can post the issue on our gitter channel where someone can help you resolve this issue.

Hey, 
Thanks a lot for the response. 
I am getting this message while I am trying to compile after the change that you suggested. 
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?


Hey,

**If docker isn't running**
Check if docker is running or not using:   *sudo service docker status*
If not running, run this:                               *sudo service docker start*

**If permission of docker unix socket is problem**
Check about permission of socket:         *sudo ls -la /var/run/docker.sock*
Grant the necessary ownership using :   *sudo chown [username]:docker /var/run/docker.sock*

Sorry I forgot to turn on the docker. 
No I am getting this error:
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Hi @Ram81
I am facing the docker error  shown in the first image as well.

```
 > [evalai_worker internal] load metadata for docker.io/library/evalai_django:latest:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
```



Hi @gchhablani 

I had the issue before and found this link (https://stackoverflow.com/questions/64221861/an-error-failed-to-solve-with-frontend-dockerfile-v0/66695181#66695181) works for me.

> Hi @Ram81 I am facing the docker error shown in the first image as well.
> 
> ```
>  > [evalai_worker internal] load metadata for docker.io/library/evalai_django:latest:
> ------
> failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
> ```

Hi,I am also encountering this issue.Do you sloved this issue?

Same issue on linux

@FrancescoSaverioZuppichini
Did [this fix](https://github.com/Cloud-CV/EvalAI/issues/3682#issuecomment-1151906090) not work for you?

Attaching some output from `DOCKER_BUILDKIT=0 docker-compose up --build`


```
...
Removing intermediate container 38088e09b835
 ---> 2b7f4ec3b136
Step 10/11 : ADD . /code
 ---> 2275cd802cda
Step 11/11 : CMD ["./docker/wait-for-it.sh", "django:8000", "--", "python", "-m", "scripts.workers.submission_worker"]
 ---> Running in f7e75dec37de
Removing intermediate container f7e75dec37de
 ---> 531320dc5549
Successfully built 531320dc5549
Successfully tagged evalai-worker_py3_8:latest
Sending build context to Docker daemon  91.32MB
Step 1/20 : FROM node:14.20.0
14.20.0: Pulling from library/node
86467c57892b: Pull complete 
2b34aaa3cba7: Pull complete 
5a443ab54eaa: Pull complete 
e6cacf42b567: Pull complete 
2334d631be5e: Pull complete 
9707f1b650a1: Pull complete 
0940c720eb18: Pull complete 
7ecc5f2d0437: Pull complete 
3b02042b8ccb: Pull complete 
Digest: sha256:6adfb0c2a9db12a06893974bb140493a7482e2b3df59c058590594ceecd0c99b
Status: Downloaded newer image for node:14.20.0
 ---> 4f585a7a4342
Step 2/20 : RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
 ---> Running in 2d5a4895c907
Warning: apt-key output should not be parsed (stdout is not a terminal)
OK
Removing intermediate container 2d5a4895c907
 ---> 525727702f34
Step 3/20 : RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
 ---> Running in 676dd40fe4c9
Removing intermediate container 676dd40fe4c9
 ---> d02b988e76e3
Step 4/20 : RUN apt-get update && apt-get install -yq google-chrome-stable
 ---> Running in 8e318c6446a8
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
Get:2 http://dl.google.com/linux/chrome/deb stable InRelease [1811 B]
Get:3 http://deb.debian.org/debian-security buster/updates InRelease [34.8 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [56.6 kB]
Err:1 http://deb.debian.org/debian buster InRelease
  At least one invalid signature was encountered.
Err:2 http://dl.google.com/linux/chrome/deb stable InRelease
  At least one invalid signature was encountered.
Err:3 http://deb.debian.org/debian-security buster/updates InRelease
  At least one invalid signature was encountered.
Err:4 http://deb.debian.org/debian buster-updates InRelease
  At least one invalid signature was encountered.
Reading package lists...
W: GPG error: http://deb.debian.org/debian buster InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian buster InRelease' is not signed.
W: GPG error: http://dl.google.com/linux/chrome/deb stable InRelease: At least one invalid signature was encountered.
E: The repository 'http://dl.google.com/linux/chrome/deb stable InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian-security buster/updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian-security buster/updates InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian buster-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian buster-updates InRelease' is not signed.
2 errors occurred:
        * Status: pull access denied for evalai_django, repository does not exist or may require 'docker login': denied: requested access to the resource is denied, Code: 1
        * Status: The command '/bin/sh -c apt-get update && apt-get install -yq google-chrome-stable' returned a non-zero code: 100, Code: 100
        ```

> @FrancescoSaverioZuppichini Did [this fix](https://github.com/Cloud-CV/EvalAI/issues/3682#issuecomment-1151906090) not work for you?

Doesn't seem to fix the issue, not sure but I am on `Ubuntu 22.04`

Seems to be a permission issue for `evalai_django`

building the containers works individually 

`docker build -t evalai_nodejs -f docker/dev/nodejs/Dockerfile .` -> worked
`docker build -t evalai_django -f docker/dev/django/Dockerfile .` -> worked
`docker build -t evalai_worker -f docker/dev/worker/Dockerfile .` -> worked
`.... same for py3.8` -> worked

So I rebuild everything with `docker-compose` and everything is fine now - no idea :face_with_head_bandage: 



Fresh installation on fresh Ubuntu machine, by running `DOCKER_BUILDKIT=0 docker-compose up --build`

```
Removing intermediate container b6a7f5dee812
 ---> fba7977f2215
Step 19/24 : CMD ["gulp", "dev:runserver"]
 ---> Running in bfa063bb2665
Removing intermediate container bfa063bb2665
 ---> cd0cc1598748
Step 20/24 : EXPOSE 8888
 ---> Running in c25e658ce9cf
Removing intermediate container c25e658ce9cf
 ---> fb7447f8523f
Step 21/24 : LABEL com.docker.compose.image.builder=classic
 ---> Running in c39ac31714df
Removing intermediate container c39ac31714df
 ---> 9fca23486383
Step 22/24 : LABEL com.docker.compose.project=evalai
 ---> Running in abb0367d95d7
Removing intermediate container abb0367d95d7
 ---> 95ccf1446a16
Step 23/24 : LABEL com.docker.compose.service=nodejs
 ---> Running in c196735007b2
Removing intermediate container c196735007b2
 ---> c363ab9d918b
Step 24/24 : LABEL com.docker.compose.version=2.16.0
 ---> Running in abfa201863a9
Removing intermediate container abfa201863a9
 ---> af91b55c3fa1
Successfully built af91b55c3fa1
Successfully tagged evalai-nodejs:latest
1 error occurred:
        * Status: pull access denied for evalai_django, repository does not exist or may require 'docker login': denied: requested access to the resource is denied, Code: 1
 ```

I am also having exactly the same issue building the project on Macbook Air M2

This thread [here](https://github.com/docker/compose/issues/8449) sheds more light on the problem, seems this is docker/buildkit related and not specific to OS. It also lists multiple solutions. **Unfortunately some work for some.**

### For my Mac M1 Air
I disabled docker compose v2 from docker dashboard
<img width="504" alt="Screenshot 2023-03-25 at 6 50 06 PM" src="https://user-images.githubusercontent.com/41443146/227723431-c5d18172-a9b7-4ba6-869c-e826df9c803d.png">

And then quit docker desktop manually and started it again to be sure that settings are updated. And things worked not completely though. I still see issues while installing `google-chrome-stable` 

and @Ram81 's fix 

`FROM --platform=linux/amd64 node:12.4.0` 

Throws the following error
<img width="856" alt="Screenshot 2023-03-25 at 9 28 10 PM" src="https://user-images.githubusercontent.com/41443146/227731580-026a66b1-36a1-4a54-b058-4a4e9f357193.png">


Here's what solved this issue

<img width="647" alt="Screenshot 2023-03-25 at 10 34 19 PM" src="https://user-images.githubusercontent.com/41443146/227731625-35c0fea4-2f89-498d-91b2-c572331a36f8.png">

This was added in `docker/dev/nodejs/Dockerfile`

But all's not well yet, I am getting the following error which seems to be coming from a wrong requirement spec for [matlplotlib as v3.3.3](https://github.com/matplotlib/matplotlib/blob/v3.3.3/setup.py) shouldn't throw this error
<img width="673" alt="Screenshot 2023-03-25 at 10 47 36 PM" src="https://user-images.githubusercontent.com/41443146/227732007-78bffc15-c14b-4d71-8cc1-ad6d570f2171.png">
<img width="615" alt="Screenshot 2023-03-25 at 10 48 09 PM" src="https://user-images.githubusercontent.com/41443146/227732013-79cf72ea-02e6-4163-80f2-802f52b2b643.png">

Okay my bad the issue was right in from of me, it's not coming from matplotlib but [numpy-1.24.2](https://github.com/numpy/numpy/blob/85f38ab180ece5290f64e8ddbd9cf06ad8fa4a5e/setup.py#L20)
Not sure how it gets here as couldn't find any requirements.txt referring to this.

Okay following worked for me
<img width="558" alt="Screenshot 2023-03-26 at 1 15 55 AM" src="https://user-images.githubusercontent.com/41443146/227738590-31af893d-a2a4-44e3-9191-4b626303cb8f.png">

This was added in `docker/dev/worker/Dockerfile`

**I now have the dev stage evalai running on my machine**

Here's a summary of what worked for me:
1. Disabling `Use Docker Compose V2` from docker dashboard and manually quitting and starting docker again.
2. Replacing line 1 in `docker/dev/nodejs/Dockerfile` with
```
FROM --platform=linux/amd64 node:14.20.0
```
3. And replacing line 7 in `docker/dev/worker/Dockerfile` with
```
RUN pip install -U cffi service_identity cython==0.29 numpy==1.21.0
```

Details of my setup/config:
Branch: master @ https://github.com/Cloud-CV/EvalAI/commit/fbfc7f04cf8570704fd41808c0c5f5cfed518505
Machine: Macbook Air M1 2020
OS Version: Ventura 13.2.1 (22D68)
Stage: dev

Closing this since #4498 solves this.