Fabrice-TIERCELIN commited on
Commit
1b20dc8
1 Parent(s): b917128

Delete clipseg/Readme.md

Browse files
Files changed (1) hide show
  1. clipseg/Readme.md +0 -84
clipseg/Readme.md DELETED
@@ -1,84 +0,0 @@
1
- # Image Segmentation Using Text and Image Prompts
2
- This repository contains the code used in the paper ["Image Segmentation Using Text and Image Prompts"](https://arxiv.org/abs/2112.10003).
3
-
4
- **The Paper has been accepted to CVPR 2022!**
5
-
6
- <img src="overview.png" alt="drawing" height="200em"/>
7
-
8
- The systems allows to create segmentation models without training based on:
9
- - An arbitrary text query
10
- - Or an image with a mask highlighting stuff or an object.
11
-
12
- ### Quick Start
13
-
14
- In the `Quickstart.ipynb` notebook we provide the code for using a pre-trained CLIPSeg model. If you run the notebook locally, make sure you downloaded the `rd64-uni.pth` weights, either manually or via git lfs extension.
15
- It can also be used interactively using [MyBinder](https://mybinder.org/v2/gh/timojl/clipseg/HEAD?labpath=Quickstart.ipynb)
16
- (please note that the VM does not use a GPU, thus inference takes a few seconds).
17
-
18
-
19
- ### Dependencies
20
- This code base depends on pytorch, torchvision and clip (`pip install git+https://github.com/openai/CLIP.git`).
21
- Additional dependencies are hidden for double blind review.
22
-
23
-
24
- ### Datasets
25
-
26
- * `PhraseCut` and `PhraseCutPlus`: Referring expression dataset
27
- * `PFEPascalWrapper`: Wrapper class for PFENet's Pascal-5i implementation
28
- * `PascalZeroShot`: Wrapper class for PascalZeroShot
29
- * `COCOWrapper`: Wrapper class for COCO.
30
-
31
- ### Models
32
-
33
- * `CLIPDensePredT`: CLIPSeg model with transformer-based decoder.
34
- * `ViTDensePredT`: CLIPSeg model with transformer-based decoder.
35
-
36
- ### Third Party Dependencies
37
- For some of the datasets third party dependencies are required. Run the following commands in the `third_party` folder.
38
- ```bash
39
- git clone https://github.com/cvlab-yonsei/JoEm
40
- git clone https://github.com/Jia-Research-Lab/PFENet.git
41
- git clone https://github.com/ChenyunWu/PhraseCutDataset.git
42
- git clone https://github.com/juhongm999/hsnet.git
43
- ```
44
-
45
- ### Weights
46
-
47
- The MIT license does not apply to these weights.
48
-
49
- We provide two model weights, for D=64 (4.1MB) and D=16 (1.1MB).
50
- ```
51
- wget https://owncloud.gwdg.de/index.php/s/ioHbRzFx6th32hn/download -O weights.zip
52
- unzip -d weights -j weights.zip
53
- ```
54
-
55
-
56
- ### Training and Evaluation
57
-
58
- To train use the `training.py` script with experiment file and experiment id parameters. E.g. `python training.py phrasecut.yaml 0` will train the first phrasecut experiment which is defined by the `configuration` and first `individual_configurations` parameters. Model weights will be written in `logs/`.
59
-
60
- For evaluation use `score.py`. E.g. `python score.py phrasecut.yaml 0 0` will train the first phrasecut experiment of `test_configuration` and the first configuration in `individual_configurations`.
61
-
62
-
63
- ### Usage of PFENet Wrappers
64
-
65
- In order to use the dataset and model wrappers for PFENet, the PFENet repository needs to be cloned to the root folder.
66
- `git clone https://github.com/Jia-Research-Lab/PFENet.git `
67
-
68
-
69
- ### License
70
-
71
- The source code files in this repository (excluding model weights) are released under MIT license.
72
-
73
- ### Citation
74
- ```
75
- @InProceedings{lueddecke22_cvpr,
76
- author = {L\"uddecke, Timo and Ecker, Alexander},
77
- title = {Image Segmentation Using Text and Image Prompts},
78
- booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
79
- month = {June},
80
- year = {2022},
81
- pages = {7086-7096}
82
- }
83
-
84
- ```