geolocal/StreetCLIP · Great work, and thanks for sharing! I wrote a pipeline to reproduce the results from the paper, but the results are different.

Zilun

Dec 7, 2023

•

edited Dec 7, 2023

Would you mind having a look at what I missed?

Thanks.

The repo

https://github.com/zilunzhang/StreetCLIP-Repoduce/blob/main/eval_img2gps.py

Result on IM2GPS3K

n=2997

Model	Source	1KM	25KM	200KM	750KM	2,500KM
CLIP@ViT-L-14-336	Paper	-	19.5	34.0	60.0	78.1
CLIP@ViT-L-14-336	OpenAI's CLIP-reproduce	4.07	20.09	31.90	54.72	72.07
StreetCLIP@ViT-L-14-336	Paper	-	22.4	37.4	61.3	80.4
StreetCLIP@ViT-L-14-336	StreetCLIP-reproduce	4.24	21.79	34.73	55.52	74.84
CLIP@ViT-B-32	OpenAI's CLIP	1.67	8.88	14.65	32.87	53.72
CLIP@ViT-B-16	OpenAI's CLIP	2.47	12.41	20.39	39.71	61.86
CLIP@ViT-L-14	OpenAI's CLIP	3.34	17.68	28.86	51.55	68.90
CLIP@ViT-H-14	OpenCLIP	3.94	18.69	30.60	51.95	71.10

Result on IM2GPS

n=237

Model	Source	1KM	25KM	200KM	750KM	2,500KM
CLIP@ViT-L-14-336	Paper	-	27.0	42.2	71.7	86.9
CLIP@ViT-L-14-336	OpenAI's CLIP-reproduce	4.64	26.58	40.08	63.71	80.17
StreetCLIP@ViT-L-14-336	Paper	-	28.3	45.1	74.7	88.2
StreetCLIP@ViT-L-14-336	StreetCLIP-reproduce	5.49	28.27	42.62	67.51	80.17
CLIP@ViT-B-32	OpenAI's CLIP	2.11	16.46	26.58	46.41	66.24
CLIP@ViT-B-16	OpenAI's CLIP	2.53	19.83	31.65	52.74	71.31
CLIP@ViT-L-14	OpenAI's CLIP	4.22	24.05	35.44	58.65	77.63
CLIP@ViT-H-14	OpenCLIP	5.49	29.54	44.30	65.82	79.75

Zilun changed discussion title from Great work, and thanks for sharing! I wrote a pipeline to reproduce the results from the paper, but the results are different. Would you mind having a look on what I missed? to Great work, and thanks for sharing! I wrote a pipeline to reproduce the results from the paper, but the results are different. Dec 7, 2023

makeandyou

Jan 27

This comment has been hidden