docs/tools.md

GPT4Tools can support 22 tools, including:

No.	Tools Name	Function
1	InstructPix2Pix	Style the image to be like the text.
2	Text2Image	Generate an image from an input text.
3	ImageCaptioning	Describe the input image.
4	Image2Canny	Detect the edge of the image
5	CannyText2Image	Generate a new real image from both the user description and a canny image.
6	Image2Line	Detect the straight line of the image.
7	Image2Hed	Detect the soft hed boundary of the image.
8	HedText2Image	Generate a new real image from both the user description.
9	Image2Scribble	Generate a scribble of the image.
10	ScribbleText2Image	Generate a new real image from both the user description and a scribble image.
11	Image2Pose	Detect the human pose of the image.
12	PoseText2Image	Generate a new real image from both the user description.
13	SegText2Image	Generate a new real image from both the user description and segmentations.
14	Image2Depth	Detect depth of the image.
15	DepthText2Image	Generate a new real image from both the user description and depth image.
16	Image2Normal	Detect norm map of the image.
17	NormalText2Image	Generate a new real image from both the user description and normal map.
18	VisualQuestionAnswering	Answer for a question based on an image.
19	Segmenting	Segment all the part of the image.
20	Text2Box	Detect or find out given objects in the picture.
21	ObjectSegmenting	Segment the certain objects in the picture.
22	ImageEditing	Remove and object or something from the photo.

You can customize the used tools by specifying {tools_name}_{devices} after args --load of gpt4tools.py. For example, enabling Text2Box, Segmenting, and ImageCaptioning:

python gpt4tools.py \
    --base_model <path_to_vicuna_with_tokenizer> \
    --lora_model <path_to_lora_weights> \
    --llm_device "cpu" \
    --load "Text2Box_cuda:0,Segmenting_cuda:0,ImageCaptioning_cuda:0"

More tools will be supported in the future!