Package: tacv
from tacv.fileUtils import ThreadedDownload# TACV - A mini package for daily tasksInstallationpipinstalltacvSome specific functions depend on other libraries (.e.gmlflow,torch,pytorch-lightning). Be forgiving and install the necessary libraries if required.ExamplesLoRA trainingAdding LoRA to your model and start training as usual with minimal code changes.Define a boilerplate class that inherit both AnyLoRA and the model you want to play, in this case GPTNeoForCausalLMfromtacv.torchimportAnyLoRAfromtransformersimportGPTNeoForCausalLMclassGPTNeoLoRA(GPTNeoForCausalLM,AnyLoRA):def__init__(self,config):GPTNeoForCausalLM.__init__(self,config)Add LoRA to layers that you think it could bring the training benefit, here we choose the Key and Value matrices in
all self-attention blocks# init your LoRA modelmodel=GPTNeoLoRA.from_pretrained("EleutherAI/gpt-neo-1.3B",low_cpu_mem_usage=True)model.init_lora(rank=8,layers_contain=["k_proj","v_proj"])# Also give "layers_end_with" and "layers_start_with" a trymodel.init_lora(rank=4,layers_end_with="q_proj",layers_start_with="wte")# drop grad of all other layers before trainingmodel.mark_only_lora_as_trainable(bias="none")The rest of training is as usual, just as you have done before using AnyLoRA.Save and load LoRA weights.# save LoRAmodel.save_lora("lora.pth")# load LoRA for inferencemodel.load_lora("lora.pth")Want more control over the training and mode behavior, visit my example of fine-tuning a GPT-Neo model for Vietnamese
attacv/torch/example_lora.py2D Object DetectionFor now, CenterNet is supported. However, use it as prototype purpose only, there is no official benchmark on accuracy.Train with your own datasetFirst, create a config file for training/model config, see full config attacv/detection/sample_config.yml.input_size:&input_size[224,448]max_object:&max_obj16num_classes:&num_classes5train_config:gpus:0# 0 means CPU, N means using N available GPU(s) for trainingepoch:600batch_size:32shuffle:Truenum_workers:4learning_rate:0.0001lr_decay_milestones:[80,160]lr_decay_gamma:0.5weight_decay:0.01checkpoint_frequency:1amp:Trueunfreeze_bbone_epoch:200initial_denom_lr:5loss_hm_offset_wh_weights:[1,1,0.1]callback:monitor:"val_loss"dirpath:"logs/exp_name_1"save_top_k:20mode:"min"val_config:batch_size:1checkpoint:""model:num_classes:*num_classesbackbone_layers:18head_conv_channel:64max_object:*max_objinput_shape:*input_sizeSecond, create your own Dataset class that returns data as described in the__getitem__()method, see following
example:fromtorch.utils.dataimportDatasetimporttorchclassMockDataset(Dataset):def__init__(self,max_objs,input_shape_HW):self.max_objs=max_objsself.input_shape_HW=input_shape_HWdef__getitem__(self,item):# read your image# image = cv2.imread(your image path)# do any transform operation, then return a tensorimage=torch.rand(3,self.input_shape_HW[0],self.input_shape_HW[1])# Shape = (3, InputShape, W)annos=torch.rand(self.max_objs,5)# Shape = (MaxObjs x 5) , each row presents for (x,y,w,h,class_id) relative to input shapemasks=torch.zeros(self.max_objs)# Shape = (MaxObjs,)  each value is False or True (1 indicates having object)masks[0:3]=Truereturn{"image":image,"annos":annos,"masks":masks}def__len__(self):return1000InitCenterNetTrainerand here we gofromtacv.detectionimportCenterNetTrainerfromtorch.utils.dataimportrandom_splitconfig_path="tacv/detection/sample_config.yml"max_objs=16# read from config fileinput_shape=(224,448)# read from config filedataset=MockDataset(max_objs=max_objs,input_shape_HW=input_shape)# Replace with your custom datasettrain_set,val_set=random_split(dataset,[900,100])trainer=CenterNetTrainer(train_set,val_set,config_path)trainer.train()Inference on single imageimportcv2importtorchfromtacv.detectionimportload_centernet_model_with_configfromtacv.detectionimportinferif__name__=="__main__":config_path="/home/tu/Projects/PycharmProjects/TaCV/tacv/detection/sample_config.yml"device="cuda:0"iftorch.cuda.is_available()else"cpu"model=load_centernet_model_with_config(config_path,load_bbone_pretrained=False)model.load_state_dict("your_checkpoint.pth")model.eval()model.to(device)image=cv2.imread("your_image.png")bboxes=infer(model,image,device)print(bboxes)# list of detections in format (xc,yc,w,h,class_id,confidence_score)File utilsFile utilsGet all file paths from a directoryfromtacv.fileUtilsimportget_all_filesfile_paths=get_all_files("dir_name")Returns a list of file absolute paths, for example['./venvCondaTest/x86_64-conda_cos6-linux-gnu/bin/ld', './venvCondaTest/conda-meta/_libgcc_mutex-0.1-main.json', './venvCondaTest/conda-meta/xz-5.2.5-h7b6447c_0.json', './venvCondaTest/conda-meta/wheel-0.37.1-pyhd3eb1b0_0.json', './venvCondaTest/conda-meta/setuptools-58.0.4-py36h06a4308_0.json', './venvCondaTest/conda-meta/ca-certificates-2021.10.26-h06a4308_2.json', './venvCondaTest/conda-meta/readline-8.1.2-h7f8727e_1.json', './venvCondaTest/conda-meta/sqlite-3.37.2-hc218d9a_0.json', './venvCondaTest/conda-meta/libgcc-ng-9.3.0-h5101ec6_17.json', './venvCondaTest/conda-meta/ncurses-6.3-h7f8727e_2.json']Save/load json data to/from filefromtacv.fileUtilsimportsave_json,load_jsonjson_file="myfile.json"json_data={"name":"Ta","age":100}# save jsonsave_json(json_file,json_data)# load jsonjson_data=load_json(json_file)Multi-threaded file downloaderfromtacv.fileUtilsimportThreadedDownloadurls=['http://localhost:11223/210207000111980_3_124202.jpg','http://localhost:11223/210430000373373_3_407200.jpg','http://localhost:11223/210426000324979_3_356188.jpg','http://localhost:11223/200819000060994_3_78314.png']downloader=ThreadedDownload(urls,"/home/tu/Desktop/LabelStudio",False,thread_count=3,url_tries=3)print(f'Downloading{len(urls)}files')downloader.run()print(f'Downloaded{len(downloader.report["success"])}of{len(urls)}')CV2 VisualizationDraw 2D points onto an imageimportcv2fromtacv.visualimportdraw_pointsimage=cv2.imread("myimage.jpg")points=[(18,19),(55,55),(102,22),(66,22)]draw_points(image,points,circular=True,color=(0,255,0),thickness=2)cv2.imwrite("new_image.jpg",image)Video and image utilsSynthesize a video from imagesfromtacv.videoimportimages2videoimage_dir="my_images"# directory containing images in the same formatvideo_path="tacv_test.mp4"# path to save the synthesized video# common use caseimages2video(image_dir,video_path,fps=24,image_ext=None,sort=False)Parameters:fps: default = 24image_ext: a string, specify image extension to synthesize the video, for example (jpg,png,...). If it
isNone. All images will be grabbed. Default isNone.sort:TrueorFalse. Indicate if the images should be sorted by name before synthesizing the video. Default
isTrue.Extract images from a videofromtacv.videoimportvideo2imagesvideo_path="tacv_test.mp4"# path to video to be extracted to imagesimage_dir="my_images"# directory to save the extracted imagesvideo2images(video_path,image_dir,exist_ok=False,image_ext="jpg",verbose=True)Parameters:exist_ok: default is False. Ifimage_diralready contains images and this flag isFalse. The process will be
cancel, otherwise it continues.image_ext: a string, specify image extension, for example (jpg,png,...). If it isNone. All images will be
grabbed. Default isNone.verbose:TrueorFalse. Set it toTrueto view the extracting process. Default isTrue.Geometry utilsCalculate 2D IOU of two polygonsfromtacv.geometryimportiou_2dpolygon_1=[[0,0],[10,10],[0,10]]polygon_2=[[0,20],[10,10],[0,0]]print(iou_2d(polygon_1,polygon_2))Command Line InterfaceSynthesize a video from imagestacv_i2vimage_dirvideo_path[optional:fpsimage_ext]Extract images from a videotacv_v2ivideo_pathimage_dirFor moreVisit args description in source codeVisittest.pyfile