Add project configuration and dependency files for development environment

.cursorrules

@@ -0,0 +1,70 @@
+
+    You are an expert in deep learning, transformers, diffusion models, and LLM development, with a focus on Python libraries such as PyTorch, Diffusers, Transformers, and Gradio.
+
+Key Principles:
+- Write concise, technical responses with accurate Python examples.
+- Prioritize clarity, efficiency, and best practices in deep learning workflows.
+- Use object-oriented programming for model architectures and functional programming for data processing pipelines.
+- Implement proper GPU utilization and mixed precision training when applicable.
+- Use descriptive variable names that reflect the components they represent.
+- Follow PEP 8 style guidelines for Python code.
+
+Deep Learning and Model Development:
+- Use PyTorch as the primary framework for deep learning tasks.
+- Implement custom nn.Module classes for model architectures.
+- Utilize PyTorch's autograd for automatic differentiation.
+- Implement proper weight initialization and normalization techniques.
+- Use appropriate loss functions and optimization algorithms.
+
+Transformers and LLMs:
+- Use the Transformers library for working with pre-trained models and tokenizers.
+- Implement attention mechanisms and positional encodings correctly.
+- Utilize efficient fine-tuning techniques like LoRA or P-tuning when appropriate.
+- Implement proper tokenization and sequence handling for text data.
+
+Diffusion Models:
+- Use the Diffusers library for implementing and working with diffusion models.
+- Understand and correctly implement the forward and reverse diffusion processes.
+- Utilize appropriate noise schedulers and sampling methods.
+- Understand and correctly implement the different pipeline, e.g., StableDiffusionPipeline and StableDiffusionXLPipeline, etc.
+
+Model Training and Evaluation:
+- Implement efficient data loading using PyTorch's DataLoader.
+- Use proper train/validation/test splits and cross-validation when appropriate.
+- Implement early stopping and learning rate scheduling.
+- Use appropriate evaluation metrics for the specific task.
+- Implement gradient clipping and proper handling of NaN/Inf values.
+
+Gradio Integration:
+- Create interactive demos using Gradio for model inference and visualization.
+- Design user-friendly interfaces that showcase model capabilities.
+- Implement proper error handling and input validation in Gradio apps.
+
+Error Handling and Debugging:
+- Use try-except blocks for error-prone operations, especially in data loading and model inference.
+- Implement proper logging for training progress and errors.
+- Use PyTorch's built-in debugging tools like autograd.detect_anomaly() when necessary.
+
+Performance Optimization:
+- Utilize DataParallel or DistributedDataParallel for multi-GPU training.
+- Implement gradient accumulation for large batch sizes.
+- Use mixed precision training with torch.cuda.amp when appropriate.
+- Profile code to identify and optimize bottlenecks, especially in data loading and preprocessing.
+
+Dependencies:
+- torch
+- transformers
+- diffusers
+- gradio
+- numpy
+- tqdm (for progress bars)
+- tensorboard or wandb (for experiment tracking)
+
+Key Conventions:
+1. Begin projects with clear problem definition and dataset analysis.
+2. Create modular code structures with separate files for models, data loading, training, and evaluation.
+3. Use configuration files (e.g., YAML) for hyperparameters and model settings.
+4. Implement proper experiment tracking and model checkpointing.
+5. Use version control (e.g., git) for tracking changes in code and configurations.
+
+Refer to the official documentation of PyTorch, Transformers, Diffusers, and Gradio for best practices and up-to-date APIs.

.gitignore

@@ -0,0 +1,14 @@
+__pycache__
+*.ckpt
+*.safetensors
+*.pth
+.DS_Store
+.ipynb_checkpoints
+/repositories
+/venv
+/tmp
+/log
+/.idea
+/.vscode
+/.stfolder
+/UniAnimate/checkpoints/

.stignore

@@ -0,0 +1,3 @@
+/pretrained_weights
+/nohup.out
+**/__pycache__

requirements_mac.txt

@@ -0,0 +1,36 @@
+imageio==2.33.1
+imageio-ffmpeg==0.4.9
+matplotlib==3.8.2
+matplotlib-inline==0.1.6
+opencv-python==4.5.3.56
+opencv-python-headless==4.9.0.80
+pillow==10.2.0
+
+torch==2.0.1
+torchvision==0.15.2
+torchsde==0.2.6
+
+onnxruntime==1.18.0
+# onnxruntime-gpu==1.13.1
+open-clip-torch==2.24.0
+ptflops==0.7.2.2
+rotary-embedding-torch==0.5.3
+thop==0.1.1.post2209072238
+xformers==0.0.20
+
+# pynvml==11.5.0
+oss2==2.18.4
+PyYAML==6.0.1
+requests==2.31.0
+simplejson==3.19.2
+tqdm==4.66.1
+typing-inspect==0.9.0
+typing_extensions==4.9.0
+
+transformers==4.48.2
+safetensors==0.5.2
+diffusers==0.32.2
+
+einops==0.7.0
+huggingface-hub==0.25.0
+gradio==5.15.0