aarondevstack commited on
Commit
d8d87bc
·
1 Parent(s): 14ff16b

chore: update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -27
README.md CHANGED
@@ -2,34 +2,51 @@
2
  license: apple-ascl
3
  library_name: coreml
4
  tags:
5
- - apple
6
- - amlr
7
- - computer-vision
8
  - depth-estimation
9
  - visionos
10
- - macos
 
 
 
 
 
 
 
11
  ---
12
 
13
- # DepthPro: High-Resolution Monocular Depth (CoreML)
14
-
15
- This repository provides CoreML-optimized weights for **DepthPro**, adapted for on-device inference on visionOS and macOS.
16
-
17
- ## Configuration
18
- - **Resolution**: 512x512 (Optimized for <1s inference)
19
- - **Precision**: Float16
20
- - **Architecture**: Multi-scale ViT-Large with Sharp Boundary Refinement
21
-
22
- ## Performance (M2 Apple Vision Pro)
23
- - **Transform + Encoder**: ~700ms (GPU Fallback)
24
- - **Decoder + Depth Head**: ~150ms
25
- - **Total Pipeline**: < 1.0s per frame
26
-
27
- ## Citation
28
- If you use this model in your research or applications, please cite the original work:
29
- ```bibtex
30
- @article{Bochkovskii2024:DepthPro,
31
- title={DepthPro: Sharp Monocular Depth Estimation with Full-Resolution Refinement},
32
- author={Bochkovskii, Alexey and Fogosar, Ambron and Koltun, Vladlen},
33
- journal={arXiv preprint arXiv:2410.02073},
34
- year={2024}
35
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apple-ascl
3
  library_name: coreml
4
  tags:
 
 
 
5
  - depth-estimation
6
  - visionos
7
+ - apple-silicon
8
+ - amlr
9
+ - computer-vision
10
+ - depth-pro
11
+ - 512x512
12
+ - ane-optimized
13
+ extra_gated_heading: DepthPro CoreML (Fast 512px - ANE Optimized)
14
+ extra_gated_button_content: Access Model
15
  ---
16
 
17
+ # DepthPro CoreML (512x512 Real-Time)
18
+
19
+ This repository contains the **Fast (512x512)** version of the DepthPro model, specifically optimized for the **Apple Neural Engine (ANE)**.
20
+
21
+ DepthPro is a state-of-the-art monocular depth estimation model. This 512px version is designed for **Real-Time Previews** and high-speed video conversion on Apple Vision Pro and Apple Silicon Macs.
22
+
23
+ ## 🚀 Key Features
24
+ - **ANE Accelerated**: Leveraging the Apple Neural Engine for ultra-low power and high-speed inference.
25
+ - **Real-Time Performance**: Ideal for interactive parameter tuning (Max Disparity, Convergence Plane).
26
+ - **VisionOS Ready**: Fully compatible with Apple Vision Pro via the `DepthProPipeline`.
27
+
28
+ ## 📊 Performance & Requirements
29
+ | Metric | Specification |
30
+ | :--- | :--- |
31
+ | **Input Resolution** | 512 x 512 pixels |
32
+ | **Compute Units** | All (Optimized for ANE) |
33
+ | **Average Latency** | < 1.0s per frame (on M2/M3 chips) |
34
+ | **Target Use Case** | Real-time 3D Preview / Quick Video Conversion |
35
+
36
+ > [!TIP]
37
+ > This model is the best choice for the initial phase of your 3D conversion workflow, allowing for near-instant feedback while adjusting 3D rendering parameters.
38
+
39
+ ## 📦 Repository Contents
40
+ The repository contains the following core components:
41
+ 1. `DepthPro_transform.mlpackage`: Image preprocessing.
42
+ 2. `DepthPro_encoder.mlpackage`: Feature extraction (ANE Optimized).
43
+ 3. `DepthPro_decoder.mlpackage`: Multiresolution fusion.
44
+ 4. `DepthPro_depth.mlpackage`: Final depth output.
45
+
46
+ ## 🛠 Usage with Swift Transformers
47
+ You can download and cache this model dynamically using `swift-transformers`:
48
+
49
+ ```swift
50
+ let hub = Hub()
51
+ let modelDir = try await hub.snapshot(repoId: "aarondevstack/DepthPro-512x512-coreml")
52
+ // Load models from the downloaded directory