bconsolvo commited on
Commit
9312cfb
1 Parent(s): 2c15921

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -10,6 +10,9 @@ datasets:
10
  ## Model Details: 90% Sparse BERT-Base (uncased) Prune Once For All
11
  This model is a sparse pre-trained model that can be fine-tuned for a wide range of language tasks. The process of weight pruning is forcing some of the weights of the neural network to zero. Setting some of the neural network's weights to zero results in sparse matrices. Updating neural network weights does involve matrix multiplication, and if we can keep the matrices sparse while retaining enough important information, we can reduce the overall computation overhead. The "sparse" in the title of the model indicates a ratio of sparsity in the weights; for more details, you can read [Zafrir et al. (2021)](https://arxiv.org/abs/2111.05754).
12
 
 
 
 
13
  | Model Detail | Description |
14
  | ----------- | ----------- |
15
  | Model Authors - Company | Intel |
@@ -21,9 +24,6 @@ This model is a sparse pre-trained model that can be fine-tuned for a wide range
21
  | License | Apache 2.0 |
22
  | Questions or Comments | [Community Tab](https://huggingface.co/Intel/bert-base-uncased-sparse-90-unstructured-pruneofa/discussions) and [Intel Developers Discord](https://discord.gg/rv2Gp55UJQ)|
23
 
24
- Visualization of Prunce Once for All method from [Zafrir et al. (2021)](https://arxiv.org/abs/2111.05754). More details can be found in their paper.
25
- ![Zafrir2021_Fig1.png](https://s3.amazonaws.com/moonup/production/uploads/6297f0e30bd2f58c647abb1d/nSDP62H9NHC1FA0C429Xo.png)
26
-
27
  | Intended Use | Description |
28
  | ----------- | ----------- |
29
  | Primary intended uses | This is a general sparse language model; in its current form, it is not ready for downstream prediction tasks, but it can be fine-tuned for several language tasks including (but not limited to) SQuADv1.1, QNLI, MNLI, SST-2 and QQP. |
 
10
  ## Model Details: 90% Sparse BERT-Base (uncased) Prune Once For All
11
  This model is a sparse pre-trained model that can be fine-tuned for a wide range of language tasks. The process of weight pruning is forcing some of the weights of the neural network to zero. Setting some of the neural network's weights to zero results in sparse matrices. Updating neural network weights does involve matrix multiplication, and if we can keep the matrices sparse while retaining enough important information, we can reduce the overall computation overhead. The "sparse" in the title of the model indicates a ratio of sparsity in the weights; for more details, you can read [Zafrir et al. (2021)](https://arxiv.org/abs/2111.05754).
12
 
13
+ Visualization of Prunce Once For All method from [Zafrir et al. (2021)](https://arxiv.org/abs/2111.05754):
14
+ ![Zafrir2021_Fig1.png](https://s3.amazonaws.com/moonup/production/uploads/6297f0e30bd2f58c647abb1d/nSDP62H9NHC1FA0C429Xo.png)
15
+
16
  | Model Detail | Description |
17
  | ----------- | ----------- |
18
  | Model Authors - Company | Intel |
 
24
  | License | Apache 2.0 |
25
  | Questions or Comments | [Community Tab](https://huggingface.co/Intel/bert-base-uncased-sparse-90-unstructured-pruneofa/discussions) and [Intel Developers Discord](https://discord.gg/rv2Gp55UJQ)|
26
 
 
 
 
27
  | Intended Use | Description |
28
  | ----------- | ----------- |
29
  | Primary intended uses | This is a general sparse language model; in its current form, it is not ready for downstream prediction tasks, but it can be fine-tuned for several language tasks including (but not limited to) SQuADv1.1, QNLI, MNLI, SST-2 and QQP. |