codebert nazneen commited on
Commit
727b99c
1 Parent(s): 02583b5

model documentation (#4)

Browse files

- model documentation (214ef9fde11acb21901120fa246f3e7316cb5bd3)


Co-authored-by: Nazneen Rajani <nazneen@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +177 -3
README.md CHANGED
@@ -1,3 +1,177 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ ---
6
+
7
+ # Model Card for UniXcoder-base
8
+
9
+
10
+
11
+ # Model Details
12
+
13
+ ## Model Description
14
+ UniXcoder is a unified cross-modal pre-trained model that leverages multimodal data (i.e. code comment and AST) to pretrain code representation.
15
+
16
+ - **Developed by:** Microsoft Team
17
+ - **Shared by [Optional]:** Hugging Face
18
+ - **Model type:** Feature Engineering
19
+ - **Language(s) (NLP):** en
20
+ - **License:** Apache-2.0
21
+ - **Related Models:**
22
+ - **Parent Model:** RoBERTa
23
+ - **Resources for more information:**
24
+ - [Associated Paper](https://arxiv.org/abs/2203.03850)
25
+
26
+ # Uses
27
+
28
+
29
+
30
+ ## Direct Use
31
+ Feature Engineering
32
+
33
+
34
+ ## Downstream Use [Optional]
35
+
36
+ More information needed
37
+
38
+ ## Out-of-Scope Use
39
+
40
+ More information needed
41
+
42
+ # Bias, Risks, and Limitations
43
+
44
+ Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
45
+
46
+
47
+ ## Recommendations
48
+
49
+
50
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
51
+
52
+
53
+ # Training Details
54
+
55
+ ## Training Data
56
+
57
+ More information needed
58
+
59
+ ## Training Procedure
60
+
61
+ ### Preprocessing
62
+
63
+ More information needed
64
+
65
+ ### Speeds, Sizes, Times
66
+ More information needed
67
+
68
+ # Evaluation
69
+
70
+
71
+ ## Testing Data, Factors & Metrics
72
+
73
+ ### Testing Data
74
+ More information needed
75
+
76
+ ### Factors
77
+
78
+ The model creators note in the [associated paper](https://arxiv.org/abs/2203.03850):
79
+
80
+ > UniXcoder has slightly worse BLEU-4 scores on both code summarization and generation tasks. The main reasons may come from two aspects. One is the amount of NL-PL pairs in the pre-training data
81
+
82
+ ### Metrics
83
+
84
+ The model creators note in the [associated paper](https://arxiv.org/abs/2203.03850):
85
+
86
+ > We evaluate UniXcoder on five tasks over nine public datasets, including two understanding tasks, two generation tasks and an autoregressive task. To further evaluate the performance of code fragment embeddings, we also propose a new task called zero-shot code-to-code search.
87
+
88
+ ## Results
89
+
90
+ The model creators note in the [associated paper](https://arxiv.org/abs/2203.03850):
91
+
92
+ >Taking zero-shot code-code search task as an example, after removing contrastive learning, the performance drops from 20.45% to 13.73%.
93
+
94
+ # Model Examination
95
+
96
+ More information needed
97
+
98
+ # Environmental Impact
99
+
100
+
101
+
102
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
103
+
104
+ - **Hardware Type:** More information needed
105
+ - **Hours used:** More information needed
106
+ - **Cloud Provider:** More information needed
107
+ - **Compute Region:** More information needed
108
+ - **Carbon Emitted:** More information needed
109
+
110
+ # Technical Specifications [optional]
111
+
112
+ ## Model Architecture and Objective
113
+
114
+ More information needed
115
+
116
+ ## Compute Infrastructure
117
+
118
+ More information needed
119
+
120
+ ### Hardware
121
+
122
+ More information needed
123
+
124
+ ### Software
125
+
126
+ More information needed
127
+
128
+ # Citation
129
+
130
+
131
+ **BibTeX:**
132
+ ```
133
+ @misc{https://doi.org/10.48550/arxiv.2203.03850,
134
+ doi = {10.48550/ARXIV.2203.03850},
135
+
136
+ url = {https://arxiv.org/abs/2203.03850},
137
+
138
+ author = {Guo, Daya and Lu, Shuai and Duan, Nan and Wang, Yanlin and Zhou, Ming and Yin, Jian},
139
+
140
+ keywords = {Computation and Language (cs.CL), Programming Languages (cs.PL), Software Engineering (cs.SE), FOS: Computer and information sciences, FOS: Computer and information sciences},
141
+
142
+ title = {UniXcoder: Unified Cross-Modal Pre-training for Code
143
+ ```
144
+
145
+
146
+ # Glossary [optional]
147
+
148
+ More information needed
149
+
150
+ # More Information [optional]
151
+
152
+ More information needed
153
+
154
+ # Model Card Authors [optional]
155
+
156
+ Microsoft Team in collaboration with Ezi Ozoani and the Hugging Face Team.
157
+
158
+ # Model Card Contact
159
+
160
+ More information needed
161
+
162
+ # How to Get Started with the Model
163
+
164
+ Use the code below to get started with the model.
165
+
166
+ <details>
167
+ <summary> Click to expand </summary>
168
+
169
+ ```python
170
+ from transformers import AutoTokenizer, AutoModel
171
+
172
+ tokenizer = AutoTokenizer.from_pretrained("microsoft/unixcoder-base")
173
+
174
+ model = AutoModel.from_pretrained("microsoft/unixcoder-base")
175
+
176
+ ```
177
+ </details>