Timothy-Vinzent commited on
Commit
1feb2ff
·
verified ·
1 Parent(s): 357fcb5

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +93 -0
app.py CHANGED
@@ -127,6 +127,99 @@ def build_interface():
127
  """
128
  with gr.Blocks() as demo:
129
  gr.Markdown("# GPT-4o Mini System Prompt Submission")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
130
  gr.Markdown("""Classification Task: Document and Clause Level Identification
131
  Challenge Description
132
  Participants must create a system prompt for a language model that classifies user queries about legal documents into two specific categories:"
 
127
  """
128
  with gr.Blocks() as demo:
129
  gr.Markdown("# GPT-4o Mini System Prompt Submission")
130
+ # General description
131
+ gr.Markdown("""Classification Task: Document and Clause Level Identification
132
+ Participants must create a system prompt for a language model that classifies user queries about legal documents into two specific categories:
133
+ 1. **Document Level**: Determines whether the query refers to a single document or multiple documents.
134
+ 2. **Clause Level**: Identifies whether the query is focused on:
135
+ - A single clause,
136
+ - Multiple clauses, or
137
+ - General information not constrained to any specific clause.
138
+
139
+ The model must return a valid JSON object with the following structure:
140
+
141
+ ```
142
+ {
143
+ "document_level": "single/multiple",
144
+ "clause_level": "single/multiple/general"
145
+ }
146
+ ```
147
+
148
+ The goal is to ensure that the model's output is concise, structured, and accurate. This task is designed to evaluate the robustness of the system prompt in handling classification tasks with short, precise outputs.
149
+ """)
150
+
151
+ # Example Inputs and Outputs in an Accordion
152
+ with gr.Accordion("Example Inputs and Expected Outputs", open=False):
153
+ gr.Markdown("""
154
+ 1. **User Message Example 1:**
155
+ - *"Please provide the contract for the lease agreement."*
156
+ - **Expected Output:**
157
+ ```
158
+ {"document_level": "single", "clause_level": "general"}
159
+ ```
160
+
161
+ 2. **User Message Example 2:**
162
+ - *"I need all clauses related to termination in the employment contract."*
163
+ - **Expected Output:**
164
+ ```
165
+ {"document_level": "single", "clause_level": "multiple"}
166
+ ```
167
+
168
+ 3. **User Message Example 3:**
169
+ - *"Can you send me the financial reports and the partnership agreement?"*
170
+ - **Expected Output:**
171
+ ```
172
+ {"document_level": "multiple", "clause_level": "general"}
173
+ ```
174
+
175
+ 4. **User Message Example 4:**
176
+ - *"What are the key clauses in the NDA?"*
177
+ - **Expected Output:**
178
+ ```
179
+ {"document_level": "single", "clause_level": "multiple"}
180
+ ```
181
+
182
+ 5. **User Message Example 5:**
183
+ - *"Tell me about the company’s financials."*
184
+ - **Expected Output:**
185
+ ```
186
+ {"document_level": "single", "clause_level": "general"}
187
+ ```
188
+
189
+ 6. **User Message Example 6:**
190
+ - *"Provide all contracts and their confidentiality clauses."*
191
+ - **Expected Output:**
192
+ ```
193
+ {"document_level": "multiple", "clause_level": "multiple"}
194
+ ```
195
+
196
+ 7. **User Message Example 7:**
197
+ - *"Extract the arbitration clause from this service agreement."*
198
+ - **Expected Output:**
199
+ ```
200
+ {"document_level": "single", "clause_level": "single"}
201
+ ```
202
+ """)
203
+
204
+ # Challenge instructions in another Accordion
205
+ with gr.Accordion("Challenge Instructions", open=False):
206
+ gr.Markdown("""
207
+ - Design a system prompt that ensures the AI generates outputs like those above when given similar user messages.
208
+
209
+ The system prompt should:
210
+ 1. Specify formatting requirements (e.g., *"Output must be a valid JSON object"*). Note that we are not using constrained decoding or any sort of JSON mode; if not correctly prompted, the LLM will output plain text.
211
+ 2. Emphasize strict adherence to classification definitions:
212
+ - *Single Document:* Refers to one document.
213
+ - *Multiple Documents:* Refers to more than one document.
214
+ - *Single Clause:* Refers to one specific clause.
215
+ - *Multiple Clauses:* Refers to more than one specific clause.
216
+ - *General Information:* Refers to general content not tied to specific clauses.
217
+
218
+ You can only submit once, so test your system prompt thoroughly before submission!
219
+ """)
220
+
221
+
222
+
223
  gr.Markdown("""Classification Task: Document and Clause Level Identification
224
  Challenge Description
225
  Participants must create a system prompt for a language model that classifies user queries about legal documents into two specific categories:"