|
description: "ToDo: add description" |
|
|
|
input_data_transformations: [] |
|
input_keys: |
|
- "code" |
|
- "public_tests_individual_io" |
|
|
|
output_keys: |
|
- "all_tests_passed" |
|
- "testing_results_summary" |
|
output_data_transformations: |
|
- _target_: martinjosifoski.CC_flows.src.data_transformations.CorrectnessFlag |
|
input_key: "raw_response.public_tests_results" |
|
output_key: "all_tests_passed" |
|
- _target_: martinjosifoski.CC_flows.src.data_transformations.TestingResultsSummaryGeneration |
|
output_key: "testing_results_summary" |
|
|
|
single_test_error_message: True |
|
|
|
no_error_template: |2- |
|
${.issue_title} |
|
All of the executed tests passed. |
|
|
|
compilation_error_template: |2- |
|
${.issue_title} |
|
The execution resulted in a compilation error. |
|
## Compilation error message: |
|
{{error_message}} |
|
timeout_error_template: |2- |
|
${.issue_title} |
|
The execution timed out, the solution is not efficient enough. |
|
runtime_error_template: |2- |
|
${.issue_title} |
|
The execution resulted in a runtime error on the following test. |
|
## [Failed test] Input |
|
``` |
|
{{test_input}} |
|
``` |
|
## [Failed test] Runtime error message |
|
{{error_message}} |
|
single_test_error_template: |2- |
|
${.issue_title} |
|
The Python code does not solve the problem in the problem description due to logical errors. It fails the following test: |
|
## [Failed test] Input |
|
``` |
|
{{test_input}} |
|
``` |
|
## [Failed test] Expected output |
|
``` |
|
{{expected_output}} |
|
``` |
|
## [Failed test] Generated output |
|
``` |
|
{{generated_output}} |
|
``` |
|
all_tests_header: |2- |
|
${.issue_title} |
|
The Python code does not solve the problem in the problem description due to logical errors. It fails on the following tests. |
|
test_error_template: |2- |
|
## [Failed test {{idx}}] |
|
### [Failed test {{idx}}] Input |
|
``` |
|
{{test_input}} |
|
``` |
|
### [Failed test {{idx}}] Expected output |
|
``` |
|
{{expected_output}} |
|
``` |
|
### [Failed test {{idx}}] Generated output |
|
``` |
|
{{generated_output}} |
|
``` |
|
tests_separator: "\n\n" |
|
|
|
issue_title: "# Issue with the last proposed solution" |
|
|
|
feedback_title: "# Feedback on the last proposed solution" |
|
|
|
no_code_template: |2- |
|
${.feedback_title} |
|
The code was not provided in the correct output format specified in the request or it was not provided at all. |
|
feedback_only_template: |2- |
|
${.feedback_title} |
|
{{feedback_content}} |
|
feedback_and_issue_template: |2- |
|
{{issue_description}} |
|
|
|
{{feedback_content}} |
|
|
