| name: "CF_CodeTesting" | |
| description: "ToDo: add description" | |
| input_data_transformations: [] | |
| input_keys: | |
| - "code" | |
| - "public_tests_individual_io" | |
| output_keys: | |
| - "all_tests_passed" | |
| - "testing_results_summary" | |
| output_data_transformations: | |
| - _target_: martinjosifoski.CC_flows.src.data_transformations.CorrectnessFlag | |
| input_key: "public_tests_results" | |
| output_key: "all_tests_passed" | |
| - _target_: martinjosifoski.CC_flows.src.data_transformations.TestingResultsSummaryGeneration | |
| output_key: "testing_results_summary" | |
| single_test_error_message: True | |
| no_error_template: |2- | |
| ${.issue_title} | |
| All of the executed tests passed. | |
| compilation_error_template: |2- | |
| ${.issue_title} | |
| The execution resulted in a compilation error. | |
| ## Compilation error message: | |
| {{error_message}} | |
| timeout_error_template: |2- | |
| ${.issue_title} | |
| The execution timed out, the solution is not efficient enough. | |
| runtime_error_template: |2- | |
| ${.issue_title} | |
| The execution resulted in a runtime error on the following test. | |
| ## [Failed test] Input | |
| ``` | |
| {{test_input}} | |
| ``` | |
| ## [Failed test] Runtime error message | |
| {{error_message}} | |
| single_test_error_template: |2- | |
| ${.issue_title} | |
| The Python code does not solve the problem in the problem description due to logical errors. It fails the following test: | |
| ## [Failed test] Input | |
| ``` | |
| {{test_input}} | |
| ``` | |
| ## [Failed test] Expected output | |
| ``` | |
| {{expected_output}} | |
| ``` | |
| ## [Failed test] Generated output | |
| ``` | |
| {{generated_output}} | |
| ``` | |
| all_tests_header: |2- | |
| ${.issue_title} | |
| The Python code does not solve the problem in the problem description due to logical errors. It fails on the following tests. | |
| test_error_template: |2- | |
| ## [Failed test {{idx}}] | |
| ### [Failed test {{idx}}] Input | |
| ``` | |
| {{test_input}} | |
| ``` | |
| ### [Failed test {{idx}}] Expected output | |
| ``` | |
| {{expected_output}} | |
| ``` | |
| ### [Failed test {{idx}}] Generated output | |
| ``` | |
| {{generated_output}} | |
| ``` | |
| tests_separator: "\n\n" | |
| issue_title: "# Issue with the last proposed solution" | |