SalesforceFoundation/CumulusCI — pull request #82 by jlantz (2014-11-04 04:58:06+00:00)
Title: Add support for generating and parsing debug logs for each test class

This feature is backwards compatible. To enable it, set the following environment variables:

* DEBUG_TESTS=true
* DEBUG_LOGDIR=PATH/TO/WRITE/LOGS

When these variables are set, a TraceFlag is created for each class. When the tests complete, the ApexLogs are retrieved, then the log body is retrieved into the DEBUG_LOGDIR directory as `.log`

NOTE: Enabling this functionality will use a lot of API calls. Each TraceFlag create is an individual call per test class. Each debug log retrieve is individual. And each expiration of the TraceFlag is individual.

# Sample Output

```
(venv2)~/dev/GrantLifecycleApp (master) $ ant runAllTests
Buildfile: /Users/jlantz/dev/GrantLifecycleApp/build.xml

runAllTests:
[echo] ----------------------------------------------------------------------------
[exec] Running Query: SELECT Id, Name FROM ApexClass WHERE NamespacePrefix = null and (Name LIKE '%_TEST' OR Name LIKE '%_TESTS')
[exec] Found 11 classes
[exec] Queuing tests for execution...
[exec] Completed: 0 Processing: 0 Queued: 11
[exec]
[exec] -------------------------------------------------------------------------------
[exec] Test Results
[exec] -------------------------------------------------------------------------------
[exec] Class: BudgetActual_TEST (8.486s)
[exec] Pass: BudgetActualBoth
[exec] DEBUG LOG INFO:
[exec] 100 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 118 / 10000 Number of DML rows
[exec] 16 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 10 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 66 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 4.19 duration
[exec] Pass: BudgetActualReportingYearTest
[exec] DEBUG LOG INFO:
[exec] 83 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 101 / 10000 Number of DML rows
[exec] 16 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 11 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 97 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 1.019 duration
[exec] Pass: BudgetActualReportingYearTestCompany2
[exec] DEBUG LOG INFO:
[exec] 79 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 101 / 10000 Number of DML rows
[exec] 16 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 11 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 97 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 1.011 duration
[exec] Pass: NoAccessBudgetActualOnlyC3
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 20 / 10000 Number of DML rows
[exec] 9 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 2 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 8 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.264 duration
[exec] Class: ConnectApiHelper_TESTS (2.517s)
[exec] Pass: testAdjacentMentions
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.01 duration
[exec] Pass: testEmptyString
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.001 duration
[exec] Pass: testInvalidMentionType
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.008 duration
[exec] Pass: testLeadingMention
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.002 duration
[exec] Pass: testLinkAndHashtagParsing
[exec] DEBUG LOG INFO:
[exec] 18 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 1 / 10000 Number of DML rows
[exec] 1 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.658 duration
[exec] Pass: testMentionInComment
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 2 / 10000 Number of DML rows
[exec] 2 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.199 duration
[exec] Pass: testMentionOnly
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.002 duration
[exec] Pass: testNoMentions
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.002 duration
[exec] Pass: testNullString
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.002 duration
[exec] Pass: testTrailingMention
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 0 / 10000 Number of DML rows
[exec] 0 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.002 duration
[exec] Class: ContactSearchUtility_TEST (1.882s)
[exec] Pass: ContactSearch
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 19 / 10000 Number of DML rows
[exec] 8 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 4 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 11 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.454 duration
[exec] Class: ConvertOptytoGrant_TESTS (3.097s)
[exec] Pass: OpportunitytoGrantConversionPage
[exec] DEBUG LOG INFO:
[exec] 91 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 28 / 10000 Number of DML rows
[exec] 12 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 6 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 13 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 1.599 duration
[exec] Class: DeliverablesWizardController_TESTS (4.228s)
[exec] Pass: NewDeliverableCreation
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 25 / 10000 Number of DML rows
[exec] 14 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 4 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 4 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 2.371 duration
[exec] Class: DepartmentUtility_TEST (4.499s)
[exec] Pass: DepartmentUtilityCreation
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 19 / 10000 Number of DML rows
[exec] 8 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 3 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 23 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 2.939 duration
[exec] Class: GrantManagementUtilities_TESTS (2.389s)
[exec] Pass: ContactUserLookupTests
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 13 / 10000 Number of DML rows
[exec] 8 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 3 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 3 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.765 duration
[exec] Pass: CurrentUserLookupTests
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 11 / 10000 Number of DML rows
[exec] 6 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 2 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 2 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.204 duration
[exec] Class: IssuesIntersectionController_TEST (3.669s)
[exec] Pass: MainComponentTest
[exec] DEBUG LOG INFO:
[exec] 8 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 36 / 10000 Number of DML rows
[exec] 17 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 9 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 30 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.937 duration
[exec] Pass: MainComponentTestNoDefaults
[exec] DEBUG LOG INFO:
[exec] 2 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 29 / 10000 Number of DML rows
[exec] 14 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 5 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 12 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.484 duration
[exec] Pass: MainComponentTestStaffDefaults
[exec] DEBUG LOG INFO:
[exec] 3 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 34 / 10000 Number of DML rows
[exec] 16 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 6 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 17 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.582 duration
[exec] Class: ManageStaffParticipation_TESTS (3.215s)
[exec] Pass: UpdateStaffonOpportunity
[exec] DEBUG LOG INFO:
[exec] 4 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 29 / 10000 Number of DML rows
[exec] 17 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 7 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 9 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 1.704 duration
[exec] Class: ParticipationControllerExtension_TEST (3.44s)
[exec] Pass: ControllerTestCancel
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 12 / 10000 Number of DML rows
[exec] 7 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 1 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.279 duration
[exec] Pass: ControllerTestErrors
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 10 / 10000 Number of DML rows
[exec] 6 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 1 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.185 duration
[exec] Pass: ControllerTestviaAction
[exec] DEBUG LOG INFO:
[exec] 1 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 21 / 10000 Number of DML rows
[exec] 15 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 15 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 22 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.596 duration
[exec] Pass: ControllerTestviaPage
[exec] DEBUG LOG INFO:
[exec] 12 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 21 / 10000 Number of DML rows
[exec] 15 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 14 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 22 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.714 duration
[exec] Class: TestDataFactory_TEST (1.494s)
[exec] Pass: SetupCustomSettingsBudgeUpdate
[exec] DEBUG LOG INFO:
[exec] 0 / 10000 Maximum CPU time
[exec] 0 / 6000000 Maximum heap size
[exec] 31 / 10000 Number of DML rows
[exec] 6 / 150 Number of DML statements
[exec] 0 / 10 Number of Email Invocations
[exec] 0 / 10 Number of Mobile Apex push calls
[exec] 0 / 100 Number of SOQL queries
[exec] 0 / 20 Number of SOSL queries
[exec] 0 / 100 Number of callouts
[exec] 0 / 50 Number of future calls
[exec] 0 / 50000 Number of query rows
[exec] 0 / 50 Number of queueable jobs added to the queue
[exec] 0.195 duration
[exec] -------------------------------------------------------------------------------
[exec] Passed: 29 Failed: 0 Compile Fail: 0 Skipped: 0
[exec] -------------------------------------------------------------------------------

BUILD SUCCESSFUL
Total time: 25 seconds
```