It is almost a certainty that you will encounter errors or failed transactions in at least 1 or more of your load tests potentially due to a myriad of reasons - and this is perfectly normal. How do we go about understanding these errors using Flood?
Using Flood, there are 4 main areas that will help in getting further information about errors and/or failed transactions:
- Failed Transactions count
- The Flood execution logs
- Transaction-based drill down error information
- Raw CSV results
Failed Transactions count
The simplest way to understand if any errors are being observed is by keeping an eye on the overall passed and failed transaction count during a test.
This aggregate summary will update every 15 seconds during test execution and will give you the total Failed Transaction count across all transaction labels. This is a great way to quickly keep an eye on failed transactions as they happen.
The Flood execution logs
Execution logs are provided for every Flood and show detail about Grid & node initilization, scenario metrics, and most importantly and errors or exceptions generated by the tool being used for the test.
It's always important to monitor these logs as you can still see a relatively clean test with regards to the number of passed transactions - however the logs may tell a completely different story.
The errors are grouped for readability so that there aren't thousands of messages that you need to sort through to find an issue.
Date and timestamps are also provided so that you are able to investigate what was happening if there are any response time spikes or other events that were observed during the test. Often a response time spike or a throughput dip can be correlated with log events happening at the same time.
Transaction-based drill down error information
Every single transaction label has a drill-down results sub-section that enables you to look at the type of errors (if any) being observed.
You are able to access this section by clicking on the respective arrow icon for the transaction you are interested in analysing further:
You can see that both of these transaction labels have a high error rate - selecting one of them will enable you to understand what type of error it is and more importantly - what the actual error is.
Once clicking on the arrow icon - you will be presented with a page similar to the one below:
There are 3 main areas that show further information about any errors observed:
- Errors & Status Codes - this widget shows you the percentage of error responses to successful 2xx or 3xx responses. Any 4xx or 5xx responses received would be reported here. In this example you can see that the error responses are non-HTTP based errors which means we should be able to see more information in the logs.
- Transaction Rate - Within the transaction rate graph there are red triangle icons that can be selected to reveal more information about the error at that particular point in time.
- Headers / Request / Response - This is the full description of any error responses observed. The Request and Response tabs display the specific request and response payloads that caused the error.
Raw CSV results
Every Flood load injection node has an archived set of results that contains essentially all test artefacts used for a Flood. Amongst these test artefacts is the full raw data of every request placed in the test.
The file is called results.csv and is located under the results directory.
Opening the file shows every single request and response as well as other helpful metrics. Using filters in Microsoft Excel can help you narrow down and group requests that returned a 404 error for example:
If the error was a non-HTTP error - the error message is recorded within the failureMessage column for your reference.