Payment Gateway Timings
HomeAway has relationships with multiple Payment Gateways. By collecting timing data, we are able to compare gateways with each other and perform other explorations. We have a centralized service (called “HAPI”) to manage gateway connections. This service presents a common interface to clients, converts the common data into gateway-specific requests, makes the request and responds back to the client. It also indexes three main timestamps associated with the payment gateways:
- When did HAPI get the initial request
- When did HAPI send the request to the gateway
- When did HAPI get the response from the gateway
By subtracting #3 from #2, we can see how long a gateway takes to process a request, and we can break them out by gateway:
From top to bottom, the gateways are “Braspag,” “Cybersource,” “Intuit,” “PayflowPro” and “Yapstone.” These gateways cover different business lines at HomeAway and provide a rare opportunity to compare gateway response times. Many reasons exist to choose between gateways; response time is just one facet. In tabular form, the summary statistics (in seconds) look like:
The geometric mean and other geometric statistics are used because this is a timing analysis, and response times are well-modeled as right-sided, log-normal curves. The minimum and maximum values show a pretty wide disparity in observations. I wanted to trim the data to keep only the points within two (geometric) standard deviations from the (geometric) mean:
Essentially, the middle data has the same “shape” as the entire dataset, but without the egregious tails.
The data and charts above were calculated via the following R statements. The “geomean” and “geosd” functions calculate the geometric mean and standard deviation; R really should have geometric statistics built-in.
Overlaying all the gateway timings on one plot is hard to interpret, but breaking them out via facet_grid() geometry is easier to comprehend.
This plot illustrates that for all types of requests made to gateways, PayflowPro is quickest to respond and (since it has a relatively narrow peak) it is the most predictable. The other gateways have other curves with other response times; the bimodal nature of the Yapstone response caught my eye.
Each gateway provides several different functions:
Authorize - Ensures that the credit card is valid and has an open-to-buy limit of a given amount
Capture - Moves the money that was initially authorized
Sale - A one-call “Authorize-and-Capture”
Credit - Moves money from the merchant to the customer, without a referenced Sale or Capture
Refund - Moves money from the merchant to the customer with a referenced Sale or Capture
Verification - A quick “card check”
Void - Cancels the Sale or Capture
Void-or-Refund - Either cancels or refunds the Sale if the “void” time limit has expired
We can visualize the speed of each operation in many ways. Here we’re going to use an empirical cumulative density function, or “ECDF”, and plot it for each operation and gateway.
The x-axis is the amount of time (in seconds) before the operation completes. This ECDF visualizes the time needed to complete ALL of the observations of a given operation - the steeper the slope, the smaller the time-spread between the fastest and slowest observation and the more predictable the response. The Yapstone (purple line) responses for Capture, Credit and Refund are very steep and tend towards the left, which means that Yapstone is able to process these operations quickly and predictably. Likewise, PayflowPro is very fast on Credits, Refunds and Sales.
Constructing different ECDFs allows us to see how each gateway processes all of the operations. Intuit and Braspag are very predictable for all of their operations, whereas the other gateways have varying operational speeds. Yapstone again reveals the most bimodal behavior and stimulates a closer look.
Here a violin plot compares the various operations. “Authorize” and “Sale” take around 2 seconds and the other operations are in the subsecond range (except for “Void” - its shape is indicative of a small number of operations - the clients who used “Void” switched to “Void_or_Refund” once it was available). These operations - “Capture,” “Credit,” “Refund” and “Void_or_Refund” - all require a referenced transaction. One conclusion suggested by this chart is that Yapstone is able to process follow-on operations quickly and predictably.
Visualizing the Yapstone operations as a dot-plot over time shows the above. The bimodal nature again stands out strongly with most “Sale” operations taking more than 1.5 seconds and most “Credit” operations taking less than 1 second. However, this chart also reveals a line of blue near the bottom. We can plot successful sales and unsuccessful ones side-by-side with a violin plot and see the following.
Yapstone is able to quickly fail certain “Sale” attempts, but others take about the same amount of time as the successful ones. The theory is that Yapstone fails some sales quickly, but needs to reach out to the payment processors for the bulk. It is simple enough to create this visualization for all the gateways:
All the gateways are able to fail-fast in certain instances.
These visualizations and data transformations are very easy to perform using the R environment. The hardest part was getting the data into a format that R can work with.