Crypto-related testing ​in fintech apps: the balance between ​losses and fast delivery

Hello everyone! My name is Roman Makitrenko. I am Head of QA at Wirex, a British fintech company that works on combining traditional and digital currencies in one app. Today we're going to talk about testing cryptocurrency operations and how to find a balance between losses due to mistakes​ and fast delivery of functionality.

Since we work with users' money, the cost of a mistake is especially critical and can depend on many factors. A bug in a fintech product can be costly for a company, not only financially, but also in terms of reputation. As a rule, consequences depend on the priority and severity of issues.​

Therefore, there are several criteria QA specialists need to consider when working with a fintech product:

• Security - Fintech applications use and store sensitive data, including personal, financial, and banking information. Hackers could try to access this information​.
• Data Protection - Data frequently changes hands in the world of fintech, so protecting and managing critical information is key for the testing phase.​ Problems with data protection could lead to legal action, compliance issues or losing users’ trust. Furthermore, such kinds of troubles might cause serious security issues.​
• Compliance - Financial organizations must adhere to a strict set of rules and regulations governed by the countries in which they operate. These laws continue to evolve, so testing should change with them​. Compliance issues might lead to restrictions in working with specific operations or even block the company's presence in certain countries or regions. ​
• Functional testing - this is a crucial stage of any testing scenario. This process ensures the product meets the requirements to operate effectively. Because fintech software involves exchanging sensitive data and information, testers should examine all possible risks, developments, stressors, and interactions with other applications and system components.​ It's important to remember if the parts of functionality don't work or have serious issues, a company could lose profit on operations and users could choose a competitor's product. It's also important to keep in mind the third-party issues. Users don't care where the problems occur; they will complain about your app, so you should  handle them like you would handle your own issues.​
• UX - The quality of a product is based on how convenient and easy it is to use. Fintech firms prioritize the consumer experience, and the QA must ensure that the product is easy for everyone to use. ​Users might mistrust the app and company and, as a result, switch to another app.​
• Performance - It's essential to ensure that your platform and services continue to work at peak times.​ As a rule, performance issues tend to accompany UX issues.

By testing any operation with crypto, it’s important to remember that the testing environment and the third-party sandboxes and blockchain testnets are not the same as Production. All these ecosystems are actively developing, and often the test environment has a more advanced version than the Production, and sometimes vice versa. Therefore, everything that we test in the sandbox needs to be double-checked with Production.

Also, before you start testing, make sure that you know where to get test cryptocurrencies since Bitcoin in testnet and mainnet are two different currencies with entirely different properties and operating speeds.

Basic functionality of a fintech app using crypto

Let's take a closer look at the basic functionality of a fintech app which uses cryptocurrencies.

Once users have passed registration and verification, they are ready to use the product on full power. ​They want to see how many currencies are in the app and the exchange rates.​

But firstly, they need to create a crypto account.​ For some blockchain platforms, this operation requires paying a commission on the blockchain. In such cases, the company incurs the costs. It divides this operation into creating an account in the System and activating the account in a blockchain address for a specific currency.

The user will need money in order to properly use the platform, so the System should provide the ability to transfer money into the System. This could involve a welcome bonus or other incentive, but the System should be able to perform such operations.

To make a profit on market speculations, users will need an option of a crypto exchange within the app.​ So this requires the function of cryptocurrency transfer outside of the System. ​

Fintech products have many other non-crypto and crypto-related functionality, but these are the basic features needed within a crypto app. Let's discuss them in more detail.​

Creating the crypto account

To register a crypto account, we use a master account from the System's side and a sub-account from the client's perspective. This approach allows us to easily maintain user accounts and save money because it usually makes accounts cheaper.

If a company pays to create addresses on the blockchain for accounts, then System addresses are used for such operations. Therefore, it is necessary to develop a particular trigger or specific logic for when and how this account should be refunded.

Let's check how the app works in practice.

The user selects "Create and activate an account", and the app sends a request to the server-side. The server processes it, sends it to the blockchain, and then waits while the blockchain confirms the account creation. Then it gets the hash of the transaction and pulls the account address through the hash. The next step is sending this data to the app and showing this information to the user.

In order to ensure that clients don’t have to wait, we could have a pool of preordered addresses on the server-side to provide for the users on request. Suppose a pool of pre-formed addresses is used. In that case, the System assigns an address for the crypto account from the generated pool of addresses for a specific currency.

Transferring money into the System ​

Users should have money in the System in order to use the app. One of the options is a crypto deposit or crypto transfer into the System (receive flow).​

Once the System identifies blockchain transactions related to the System's accounts, it waits for transaction confirmations. Some apps don’t wait for these confirmations, which can be risky. Then, the System updates the customer balance and finally sends notifications that the transfer has successfully completed.​ ​

A debit operation will also need to be done. This operation can use additional, specific logic. It’s crucial to ensure that the System's balance has been appropriately updated in all services, including those that generate additional information such as the system core, back office, transaction history and user notifications, etc. It’s also important to check that this balance was updated before and after the transaction, including checking to make sure that the changes went through on the blockchain.

Rounding the amount

One of the most common test cases in a fintech product, especially one that works with cryptocurrencies, is the difficulty with rounding. For example, a user sends 1.999998 to the System, but as a result of rounding/type conversion, could send 1.999999 out of the System.

Cryptocurrencies, unlike traditional currency, have high precision. An answer from the crypto exchange can look like 8 dots 8. For some coins, it might be significantly bigger. Sometimes, it's more than 15 figures before and after the dot.​ Remember that automatic calculations could be wrong. Therefore, it’s better to fully test libraries that are doing the calculations with such digits and high precision as we know that rounding is everywhere in the System, and it needs to be tested. Some operations will be mathematically rounded, and others rounded up or down.

Transaction confirmation ​

Sometimes, the System should wait for few confirmations, and now I will show why this is crucial. Blockchain logic depends on the currency, and it might have many specifics. That means some transactions, despite the fact they are correct, might not be confirmed at all.

For example, the well-known currency Ethereum has smart contracts that look just like any other regular contract between two or more parties; whether this contract will be fulfilled or not is impossible to say in advance. You also need to consider that if a transaction is set to a lower blockchain commission, it will take longer to confirm as there are transactions with a higher commission which will take priority. The higher the commission fee, the more earnings for miners, and the sooner they will confirm it. ​

Also, it's necessary to keep in mind that different types of cryptocurrencies could use the same blockchain. For instance, the Ethereum blockchain processes ETH (parent currency) transactions and operations with MakerDao (child currency) and many others cryptocurrencies. So the contract might contain few coins from the one blockchain, but they don't all have the same price.​

The crypto exchange operations

Most users speculate on the exchange rates of cryptocurrencies and their volatility. That means clients buy assets at a low price and sell them high. To provide user ability for crypto trading, there should be Exchange Flow in the System.​

The process in the System is following:​

• Firstly, the customer selects a currency to sell and buy​
• Then the client enters the amount they wish to exchange or receive ​
• The next step is transaction​ confirmation
• Finally, the System shows notification the operation was successfully completed​ ​

As the System, we need to make sure there’s enough money on the user's balance to proceed with the requested transaction.​ Also, we need to estimate transaction costs for the user: how much they will pay for it will be based on the amount we, as a company, pay for it, the desired profit (margin), fees, and other expenses. ​

Limits - As it might be expected, each operation should have specific limitations. There are variations between upper and lower limits on a particular scale. For example, the System doesn't work with transactions less than one cent in equivalent or bigger than 1000 bitcoins, as it associated these with risk. Also, five operations in a row may indicate suspicious activity. It could be any other technical, regulatory constraint. Such cases will require further examination.

The double conversion - It's necessary to understand that not every crypto exchange provides exchange rates for every cryptocurrency pair, and sometimes double conversion is needed to make the required operation. For example, to buy a pair of Dogecoin coins, you need to exchange EUR for BTC, and then BTC to Dogecoin. At the same time, cryptocurrency rates may differ on each of the exchanges, and some may not be on the market at all. But as soon as the System is connected to several cryptocurrency exchanges, then double-conversion becomes available.

As well as any system, the crypto exchanges require maintenance. When this happens, what should the System do with crypto rates and exchanges? Should it be closed? Or should it have some limitations or delays? The thing is, continuing trading based on the last known rate is quite risky. It’s a good task for Product teams to think about such cases. These corner cases have to be very accurately tested, together with other functional logic.​

It's also essential to control the balance on the System's accounts within the crypto exchanges because, in case of a shortage of funds, the System must report the problem. The same should happen if the user sold the cryptocurrency to the System but did not convert the money on the exchange. ​

It's vital to update the exchange rate after a short period of time because, due to the volatility of cryptocurrencies, their price can differ significantly during the day. The user could select currencies to trade, enter a required amount, get a confirmation message but not press the "Confirm" button. This scenario should also be taken into account when testing a crypto fintech product.

Also, it's important to answer questions like ‘what should the System do if one user is selling and another is buying crypto?’ Should it be traded on an exchange or done internally to save fees and other expenses? It is pretty risky, but it could save money.​

And of course, it's important to remember that there are always exceptions. There might be API rate limits, a quota period, a throttle value, and any other or even unexpected errors. ​

For example, if the System has 10 currencies and allows users to exchange crypto from one to another, then how many prices are in the System? The answer is 45, and there are only exchanges between currencies.

The number of combinations is denoted by Сnm and is calculated by the formula:

Where n = 10 is the number of currencies and m = 2 is the number of currencies in a pair.

For each pair, the System has to have two prices: a price for buying and a price for selling. Let's calculate. 45Х2=90. The minimum amount of prices produced will be 180, which will include prices from the crypto- exchange and cross-currency rates, and that’s just the beginning. You then need to include the prices that include fees and company profits.

If we add it all up, we get more than 500 prices.​ Let's add one more currency to our System and calculate how the number of prices will increase. It's a good time to estimate your manual regression. ​

Transfer outside of the System

This case has the following flow:​

• Firstly, the user selects the currency which they want to send. ​
• Then, they enter the amount of crypto they want to send.​
• The next step is transaction confirmation. ​
• Finally, we receive the confirmation, and the transaction is on a blockchain.​

Now, let's look at the operations that are performed on the backend.​

The System checks the balance on the user's account. Then, it calculates the commission. We need to remember that miners must confirm every transaction on the blockchain which means that such transfers require a commission for the miners.

After sending the transfer, it is necessary to perform a credit operation and update balances on all services in the System. When checking the balance, we need to make sure that there is enough money on the users' balance to cover the entered amount and the blockchain fee.

When withdrawing money outside the System, it’s necessary to check the existing limits.

Change in the fee value when withdrawing money from the System. Blockchain fees change depending on the network traffic – the higher the traffic, the more transactions are in the confirmation queue. Therefore, for the client's transaction to be confirmed faster, paying a higher commission to the miners is necessary. These changes should also be considered when testing your product.

What's the best way to test the app: manually or automatically?

I believe that regression cannot be tested manually if you have at least 10 currencies. It could be possible if you only have two or three, but with more than that it’s time to start automating your testing.​

Could it be fully automated? ​I don’t think this would be possible. Especially due to external factors such as third-party maintenance and the absence of functionality on the sandbox, manual checks are still necessary.​ If you write an entire CI with so many additional systems involved, you're bound to have false test failures. There is no way to handle all this either. Of course, you can close all external systems with mocks, but there’s no guarantee that they will not change the contract, which you could then miss.

Separately, no one can manage automatic testing alone, it’s only by working as a team you can achieve the result at the proper level.​ QA will provide corner cases and specific scenarios, and developers will share knowledge on making your automation stable, scalable, maintainable, and high-performance.​ Product Owners will show the company's next steps, how it should be scalable and which performance you should provide.​ ​

Wirex has automation on each side from the backend to all frontend clients like iOS, Android, Web app and web portal.

Developers, in their turn, are implementing automation at all levels: from unit testing to integration testing. The main focus for developers is on the technical behaviour of the system, while QA is more focused on the behaviour of the business. Does this mean that QA does not automate services? No, we do that too. Like developers, testers also automate services from a business point of view since each service has its own business purpose. We work as a team where developers use QA code and vice versa.

Who develops auto tests in the QA team? Everybody. There is no manual or automation QA. We are all full-stack QAs. When I say everybody, I mean each team member, including me. We are supporting and helping each other to study and evolve in manual and automation testing. ​