Manual Testing vs Automated Testing
The number one goal of any project is quality, and quality must be checked at the stage of development to avoid further complications. In fact, without testing, there is no certainty that anything will work correctly, and that something won’t go wrong unexpectedly. Smart contracts are no exception.
Smart contract testing is one of the key measures to improve smart contract security. In contrast to traditional software, smart contracts typically can not be updated after launch, which makes it a necessity to test them thoroughly before deployment.
Basically there are two types of testing smart contracts, which are manual, carried out by experts directly, and automated, carried out by means of machine work. Both are efficient, so the developer is free to choose each of them, however, there are some differences, which could affect that choice.
Automated testing
Automated testing is performed automatically by means of special tools or scripts. It comes about to make the testing process less complicated and time consuming for the experts. But all the tools, scripts and scenarios have to be prepared beforehand.
This type of testing always involves some planning and documentation and cannot be performed at random and on the spot, so it requires some additional time and budget.
On the other hand it is quicker at the running stage and can be performed repeatedly. It can be more reliable in some cases, unless the human factor comes into play. This type of check can be performed at several platforms in parallel, using various frameworks also reduces testing time.
Pros:
- The speed of testing is higher
- The testing process can be reused later in similar situations
- Faster reporting of results
Cons:
- Requires pre-planning, autotests can overlook unusual, uncommon issues beyond these plans or patterns
- The tools are not fully foolproof and have their limitations
Manual testing
Manual testing is a human-led process and involves a person performing manual testing steps. Code auditing, where developers and/or auditors review each line of the contract code, is an example of manual testing.
It is a thorough, deep-diving process, when the expert is searching for vulnerabilities and simulates various conditions which can cause a failure. Manual testing requires high-skilled operators to achieve impressive results since the proccess relies on excellent skills, knowledge and professional experience of an expert.
It is more time consuming, both to perform the testing and to submit the results.
Pros:
- Offers accurate feedback;
- Human skill and intuition help to detect uncommon and unexpected issues;
- Bugs and glitches can be tested ad hoc;
- Helps to discover logical loop-holes.
Cons:
- The expert’s professional experience and technical skills are determining factors of manual testing success;
- Prone to mistakes and errors due to the human factor;
- Some tasks can not be performed or are very time consuming because they require specific software;
- Takes more time and, at times, is more expensive.
Manual testing and automated testing use cases
Each type has specific situations when it’s likely to be applied.
Manual testing is usually preferable in the following cases:
- Exploratory testing. The title speaks for itself, and this type involves simultaneous testing of all sides of the contract. It requires minimum planning and maximum actual testing work;
- Business logic testing. It is needed to explore logic above the code, so it simply cannot be performed by a machine;
- Ad-Hoc testing. Or ‘intuitive testing’. It is performed without planning or documentation at all and based on improvisation.
Automated testing is more likely to be chosen in case of:
- ‘Default’ testing. Testing used to find ‘common’ and well-known vulnerabilities;
- Regression testing. Used in case of constant changes. It helps to ensure that recent changes to the code do not affect existing features;
- Performance testing. Checks the speed, scalability and stability and helps to spot possible bottlenecks.
Conclusion
It is very hard and not totally effective to rely only on one of the types ignoring the other, especially in smart contract testing and auditing. Each type covers its own field and both are excellent and indispensable. So a winning strategy is to combine them.
Nevertheless, manual testing can help to test logic and find some hidden bugs intuitively. Human skills and professional experience help to detect more complicated bugs and errors than autotests can normally find.
About SmartState
Launched in 2019 and located in Dubai, SmartState is retaining the place of one of the leading DeFi security auditing companies. We carry out security tests of the code core, smart contracts and blockchain for all types of errors or vulnerabilities.
We specialize in manual testing, so the SmartState’s tech team of white-hat security professionals carefully measures up a project’s git and supports clients with guidelines and recommendations for the further advancement.
Our security audit reports review the threats and vulnerabilities with which codebases may be exploited in the future, because the network achieves scalability and expands to accommodate more use cases and functionality.
Stay tuned and find more about us and what we provide on our:
