Pull to refresh
147.2

IT systems testing *

Testing everything

Show first
Rating limit
Level of difficulty

Lessons learned from testing Over 200,000 lines of Infrastructure Code

Reading time 11 min
Views 3.1K


IaC (Infrastructure as Code) is a modern approach and I believe that infrastructure is code. It means that we should use the same philosophy for infrastructure as for software development. If we are talking that infrastructure is code, then we should reuse practices from development for infrastructure, i.e. unit testing, pair programming, code review. Please, keep in mind this idea while reading the article.


Russian Version

Read more →
Total votes 5: ↑5 and ↓0 +5
Comments 4

Handling Objections: Static Analysis Will Take up Part of Working Time

Reading time 5 min
Views 992
bugTalking to people at conferences and in comments to articles, we face the following objection: static analysis reduces the time to detect errors, but takes up programmers' time, which negates the benefits of using it and even slows down the development process. Let's get this objection straightened out and try to show that it's groundless.
Read more →
Total votes 33: ↑32 and ↓1 +31
Comments 0

Configuration of the Warnings Next Generation plugin for integration with PVS-Studio

Reading time 3 min
Views 1.6K

Picture 4


The PVS-Studio 7.04 release coincided with the release of the Warnings Next Generation 6.0.0 plugin for Jenkins. Right in this release Warnings NG Plugin added support of the PVS-Studio static code analyzer. This plugin visualizes data related to compiler warnings or other analysis tools in Jenkins. This article will cover in detail how to install and configure this plugin to use it with PVS-Studio, and will describe most of its features.
Read more →
Total votes 34: ↑33 and ↓1 +32
Comments 0

Testing SQL Server code with tSQLt

Reading time 20 min
Views 2.3K
FYI: this article is an expanded version of my talk at SQA Days #25.

Based on my experience with colleagues, I can state: DB code testing is not a widely spread practice. This can be potentially dangerous. DB logic is written by human beings just like all other «usual» code. So, there can be failures which can cause negative consequences for a product, business or users. Whether these are stored procedures helping backend or it is ETL modifying data in a warehouse — there is always a risk and testing helps to decrease it. I want to tell you what tSQLt is and how it helps us to test DB code.

Read more →
Total votes 8: ↑7 and ↓1 +6
Comments 1

Escaping the Thicket of Tests: Building a Shortcut from a Fixture to an Assertion

Reading time 15 min
Views 1.1K


In this article, I would like to propose an alternative to the traditional test design style using functional programming concepts in Scala. This approach was inspired by many months of pain from maintaining dozens of failing tests and a burning desire to make them more straightforward and more comprehensible.


Even though the code is in Scala, the proposed ideas are appropriate for developers and QA engineers who use languages supporting functional programming. You can find a Github link with the full solution and an example at the end of the article.

Read more →
Total votes 8: ↑6 and ↓2 +4
Comments 0

How to Make Emails and Not Mess Up: Practical Tips

Reading time 23 min
Views 3K


A developer, who first encountered generating emails, has almost no chance to write an application, that will do it correctly. Around 40% of emails, generated by corporate applications, are violating some form of standard, and due to this, there are problems with delivery and display. There are reasons for this: emails are technically more difficult than the web, and operating emails is regulated by a few hundred standards, as well as an uncountable number of generally accepted (and not as much) practices, whereas the email clients are more varied and unpredictable than browsers. Testing may significantly improve the situation, but materials that are dedicated to testing the email system, are practically non-existent.

Mail.ru regularly interacts with its users by email. In our projects, all the components responsible for generating emails and even individual mailings, are subject to mandatory testing. In this article, we will share our experience (learning from our mistakes).
Read more →
Total votes 40: ↑40 and ↓0 +40
Comments 2

Open source tool for intent-based search quality validation

Reading time 2 min
Views 947
No matter how cool your search solution is, without a reliable way of testing it you will not be able to do any improvement without breaking something valuable.

Even a tiny increase in overall search result quality might have a noticeable effect on conversion rate. Naturally, you can use exit-rate to judge if your search needs improvements and do required adjustments, but will you be able to make sure those changes are safe? What if improved relevancy for a specific category breaks search for several other categories?

image

Moreover, high exit-rate means no matter what you are going to do, the problem has taken place, so dozens of customers already didn’t manage to find what they were up to. Basically, the only way to proceed is to have a reliable search quality validation approach.
Read more →
Total votes 7: ↑6 and ↓1 +5
Comments 0

Quality as Team's responsibility. Our QA experience

Reading time 7 min
Views 2K

Disclaimer: This is a translation of an article. All rights belongs to author of original article and Miro company.


I'm a QA Engineer in Miro. Let me tell about our experiment of transferring partially testing tasks to developers and of transforming Test Engineer role into QA (Quality assurance).


First briefly about our development process. We have daily releases for client side and 3 to 5 weekly releases of server side. Team have 60+ people spitted onto 10 Functional Scrum Teams.


I'm working in Integration team. Our tasks are:


  • Integration of our service into external products
  • Integration of external products into our service
    For example we have integrated Jira. Jira Cards — visual representation of tasks so it's useful to work with tasks not opening Jira at all.

    image

How the experiment starts


All starts with trivial issue. When someone of Test Engineers had sick leave then team performance was degraded significantly. Team was continued working on tasks. However when code was reached testing phase task was hold on. As a result new functionality didn't reach production in time.


Going onto vacation by Test Engineer is a more complex story. He/she needs to find another Test Engineer who ready to take extra tasks and conduct knowledge sharing. Going onto vacation by two Test Engineers at the sane time is not an applicable luxury.

Read more →
Total votes 9: ↑8 and ↓1 +7
Comments 0

Introduce Static Analysis in the Process, Don't Just Search for Bugs with It

Reading time 15 min
Views 5.1K
This article is an authorized translation of the original post. The translation was made with the kind help of the guys from PVS-Studio. Thank you, guys!

What encouraged me to write this article is considerable quantity of materials on static analysis, which recently has been increasingly coming up. Firstly, this is a blog of PVS-Studio, which actively promotes itself on Habr posting reviews of errors, found by their tool in open source projects. PVS-Studio has recently implemented Java support, and, of course, developers from IntelliJ IDEA, whose built-in analyzer is probably the most advanced for Java today, could not stay away.

When reading these reviews, I get a feeling that we are talking about a magic elixir: click the button, and here it is — the list of defects right in front of your eyes. It seems that as analyzers get more advanced, more and more bugs will be found, and products, scanned by these robots, will become better and better without any effort on our part.

Well, but there are no magic elixirs. I would like to talk about what is usually not spoken in posts like «here are things that our robot can find»: what analyzers are not able to do, what's their real part and place in the process of software delivery, and how to implement the analysis properly.


Ratchet (source: Wikipedia).
Read more →
Total votes 32: ↑31 and ↓1 +30
Comments 0

Zen of Unit Testing

Reading time 4 min
Views 2.8K


Ability to write good unit tests is an important feature of any developer. But how to understand that your unit tests are correct? Good unit test is like a good chess game. In our case chessmen are the approaches which we are going to discuss in this post. There is no best chessman in a chess game because everything depends on the positions (and a player). Likewise, in unit testing you don't have to distinguish only one approach. In other words, you should use all approaches together to get the best result. So, if you want to win this game, then welcome under the cut.

Read more →
Total votes 23: ↑21 and ↓2 +19
Comments 0

Test me if you can. Do YML developers Dream of testing ansible?

Reading time 3 min
Views 3.5K

kitchen-ci schema


It is text version of the presentation 2018-04-25 at Saint-Petersburg Linux User Group. Configuration example locates at https://github.com/ultral/ansible-role-testing


I suppose that that you make configuration management, not bash. It means that you have to test it some how. Have you ever tested ansible roles? How do you do it?

Read more →
Total votes 18: ↑17 and ↓1 +16
Comments 2

How to test your own OS distribution

Reading time 3 min
Views 1.8K

intro


Russian version


Let's imagine that you are developing software and hardware appliance. The appliance consists of custom OS distributive, upscale servers, a lot of business logic, as a result, it has to use real hardware. If you release broken appliance, your users will not be happy. How to do stable releases?


I'd like to share my story how we dealt with it.

Читать дальше →
Total votes 17: ↑15 and ↓2 +13
Comments 0
2

Authors' contribution