DDoS attacks send ripples on the ocean of the Internet, produced by creations of various sizes - botnets. Some of them feed at the top of the ocean, but there also exists a category of huge, deep water monstrosities that are rare and dangerous enough they could be seen only once in a very long time.

November 2021 we encountered, and mitigated, several attacks from a botnet, that seems to be unrelated to one described and/or well-known, like variants of Mirai, Bashlite, Hajime or Brickerbot.

Although our findings are reminiscent of Mirai, we suppose this botnet is not based purely on propagating Linux malware, but a combination of brute forcing and exploiting already patched CVEs in unpatched devices to grow the size of it. Either way, to confirm how exactly this botnet operates, we need to have a sample device to analyze, which isn’t our area of expertise.

This time, we won’t give it a name. It is not 100% clear what we are looking at, what are the exact characteristics of it, and how big this thing actually is. But there are some numbers, and where possible, we have made additional reconnaissance in order to better understand what we’re dealing with.

But let us first show you the data we’ve gathered, and leave conclusions closer to the end of this post.

IaC Development Life Cycle

This is the translation of my speech at T-Meetup: DevOps Life Cycle.

I believe that you have heard about SDLC (Systems development life cycle). Is it possible that the same things are applicable for the IaC?

The third quarter of 2021 brought a massive upheaval in the scale and intensity of DDoS attacks worldwide.

It all led to September when together with Yandex, we uncovered one of the most devastating botnets since the Mirai and named it Meris, as it was held accountable for a series of attacks with a very high RPS rate. And as those attacks were aimed all over the world, our quarterly statistics also changed.

This quarter, we've also prepared for your consideration a slice of statistics on the application layer (L7) DDoS attacks. Without further ado, let us elaborate on the details of DDoS attacks statistics and BGP incidents for Q3, 2021.

Introduction

For the last five years, there have virtually been almost no global-scale application-layer attacks.

During this period, the industry has learned how to cope with the high bandwidth network layer attacks, including amplification-based ones. It does not mean that botnets are now harmless.

End of June 2021, Qrator Labs started to see signs of a new assaulting force on the Internet – a botnet of a new kind. That is a joint research we conducted together with Yandex to elaborate on the specifics of the DDoS attacks enabler emerging in almost real-time.

This article continues the series of articles on load tests. Today we will analyze the testing methodology and answer the question: "How many IP cameras can be connected to a WebRTC server?"

Do you remember how just a few years ago it was a disaster to lose a camera at the end of a vacation? All memorable pictures and videos then disappeared along with the lost device. Probably, this fact prompted the great minds to invent cloud storage, so that the safety of records no longer depends on the presence of the devices on which these records are made.

We continue to review variants of load tests. In this article we will go over the testing methodology and conduct a load test that we will use to try and determine the number of users that could watch and stream at the same time, meaning the users will simultaneously publish and view the streams.

This article is a continuation of our series of write-ups about load tests for our server. We have already discussed how to compile metrics and how to use them to choose the equipment, and we also provided an overview of various load testing methods. Today we shall look at how the server handles stream mixing.

In the previous article we went over a load test whose data could be used to choose a load-appropriate server. In the course of the testing, we would publish a stream on one WCS, and we would pick up that stream several times using a second WCS. The acquired results could be used as a basis for decisions on server operability.

Some would (justly) have concerns regarding the possible biases in such a test — after all, one of our servers was used to test another one of our servers. Could it be that we were using a specially optimized code that skewed the results in our favor?

If you use Zabbix to monitor your infrastructure objects but have not previously thought about collecting and storing logs from these objects then this article is for you.

In any project, a great deal of importance is placed on the selection of server hardware and WebRTC streaming is no exception. One of the key principles of such a selection is balance – the hardware should be powerful enough to handle the streams with no drops in quality, but not too powerful so as to waste resources. So, how does one choose the right server?

Monitoring systems are a vital tool for any system administrator, because they can be used to extract specific information from services, such that:

This article is aimed to describe FreeBSD OS as a platform to launch cloud guest OSes with simple API capabilities, using MyBee and DevOPS I really like!

Network architecture at hyperscalers is a subject to constant innovation and is ever evolving to meet the demand. Network operators are constantly experimenting with solutions and finding new ways to keep it reliable and cost effective. Hyperscalers are periodically publishing their findings and innovations in a variety of scientific and technical groups.

The purpose of this article is to summarize the information about how hyperscalers design and manage networks. The goal here is to help connecting the dots, dissect and digest the data from a variety of sources including my personal experience working with hyperscalers.

DISCLAIMER: All information in this article is acquired from public resources. This article contains my own opinion which might not match and does not represent the opinion of my employer.

Microservices Architecture is a well-known pattern for building a complex system that consists of loosely coupled modules. It provides better scalability, and it is easier to develop a system in multiple teams so that they don’t interfere with each other too much. However, it is important to choose the right way of communication between the services. Otherwise, this kind of architecture can do more harm than good.

The PVS-Studio website turns 15 this year. This is quite significant for any internet resource. Back then, when our website was born, Russia announced 2006 as a year of humanities. That same year, in June, Denis Kryuchkov established a new platform, "Habrhabr" (now known as Habr). In November, Microsoft officially completed OS Windows Vista. That same month we registered the viva64.com domain.

We celebrated our domain's 10th anniversary with the website's redesign. After that, we would only change the resource capacity and features, but we'd never touch the design in any way. During this time, the number of articles grew so much that we needed to add tags to facilitate navigation. Right now we are also working on our YouTube channel. This means, you will see more and more new videos on our website as well. We keep adding new web pages at a tremendous rate, while the website's usability stays the same.

Time has come for big changes!

I have already written about AIOps and machine learning methods in working with IT incidents, about hybrid umbrella monitoring and various approaches to service management. Now I would like to share a very specific algorithm, how one can quickly get information about functioning conditions of business applications using synthetic monitoring and how to build, on this basis, the health metric of business services at no special cost. The story is based on a real case of implementing the algorithm into the IT system of one of the airlines.

Currently there are many APM systems, such as Appdynamics, Dynatrace, and others, having a UX control module inside that uses synthetic checks. And if the task is to learn about failures quicker than customers, I will tell you why all these APM systems are not needed. Also, nowadays health metrics are a fashionable feature of APM and I will show how you can build them without APM. 

The year 2021 started on such a high note for Qrator Labs: on January 19, our company celebrated its 10th anniversary. Shortly after, in February, our network mitigated quite an impressive 750 Gbps DDoS attack based on old and well known DNS amplification. Furthermore, there is a constant flow of BGP incidents; some are becoming global routing anomalies. We started reporting in our newly made Twitter account for Qrator.Radar.

Nevertheless, with the first quarter of the year being over, we can take a closer look at DDoS attacks statistics and BGP incidents for January - March 2021.

In this article, we benchmark the performance of MySQL 8 default configuration vs. innodb_dedicated_server enabled configuration vs. the configuration recommended by MySQL Performance Tuning Service.