Load testing of IBProvider v3.50 and Firebird v3.0.4

Modern software and hardware have become extremely complex. You can feel it even in such a small project as IBProvider is. We tried to simplify it as much as possible. Nevertheless, it has become multilevel and multithreaded and it replaced some server components. That is why to control IBProvider quality we test it around the clock in a variety of ways.

In this note we would like to inform you about running 6.7 million various tests for blobs. On our scale, it is a big testing. Usually we run each IBProvider update through ~0.7 million tests.

Performing these 6.7 million tests in one thread would take 150 days. Parallel run in 10 threads reduced this time to 15 days.

We won’t bore you with details and will provide only key information.

Hardware

• Intel 6950x
• RAM: 128GB
• Storage: RAM Disk
• Windows 10 Pro x64

Tested components

• IBProvider v3.50.0.28657, x64, vc15 (VS2017).
• Firebird v3.0.4.32972, x64, SuperServer, VS2015.

We used direct (without fbclient.dll) connection to server via TCP/IP (localhost).

The connection pool was served by the components of «LCPI OLE DB Services».

Database

• Dialect: 3.
• Page size: 16KB.

Tests

Strategy: Bounded Exhaustive Testing.

We were checking the work with text blobs including with ICU encodings.

During tests running, forced garbage collection in the database was launched once without pausing the tests.

Results

20 tests found that need to be updated.

The tests worked 504 hours of «CPU User Time» (21 day). Resources (memory, descriptors) are ok, no leakages (logs).

The server worked 1221 hours of «CPU User Time» (50 days) in total. Server state before tests started. Server state after tests completed:

Database size: 57.4GB. Forced garbage collection helped to stop its growth. Database statistics:

Database "D:\DATABASE\RAM\IBP_TEST_FB30_D3_2.GDB"
Gstat execution time Sun Jun  3 19:44:51 2018
 
Database header page information:
        Flags                   0
        Generation              20148796
        System Change Number    0
        Page size               16384
        ODS version             12.2
        Oldest transaction      20147092
        Oldest active           20147093
        Oldest snapshot         20145317
        Next transaction        20147100
        Sequence number         0
        Next attachment ID      27215
        Implementation          HW=AMD/Intel/x64 little-endian OS=Windows CC=MSVC
        Shadow count            0
        Page buffers            0
        Next header page        0
        Database dialect        3
        Creation date           Apr 18, 2018 23:07:38
        Attributes
 
    Variable header data:
        *END*
Gstat completion time Sun Jun  3 19:44:51 2018

Additionally

System loading during the work.

Conclusions

The performed testing confirms the following key characteristics of IBProvider v3.50:

• Consistent interaction with Firebird database server.
• The quality of algorithms and mechanisms.
• Stable work under ultimate multithreaded load.

In addition, the performed testing can be considered preliminary confirmation of the stability of the next FB3 update – v3.0.4.