0. Total results for each kind of outcome and server mode
1. ERRORS MAP: distribution of Firebird errors which occured during runs (0 rows)
2. FRESH FAILS: tests which failed for LAST execution only (0 rows)
3. FRESH ERRORS: tests which raised run-time error for LAST execution only (0 rows)
4. STILL FAILS: tests which fail for at least two last runs (6 rows)
[CURRENT] 5. Has fails: tests with at least one fail for previous runs (33 rows)
6. Has runtime errors: tests with at least one runtime error for previous runs (0 rows)
7. FRESH PASSED: successful tests with previous outcome = FAIL/ERROR/SKIPPED or absent (0 rows)
8. All tests: outcomes (2272 rows)
9. All tests: durations (4498 rows)
A. Skipped: tests which was intentionally excluded from execution (3 rows)

Generated 02.10.2022 08:23
Tested: FB50.   Build: 5.0.0.763;   CS: 02.10.22 07:20:00 - 02.10.22 08:21:27;   SS: 02.10.22 06:03:38 - 02.10.22 07:16:47;
OS and hardware info:  Linux 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5 (2019-06-19) ; logical CPUs: 2 ; total memory: 7.6 Gb 

Test with at least one result = "FAIL" for last 30 runs:
NN Test name mode 5.0.0.763
2022-10-02
5.0.0.762
2022-10-01
5.0.0.760
2022-09-30
5.0.0.758
2022-09-29
5.0.0.756
2022-09-27
5.0.0.745
2022-09-26
5.0.0.743
2022-09-24
5.0.0.742
2022-09-23
5.0.0.739
2022-09-21
5.0.0.736
2022-09-18
5.0.0.734
2022-09-17
5.0.0.733
2022-09-16
5.0.0.731
2022-09-15
5.0.0.730
2022-09-14
5.0.0.727
2022-09-10
5.0.0.721
2022-09-09
5.0.0.714
2022-09-07
5.0.0.706
2022-09-07
5.0.0.701
2022-09-03
5.0.0.698
2022-09-02
5.0.0.694
2022-09-02
5.0.0.675
2022-08-16
5.0.0.628
2022-08-09
5.0.0.623
2022-08-03
5.0.0.600
2022-08-02
5.0.0.599
2022-08-01
5.0.0.598
2022-07-26
5.0.0.592
2022-07-23
5.0.0.591
2022-07-20
5.0.0.590
2022-07-17
Test TITLE
1 CS     F         F         F         F         F         F         F         F         F         F     P P P P P P P P P P P P P P P P P P P P
Improve sorting performance when long VARCHARs are involved
2 SS     F         F         F         F         F         F         F         F         F         F     P P P P P P P P P P P P P P P P P P P P
Improve sorting performance when long VARCHARs are involved
3 CS P P P     F         F         F     P P P P P P P P P P P P P P P P P P P P P P P P
Using both the procedure name and alias inside an explicit plan crashes the server
4 SS P P P     F         F         F     P P P P P P P P P P P P P P P P P P P P P P P P
Using both the procedure name and alias inside an explicit plan crashes the server
5 SS P P P P P P P P P P     F     P P P P P P P P P P P P P P P P P P P
At least 5 seconds delay on disconnect could happen if disconnect happens close after Event Manager initialization
6 SS P P P P P P P P P P P P P P P P P     F     P P P P P     F         F     P     F     P P P
FBTRACEMGR should understand 'role <name>' command switch (needed to explicitly connect with role with 'TRACE_ANY_ATTACHMENT' privilege)
7 CS P P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P
Performance regression when bulk inserting into table with indices
8 CS P P P P P P P P P P P P P P P P P P P P P P P P     F     P P P P P
Bad optimization of some operations with views containing subqueries
9 CS P P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P
Badly selective index could be used for extra filtering even if selective index is used for sorting
10 CS P P P P P P P P P P P P P P P     F     P P P P P P P P P P P P P P
Sometime it is impossible to cancel\kill connection executing external query
11 SS P P P P P P P P P P P P P P P     F     P P P P P P P P P P P P P P
Sometime it is impossible to cancel\kill connection executing external query
12 SS P P P P P P P P P P P P P P P P P     F         F         F         F         F     P P P P P P P P
Built-in cryptographic functions
13 CS     F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F     P P P P P P P P
Ability to query Firebird configuration using SQL
14 SS     F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F     P P P P P P P P
Ability to query Firebird configuration using SQL
15 CS     F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F    
When the statement timeout is set, it causes the lock manager to delay reporting deadlocks until timeout is expired
16 SS     F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F         F    
When the statement timeout is set, it causes the lock manager to delay reporting deadlocks until timeout is expired
17 CS P P P P P P P P P P P P P     F     P P P P P P P P P P P P P P P P
Significant performance regression of SIMILAR TO and SUBSTRING(SIMILAR) when pattern is taken from variable (rather than directly specified)
18 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$FIELDS
19 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$FIELDS
20 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$FUNCTION_ARGUMENTS
21 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$FUNCTION_ARGUMENTS
22 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$FUNCTIONS
23 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$FUNCTIONS
24 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$PROCEDURE_PARAMETERS
25 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New DB - RDB$PROCEDURE_PARAMETERS
26 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New database: content of RDB$PROCEDURES
27 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
New database: content of RDB$PROCEDURES
28 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$SECURITY_CLASSES
29 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$SECURITY_CLASSES
30 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$TYPES
31 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$TYPES
32 CS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$USER_PRIVILEGES
33 SS P P P P P P P P P P P P P P P P P P P P P     F     P P P P P P P P
Empty DB - RDB$USER_PRIVILEGES

Elapsed time, milliseconds:
NN Test name mode median_ms 5.0.0.763
2022-10-02
5.0.0.762
2022-10-01
5.0.0.760
2022-09-30
5.0.0.758
2022-09-29
5.0.0.756
2022-09-27
5.0.0.745
2022-09-26
5.0.0.743
2022-09-24
5.0.0.742
2022-09-23
5.0.0.739
2022-09-21
5.0.0.736
2022-09-18
5.0.0.734
2022-09-17
5.0.0.733
2022-09-16
5.0.0.731
2022-09-15
5.0.0.730
2022-09-14
5.0.0.727
2022-09-10
5.0.0.721
2022-09-09
5.0.0.714
2022-09-07
5.0.0.706
2022-09-07
5.0.0.701
2022-09-03
5.0.0.698
2022-09-02
5.0.0.694
2022-09-02
5.0.0.675
2022-08-16
5.0.0.628
2022-08-09
5.0.0.623
2022-08-03
5.0.0.600
2022-08-02
5.0.0.599
2022-08-01
5.0.0.598
2022-07-26
5.0.0.592
2022-07-23
5.0.0.591
2022-07-20
5.0.0.590
2022-07-17
Test TITLE
1 CS 1055 990 1090 1080 1000 970 1050 1060 980 1100 1140 1070 1150 1120 1080 1020 1080 900 930 1010 1010 980 1130 1160 1110 1070 1030 1000 1050 1070 1030
Improve sorting performance when long VARCHARs are involved
2 SS 1295 1200 1200 1310 1190 1200 1300 1430 1190 1340 1420 1370 1410 1390 1330 1350 1390 1290 1230 1260 1190 1000 1280 1280 1240 1210 1350 1390 1300 1350 1200
Improve sorting performance when long VARCHARs are involved
3 CS 730 660 670 730 730 710 760 760 770 840 800 790 800 800 760 790 770 660 700 760 650 650 780 690 730 780 680 680 730 680 690
Using both the procedure name and alias inside an explicit plan crashes the server
4 SS 1000 980 1000 970 1000 1020 970 970 970 1040 1050 1140 1140 1060 1060 990 1090 980 980 910 940 1020 1030 930 970 970 1080 1070 1040 1060 980
Using both the procedure name and alias inside an explicit plan crashes the server
5 SS 16220 16080 15960 16860 15950 14800 15290 14780 15190 15760 16430 16060 16610 16170 17280 16340 17820 16590 16240 16970 16160 15870 16430 16040 16200 16320 17970 16820 17330 16810 15800
At least 5 seconds delay on disconnect could happen if disconnect happens close after Event Manager initialization
6 SS 4795 4770 4760 4880 4800 4940 4780 4860 4800 4940 4970 4990 4910 4990 4930 4990 4880 4660 4560 4570 4650 4780 4630 4730 4610 4690 4000 4790 4800 4810 4710
FBTRACEMGR should understand 'role <name>' command switch (needed to explicitly connect with role with 'TRACE_ANY_ATTACHMENT' privilege)
7 CS 70150 56710 56370 59810 55400 60330 59280 59560 55340 69570 70180 74030 71550 72570 70120 71820 71490 69800 72210 74690 73720 71550 80210 82980 77330 75330 72520 23520 22670 22780 22410
Performance regression when bulk inserting into table with indices
8 CS 2885 2710 2650 2730 2670 2760 2760 2710 2770 2900 2960 2990 2960 2950 2970 3030 2910 2730 2960 3050 3000 2880 3060 3030 2690 2870 2910 2830 2890 2800 2800
Bad optimization of some operations with views containing subqueries
9 CS 5570 5580 5610 5600 5470 5520 5570 5570 5490 5620 5810 5670 5820 5950 5650 5670 5920 5340 5620 5290 5660 5410 5480 5510 5460 5470 5460 5600 5370 5490 5440
Badly selective index could be used for extra filtering even if selective index is used for sorting
10 CS 132460 132100 132810 132290 132630 133140 132930 132010 132400 131830 132550 133100 133130 132210 132720 131470 132610 131540 132520 131730 132200 132540 131660 132790 132470 132900 132510 132450 131330 131340 131780
Sometime it is impossible to cancel\kill connection executing external query
11 SS 133545 133650 133610 133650 132610 132620 133590 133550 132590 134770 133690 133790 134760 133740 133780 132710 132820 132490 132460 133440 133540 133510 132560 134480 132450 133480 132470 133630 133600 133640 133530
Sometime it is impossible to cancel\kill connection executing external query
12 SS 2205 1950 2010 2120 2010 1970 1930 2010 1910 2190 2860 2260 2190 2960 2440 2340 2430 2300 2340 2000 1960 2090 2310 2310 2310 2510 2140 2130 2310 2230 2220
Built-in cryptographic functions
13 CS 2635 2470 2430 2580 2600 2470 2370 2600 2480 2670 2620 2660 2740 2780 2600 2650 2710 2740 2570 2590 2710 2610 2830 2720 2790 2690 2710 2680 2600 2610 2760
Ability to query Firebird configuration using SQL
14 SS 2795 2560 2540 2780 2000 2610 2690 2630 2670 2910 2850 2710 2810 2000 2820 2720 2800 2840 2790 2720 2840 2740 2790 2840 2930 2810 2850 2860 2800 3010 2850
Ability to query Firebird configuration using SQL
15 CS 4170 4180 4570 3850 4030 4420 3880 3860 3820 3700 4160 4690 4340 3910 4540 4450 3990 4580 4570 4560 4670 4400 4320 3940 3770 4460 4510 4000 3940 3800 3840
When the statement timeout is set, it causes the lock manager to delay reporting deadlocks until timeout is expired
16 SS 4375 3990 4360 4000 4030 4550 4700 4030 4670 4070 4550 4660 3900 4480 4130 4790 4770 4140 4000 4390 4050 3930 4010 4450 4180 4390 4790 4630 4930 4600 4140
When the statement timeout is set, it causes the lock manager to delay reporting deadlocks until timeout is expired
17 CS 4580 4370 4490 4360 4760 4360 4240 4550 4770 4700 4630 4540 4510 4380 4810 4540 4360 4600 4450 4430 4710 4560 4720 4610 4520 4810 4700 4950 5180 4840 4880
Significant performance regression of SIMILAR TO and SUBSTRING(SIMILAR) when pattern is taken from variable (rather than directly specified)
18 CS 665 620 610 650 670 650 650 650 660 680 660 710 690 720 700 700 680 640 630 630 710 670 630 690 680 660 640 670 680 640 710
Empty DB - RDB$FIELDS
19 SS 865 840 760 840 850 800 780 760 810 930 940 980 890 960 870 860 850 860 840 920 830 900 820 870 910 970 900 900 850 880 890
Empty DB - RDB$FIELDS
20 CS 560 530 550 590 480 530 600 480 480 540 610 540 630 560 630 600 580 560 490 550 600 520 650 590 570 570 570 580 540 540 530
New DB - RDB$FUNCTION_ARGUMENTS
21 SS 775 740 720 740 740 700 760 680 770 830 0 800 820 810 800 830 740 760 740 740 800 780 730 850 770 900 800 780 810 790 0
New DB - RDB$FUNCTION_ARGUMENTS
22 CS 560 560 580 560 540 550 540 590 510 630 600 550 560 610 590 550 610 580 560 520 590 520 540 620 530 590 610 550 520 530 570
New DB - RDB$FUNCTIONS
23 SS 775 760 710 710 740 690 710 690 740 0 800 860 840 820 0 810 800 720 810 680 760 780 730 810 800 820 820 860 760 850 770
New DB - RDB$FUNCTIONS
24 CS 570 560 560 570 550 540 600 560 560 580 610 610 620 590 620 590 630 530 560 520 610 530 540 610 580 570 590 570 570 570 550
New DB - RDB$PROCEDURE_PARAMETERS
25 SS 785 710 0 720 740 920 710 760 0 900 760 840 880 970 820 840 820 770 700 800 770 760 710 770 760 870 760 800 820 820 820
New DB - RDB$PROCEDURE_PARAMETERS
26 CS 570 570 550 590 0 550 580 530 510 580 580 560 560 650 560 610 630 600 540 510 530 570 570 590 580 570 590 590 560 550 560
New database: content of RDB$PROCEDURES
27 SS 755 720 740 740 700 730 730 730 740 820 790 820 770 770 810 800 850 730 700 0 0 700 740 790 720 830 830 840 800 840 710
New database: content of RDB$PROCEDURES
28 CS 720 690 760 700 670 670 690 630 690 0 0 830 770 790 780 740 0 720 720 670 700 720 810 710 770 740 740 680 710 760 700
Empty DB - RDB$SECURITY_CLASSES
29 SS 920 830 910 900 900 850 840 840 850 970 990 950 1000 1030 920 950 950 900 810 880 920 880 1030 960 950 940 910 1010 1000 1000 890
Empty DB - RDB$SECURITY_CLASSES
30 CS 620 580 580 680 620 590 640 600 550 660 640 630 660 620 630 620 660 600 550 560 620 630 630 590 570 630 620 640 620 620 560
Empty DB - RDB$TYPES
31 SS 820 760 800 820 790 760 780 810 780 930 910 780 860 890 840 840 860 820 830 0 840 800 800 900 800 830 790 870 860 850 800
Empty DB - RDB$TYPES
32 CS 740 670 730 770 710 0 740 700 690 780 0 790 730 790 810 810 800 710 790 760 0 740 800 740 790 720 810 720 710 720 740
Empty DB - RDB$USER_PRIVILEGES
33 SS 945 820 940 920 870 910 850 800 860 930 1040 1080 960 1050 930 1030 970 860 950 870 950 970 940 970 900 960 940 1030 970 1140 1070
Empty DB - RDB$USER_PRIVILEGES

List of tests in this report:


http://tracker.firebirdsql.org/browse/core-2650
http://tracker.firebirdsql.org/browse/core-3064
http://tracker.firebirdsql.org/browse/core-5034
http://tracker.firebirdsql.org/browse/core-5269
http://tracker.firebirdsql.org/browse/core-5302
http://tracker.firebirdsql.org/browse/core-5393
http://tracker.firebirdsql.org/browse/core-5435
http://tracker.firebirdsql.org/browse/core-5685
http://tracker.firebirdsql.org/browse/core-5970
http://tracker.firebirdsql.org/browse/core-6444
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_07.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_11.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_12.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_19.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_20.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_26.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_30.fbt
https://github.com/FirebirdSQL/fbt-repository/blob/master/tests/functional/basic/db/db_31.fbt
https://github.com/FirebirdSQL/firebird/issues/6802
https://github.com/FirebirdSQL/firebird/issues/6875

=== end of list ===