Opened 7 years ago

Closed 5 years ago

#2293 closed defect (wontfix)

solve frequent BenchMarkTest.run failures

Reported by: jreed Owned by: UnAssigned
Priority: medium Milestone: Remaining BIND10 tickets
Component: build system Version: bind10-old
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: Core Feature Depending on Ticket:
Estimated Difficulty: 5 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no

Description

We have had near 100 timing failures in BenchMarkTest?. This is on virtual machines: FreeBSD, NetBSD, and Debian5 (but none of the other VMs) and on physical Solaris Sparc machine (but no other physical systems).

Here is an example:

[ RUN      ] BenchMarkTest.run
benchmark_unittest.cc:119: Failure
Expected: (sleep_time + duration_margin) > (bench.getDuration() * ONE_MILLION), actual: 75000 vs 75077
benchmark_unittest.cc:125: Failure
Expected: ((sleep_time + duration_margin) / static_cast<double>(sub_iterations)) > (bench.getAverageTime() * ONE_MILLION), actual: 3260.8695652173915 vs 3264.217391304348
benchmark_unittest.cc:128: Failure
Expected: (static_cast<double>(sub_iterations) / (sleep_time + duration_margin)) < (bench.getIterationPerSecond() / ONE_MILLION), actual: 0.00030666666666666668 vs 0.00030635214513099882
[  FAILED  ] BenchMarkTest.run (129 ms)

Let's rework this so BenchMarkTest?.run does not fail like this anymore.

Subtickets

Change History (4)

comment:1 Changed 7 years ago by jinmei

Maybe just disable the offending tests (either by default or on these VMs)? After all it's just an internal tool.

comment:2 Changed 7 years ago by shane

  • Component changed from Unclassified to build system
  • Milestone New Tasks deleted
  • Owner set to UnAssigned

One problem with disabling these is that users who run the tests themselves on slower machines may get test failures.

Perhaps we should make it an optional test, and enable it explicitly on most of our test servers instead?

comment:3 Changed 6 years ago by tomek

  • Milestone set to Remaining BIND10 tickets

comment:4 Changed 5 years ago by tomek

  • Resolution set to wontfix
  • Status changed from new to closed
  • Version set to old-bind10

This task is no longer applicable to Kea. For Bundy
(a project that continues DNS development after BIND10 cancellation),
see http://bundy-dns.de

Closing ticket.

Note: See TracTickets for help on using tickets.