Opened 8 years ago

Closed 7 years ago

#1566 closed task (complete)

create a standard set of benchmarks

Reported by: jelte Owned by: jreed
Priority: high Milestone: Non-Development-2012Q3
Component: Unclassified Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: DNS Feature Depending on Ticket:
Estimated Difficulty: 0 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no

Description (last modified by jelte)

In order to be able to actually measure performance improvements over time, we need a standardized set of benchmarks; so that we don't just say 'X qps' (as one metric), but have a number of different scenarios, e.g. 'Authoritative server for X zones (with X delegations and Y records), tracing a file with...' etc., and a fixed system to run them on. We already have a number of arbitrarily picked benchmarks, but nothing 'official'. The plan is to run these on a dedicated benchmarking machine.

The task for this ticket is to write a document that defines these benchmarks, so that they can then be implemented in followup tasks.

There are some short notes about this in the F2F notes:

http://bind10.isc.org/wiki/January2012MeetingMinutes#DNSBenchmarking
and a short section before that anchor as well.

Subtickets

Change History (6)

comment:1 Changed 8 years ago by jelte

  • Description modified (diff)

comment:2 Changed 8 years ago by jelte

  • Milestone changed from Sprint-20120207 to Non-Development-2012Q1

comment:3 Changed 8 years ago by shane

  • Owner set to jreed
  • Priority changed from major to critical
  • Status changed from new to assigned

comment:4 Changed 8 years ago by jreed

I had an old, mostly unmaintained catchall wikipage at http://bind10.isc.org/wiki/TestFarm.

The new page will be at http://bind10.isc.org/wiki/DnsBenchmarks. I plan to send an email by Monday afternoon with my proposals for this.

comment:6 Changed 7 years ago by shane

  • Resolution set to complete
  • Status changed from assigned to closed

Jeremy, I think we can consider this ticket resolved for now. If we want more work (which we surely will) it can live in other tickets.

Note: See TracTickets for help on using tickets.