Opened 7 years ago

Closed 7 years ago

#2605 closed task (fixed)

DHCP Testing: Performance Tool

Reported by: stephen Owned by: tmark
Priority: medium Milestone: Sprint-DHCP-20130411
Component: dhcp Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: DHCP Feature Depending on Ticket:
Estimated Difficulty: 0 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no

Description

Write a description of the tests for section 7 ("Performance Tool") of the Test Plan, and run them. (Ultimately the tests will be automated, but for the release of the 2012 work, the testing is manual.)

The description and results should be attached to this ticket.

Note that the tool has been tested in the past. This ticket is to create the beginnings of a full set of system tests for the tool.

Subtickets

Attachments (1)

trac2605.artifacts.tgz (75.2 KB) - added by tmark 7 years ago.
Test Artifacts from test runs

Download all attachments as: .zip

Change History (13)

comment:1 Changed 7 years ago by tmark

  • Owner set to tmark
  • Status changed from new to accepted

Changed 7 years ago by tmark

Test Artifacts from test runs

comment:2 Changed 7 years ago by tmark

Created two sets of tests, one for v4 one for v6,
three tests each:

  1. A two-packet test
  2. A four-packet test
  3. A test with 105 simulated clients

tests/common/v4.perfdhcp

v4.four_packet.1.txt
v4.plus_100_clients.1.txt
v4.two_packet.1.txt
kea.v4.perfdhcp.1.cfg

tests/common/v6.perfdhcp

v6.two_packet.1.txt
v6.four_packet.1.txt
v6.plus_100_clients.1.txt
kea.v6.perfdchp.1.cfg

Three issues were discovered and tickets created:

#2783: Using -n<x> to speifcy number of exhanges always yields x-1 exchanges, with
1 drop

#2784: perfdhcp cores when given command line "-6" and v4 server address

#2785: perfdhcp sent/received durations do not appear to be accurate

Test runtime artifacts are in the attached tar ball.

comment:3 Changed 7 years ago by tmark

  • Owner changed from tmark to UnAssigned
  • Status changed from accepted to reviewing

comment:4 Changed 7 years ago by marcin

  • Owner changed from UnAssigned to marcin

comment:5 follow-up: Changed 7 years ago by marcin

  • Owner changed from marcin to tmark

Reviewed commit faea2bea67573fa7932234fa0504e4c107e1a323

General comments to all tests
I suggest that test procedure does not specify example configuration but rather instructs to use explicit values. There could be a statement in the ''Test Setup'' instructing to use different configuration than given in the test procedure in case of conflict with test execution environment. But, it should be then mentioned that the cfg file should be modified accordingly.

v6.four_packet.1
(1.) Test procedure does not instruct to start the server.

(2.) I understand what to do. But, the person who is not familiar with the subject may get confused when reading ''Build/Install? BIND10/perfdhcp on the client machine''. Should I build BIND10 and install perfdhcp? Or, Build and Install both BIND10 and perfdhcp?

(Pass Criteria):
Checking perfdhcp output is ok. Once perfdhcp generates 4 or 2-way exchanges for the number of clients it simulates the client id IS EXPECTED to be reset and the next exchanges are initiated by the clients for which leases should exist in the data base. In other words, if you simulate 100 clients and you setup perfdhcp to finish after 105 requests you should have 100 leases in the lease database. Right now, the test simulates 105 clients and does exactly 105 requests so the client reset is not verfied.

For all other tests exactly the same comments apply.

comment:6 in reply to: ↑ 5 ; follow-up: Changed 7 years ago by tmark

  • Owner changed from tmark to marcin

Replying to marcin:

Reviewed commit faea2bea67573fa7932234fa0504e4c107e1a323

General comments to all tests
I suggest that test procedure does not specify example configuration but rather instructs to use explicit values. There could be a statement in the ''Test Setup'' instructing to use different configuration than given in the test procedure in case of conflict with test execution environment. But, it should be then mentioned that the cfg file should be modified accordingly.

v6.four_packet.1
(1.) Test procedure does not instruct to start the server.

(2.) I understand what to do. But, the person who is not familiar with the subject may get confused when reading ''Build/Install? BIND10/perfdhcp on the client machine''. Should I build BIND10 and install perfdhcp? Or, Build and Install both BIND10 and perfdhcp?

Corrected the above issues for all the tests.

(Pass Criteria):
Checking perfdhcp output is ok. Once perfdhcp generates 4 or 2-way exchanges for the number of clients it simulates the client id IS EXPECTED to be reset and the next exchanges are initiated by the clients for which leases should exist in the data base. In other words, if you simulate 100 clients and you setup perfdhcp to finish after 105 requests you should have 100 leases in the lease database. Right now, the test simulates 105 clients and does exactly 105 requests so the client reset is not verfied.

Actually both of the 105 client tests already have a step under pass criteria
to verify the number of leases. You just didn't look hard enough.

I did add text under the server startup on how to collect the lease data. I also added steps
to verify the lease data for the other four tests just for completeness.

comment:7 in reply to: ↑ 6 Changed 7 years ago by marcin

  • Owner changed from marcin to tmark

Replying to tmark:

(Pass Criteria):
Checking perfdhcp output is ok. Once perfdhcp generates 4 or 2-way exchanges for the number of clients it simulates the client id IS EXPECTED to be reset and the next exchanges are initiated by the clients for which leases should exist in the data base. In other words, if you simulate 100 clients and you setup perfdhcp to finish after 105 requests you should have 100 leases in the lease database. Right now, the test simulates 105 clients and does exactly 105 requests so the client reset is not verfied.

Actually both of the 105 client tests already have a step under pass criteria
to verify the number of leases. You just didn't look hard enough.

Yes. I saw there is a step to verify that 105 leases has been added to the database. But, I was rather referring to the fact that you simulate 105 clients and run 105 exchanges which results in 105 unique leases. My suggestion was to simulate 100 clients and trigger 105 requests and expect to have 100 unique leases. Similar scenario is covered by v4.four_packet test but it simulates 1 client requesting lease five times. This is sort of special case when you simulate only one client so I thought it could be extended in the v4.plus_100_clients test.

comment:8 Changed 7 years ago by tmark

  • Owner changed from tmark to marcin

Altered the 100 plus client tests to simulate 101 clients
through 105 exchanges. This adds verification that perfdhcp will
reset the client id after 101 exchanges.
Corrected misspelling of configuration file name as well as config
file contents which contained "syntax" errors.

comment:9 Changed 7 years ago by marcin

  • Owner changed from marcin to tmark

Just two minor things ...

v4.plus_100_clients.1.txt
It would make sense to call output file v4_101_client.out instead of v4_105_client.out as you simulate 101, not 105 clients. This has been done for other tests already.

The following sentence is not quite right:

The first 101 exchanges will use unique client ids, after which the client id will be reset and perfdhcp will continue but simulating only one client for four more exchanges

In fact it will just reset the clientid and will start over increasing it for following exchange. So for example, for the remaining four exchanges may use client id's like this

  • 00000001
  • 00000002
  • 00000003
  • 00000004

Because clients with these ids already have a lease the new leases will not be allocated but rather existing leases will be returned.

Please update the test description accordingly.

comment:10 Changed 7 years ago by tmark

  • Owner changed from tmark to marcin

Output file name changed as suggested and description of client id behavior has been updated in both v4 and v6 versions.

comment:11 Changed 7 years ago by marcin

  • Owner changed from marcin to tmark

Reviewed be62f17b8ce19d79af3885b0162a44edbcc5fb83

Everything is ok. Please merge.

comment:12 Changed 7 years ago by tmark

  • Resolution set to fixed
  • Status changed from reviewing to closed

Changes merged.

Note: See TracTickets for help on using tickets.