Opened 3 years ago

Closed 3 years ago

#5007 closed defect (fixed)

Kea always updates IPV6 rDNS entry at each renew time

Reported by: zaffron Owned by: tmark
Priority: medium Milestone: Kea1.1-final
Component: ddns Version: 1.1.0-beta
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: Medium
Sub-Project: DHCP Feature Depending on Ticket:
Estimated Difficulty: 0 Add Hours to Ticket: 2
Total Hours: 11 Internal?: no

Description

I have IPv6 & DDNS configured.

Kea does update both the forward & reverse mapping in named.

However, when a system wants to renew the address, kea attempts to update the rDNS entry.

It comes back with the error ".... update unsuccessful: laptop-eth6.XXXXXXX.co.uk: 'name not in use' prerequisite not satisfied (YXDOMAIN)".

It then deletes the entry and reinserts it.

The DDNS entries are correct & only happens with the reverse entry, the forward entry just happily sits there.

This has happened both with 1.1.0-beta & 1.0.0 running on Linux.

It happens with both x64 Linux & ARM.

It also happens on the same subnet as the server & on a different subnet through a Cisco router.

IPv4 works as expected.

Subtickets

Attachments (4)

output-ddns.txt.gz (1.5 KB) - added by zaffron 3 years ago.
output-dhcp6.txt.gz (15.7 KB) - added by zaffron 3 years ago.
output-named.txt.gz (1010 bytes) - added by zaffron 3 years ago.
kea_conf.gz (2.9 KB) - added by zaffron 3 years ago.

Download all attachments as: .zip

Change History (17)

comment:1 Changed 3 years ago by tmark

Hello:

While we're looking at this it would helpful to see the log output for both kea-dhcp6 and kea-ddns, for the initial lease assignment and the subsequent renew. If you don't mind sharing your config that would help as well.

Thanks,

Thomas

Changed 3 years ago by zaffron

Changed 3 years ago by zaffron

Changed 3 years ago by zaffron

comment:2 Changed 3 years ago by zaffron

4 files uploaded. I had the lease time set to 3 minutes for testing.

Changed 3 years ago by zaffron

comment:3 Changed 3 years ago by tmark

  • Owner set to tmark
  • Status changed from new to reviewing
  • Total Hours changed from 0 to 1

Hello Zaffron:

First, let me thank for you supplying the files, they were very helpful. From what I can see everything is operating as it should be. For ease of life, we lovingly refer to
kea-dhcp-ddns as D2. It's just shorter. When D2 attempts to add (or replace) DNS entries it follows the state diagram shown here:

http://kea.isc.org/wiki/DhcpDdnsDesign#NameAddTransactionStateModel

(You'll find a text walk thru below the diagram).

If the request has both forward and reverse udpates, it will attempt the forward update first. After selecting a forward DNS server, it attempts to add the forward entry (see state "AddingFwdAddrs?"). The DNS update sent to the DNS server for an add always contains a prerequisite that the name not already exist.

The "error" message you see in named's output, is simply the named logging the fact that this prequisite has failed. In other words, the name that D2 as asked to add is already use, hence the status YXDOMAIN. YXDOMAIN simply means "a name that ought not to exist, does exist".

Per the state model, when the add fails because the name is already in use, we then attempt to replace the DNS entries (see state ReplacingFwdAddrs?). This consists
of sending a DNS update to the DNS server that first deletes the existing entries and then adds the new ones.

Looking at an excerpt from your named-output.txt:

This line is the add prerequisite detecting that the name is actually in use:

2016-09-13T16:03:50.476272+01:00 eth7 named[29918]: client 81.187.35.205#42807/key ipv6.bgcomp.co.uk: updating zone 'ipv6.bgcomp.co.uk/IN': update unsuccessful: powerpi.ipv6.bgcomp.co.uk: 'name not in use' prerequisite not satisfied (YXDOMAIN)

D2 then creates the replacement DNS update, which first deletes the entries:

2016-09-13T16:03:50.476942+01:00 eth7 named[29918]: client 81.187.35.205#48572/key ipv6.bgcomp.co.uk: signer "ipv6.bgcomp.co.uk" approved
2016-09-13T16:03:50.477296+01:00 eth7 named[29918]: client 81.187.35.205#48572/key ipv6.bgcomp.co.uk: updating zone 'ipv6.bgcomp.co.uk/IN': deleting rrset at 'powerpi.ipv6.bgcomp.co.uk' AAAA

and then adds them:

2016-09-13T16:03:50.477492+01:00 eth7 named[29918]: client 81.187.35.205#48572/key ipv6.bgcomp.co.uk: updating zone 'ipv6.bgcomp.co.uk/IN': adding an RR at 'powerpi.ipv6.bgcomp.co.uk' AAAA
2016-09-13T16:03:50.477873+01:00 eth7 named[29918]: client 2001:8b0:ca:2::fd#35197/key 7.0.0.0.a.c.0.0.0.b.8.0.1.0.0.2.ip6.arpa: signer "7.0.0.0.a.c.0.0.0.b.8.0.1.0.0.2.ip6.arpa" approved

This is reflected in D2's log:

We dequeue the request form kea-dhcp6 and start the NCR transaction for it:

2016-09-13 16:03:50.474 DEBUG [kea-dhcp-ddns.dhcp-to-d2/27872] DHCP_DDNS_QUEUE_MGR_QUEUE_RECEIVE Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD: received and queued a request.
2016-09-13 16:03:50.474 DEBUG [kea-dhcp-ddns.d2-to-dns/27872] DHCP_DDNS_STARTING_TRANSACTION Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD:

We attempt to add the entry, and "succeed" in detecting the name is in use (status = YXDOMAIN):

2016-09-13 16:03:50.474 DEBUG [kea-dhcp-ddns.d2-to-dns/27872] DHCP_DDNS_UPDATE_REQUEST_SENT Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD: Foward Add to server: 81.187.35.205 port:53
2016-09-13 16:03:50.475 DEBUG [kea-dhcp-ddns.asiodns/27872] ASIODNS_FETCH_COMPLETED upstream fetch to 81.187.35.205(53) has now completed
2016-09-13 16:03:50.475 DEBUG [kea-dhcp-ddns.d2-to-dns/27872] DHCP_DDNS_UPDATE_RESPONSE_RECEIVED Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD: to server: 81.187.35.205 port:53 status: SUCCESS, rcode: YXDOMAIN

So now we attempt to and succeed at replacing then entries:

2016-09-13 16:03:50.475 DEBUG [kea-dhcp-ddns.d2-to-dns/27872] DHCP_DDNS_UPDATE_REQUEST_SENT Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD: Forward Replace to server: 81.187.35.205 port:53
2016-09-13 16:03:50.476 DEBUG [kea-dhcp-ddns.asiodns/27872] ASIODNS_FETCH_COMPLETED upstream fetch to 81.187.35.205(53) has now completed
2016-09-13 16:03:50.476 DEBUG [kea-dhcp-ddns.d2-to-dns/27872] DHCP_DDNS_UPDATE_RESPONSE_RECEIVED Request ID 000201286220659FA678046CEF1D9849E09FF148F6FFBB03CB031DF2C763D74140CAFD: to server: 81.187.35.205 port:53 status: SUCCESS, rcode: NOERROR

Hopefully this clears things up for you. It is great to see people using Kea and putting it through its paces. If you have suggestions, questions, or find other issues please let us know. You may also find answers by posting to our user community list, kea-users@…. Our users are pretty enthusiastic and often answer before we can.

Thanks

Thomas Markwalder
ISC Software Engineering

comment:4 Changed 3 years ago by zaffron

I'm sorry, but the weevils have eaten the cable...

When the client has requested an IPv6 address, D2 correctly asked named to update itself.

When the client requests to renew the IPv6 address, D2 should not be doing anything, it should be a NOOP. There is no need to ask named to update itself, it already did that. Doing so ony generates errors.

The IPv4 D2 does not do this and that should be the correct behavior.

During renew IPv6 D2, should not be going anywhere near that state diagram as there is no need whatsoever.

Again, it not at the initial request stage for an IPv6 address, it is when the client attempts to renew it.

Last edited 3 years ago by zaffron (previous) (diff)

comment:5 Changed 3 years ago by tmark

Ah, that is indeed a different issue. My apologies for not picking up on that. We'll look into it.

Thanks

Thomas

comment:6 Changed 3 years ago by tmark

  • Add Hours to Ticket changed from 0 to 7
  • Owner changed from tmark to Unassigned
  • Total Hours changed from 1 to 6

As reported, kea-dhcp6 does indeed conduct DNS updates on renewals even when the FQDN has not changed.

The approach taken was to use the AllocEngine::ClientContext6 flags, fwd_dns_udpate_ and rev_dns_update, to gate whether or not NameChangeRequests? are created by Dhcp6Srv::createNameChangeRequests. These flags normally reflect the values of the flags in the lease, Lease::fqdn_fwd_ and Lease::fqdn_rev_.

In cases where the Allocation Engine reuses/extends a a lease and it detects that the FQDN has not changed, it sets both the context flags to false. The lease flags are left intact so they'll continue to reflect the state of these flags when the lease was created.

Proposed ChangeLog?:

11xx.   [bug]       tmark
    Modifed kea-dhcp6 to avoid requesting DNS updates when
    existing leases are renewed without changes to the FQDN.
    (Trac #5007, git TBD)

Ticket is ready for review.

comment:7 Changed 3 years ago by marcin

  • Owner changed from Unassigned to marcin

comment:8 follow-up: Changed 3 years ago by marcin

  • Owner changed from marcin to tmark

Reviewed commit 46f4c9fdd841b3cb26afb925b9386fbe3534e89d

src/bin/dhcp6/dhcp6_srv.cc
The comment in line 1201. Is this really true: "Primarily when existing leases are being extended without FQDN changes in which case the context flags will both be false." ??

If FQDN is not changed it does not mean that the both flags are false. It only means that the nae hasn't changed but the name can be still in DNS and AAAA and/or PTR may exist. Am I missing something?

src/lib/dhcpsrv/alloc_engine.cc
I was scratching my head why you need this:

// Callers should use the context flags to drive whether or
// not we do an update, not the lease flags.  Changing those
// would alter the flags in the database, which we want to
// preserve them as they were were when the lease was created.
ctx.fwd_dns_update_ = false;
ctx.rev_dns_update_ = false;

First of all, note that there are some errors and word repetitions in the commentary. But, in general the commentary doesn't clearly explain how this affects the operation of the server.

I think I now understand that for the cases when you don't send removal NCR (because the FQDN information hasn't changed) you also zero the context flags to avoid sending ADD NCRs in createNameChangeRequests function. This seems reasonable, because no removal means nothing to add if FQDN inforamtion exists in DNS already. But.... doesn't it assume that you're dealing with only one IPv6 address for a client? What if the client has multiple addresses assigned (multiple IA_NAs)? Don't you want to send an NCR for each address assigned? The createNameChangeRequests function iterates over the existing IA_NAs, so in my opinion the fact that we don't change the FQDN information for one lease, doesn't mean that we (for sure) shouldn't change the FQDN information for another lease, in which case you probably shouldn't be resetting global context values based on the fact that you don't replace DNS entry for a particular lease? In fact, the Dhcpv6Srv::assignIA_NA function resets the context values so your setting to "false" will be overriden anyway if there is more than one IA_NA being processed.

Sorry, for a little lengthy commentary. This is because I am still uncertain how this all should work in case there are multiple IA options, but I suspect that the current behavior is broken and we should have a unit test that covers the case of at least two IA_NAs.

I am thinking that the FQDN information within the ClientContext6 should really be a per-IA context information (namely belong to ClientContext6::IAContext()).

comment:9 in reply to: ↑ 8 Changed 3 years ago by tmark

  • Add Hours to Ticket changed from 7 to 2
  • Owner changed from tmark to marcin
  • Total Hours changed from 6 to 8

Replying to marcin:

Reviewed commit 46f4c9fdd841b3cb26afb925b9386fbe3534e89d

src/bin/dhcp6/dhcp6_srv.cc
The comment in line 1201. Is this really true: "Primarily when existing leases are being extended without FQDN changes in which case the context flags will both be false." ??

If FQDN is not changed it does not mean that the both flags are false. It only means that the nae hasn't changed but the name can be still in DNS and AAAA and/or PTR may exist. Am I missing something?

src/lib/dhcpsrv/alloc_engine.cc
I was scratching my head why you need this:

// Callers should use the context flags to drive whether or
// not we do an update, not the lease flags.  Changing those
// would alter the flags in the database, which we want to
// preserve them as they were were when the lease was created.
ctx.fwd_dns_update_ = false;
ctx.rev_dns_update_ = false;

First of all, note that there are some errors and word repetitions in the commentary. But, in general the commentary doesn't clearly explain how this affects the operation of the server.

I think I now understand that for the cases when you don't send removal NCR (because the FQDN information hasn't changed) you also zero the context flags to avoid sending ADD NCRs in createNameChangeRequests function. This seems reasonable, because no removal means nothing to add if FQDN inforamtion exists in DNS already. But.... doesn't it assume that you're dealing with only one IPv6 address for a client? What if the client has multiple addresses assigned (multiple IA_NAs)? Don't you want to send an NCR for each address assigned? The createNameChangeRequests function iterates over the existing IA_NAs, so in my opinion the fact that we don't change the FQDN information for one lease, doesn't mean that we (for sure) shouldn't change the FQDN information for another lease, in which case you probably shouldn't be resetting global context values based on the fact that you don't replace DNS entry for a particular lease? In fact, the Dhcpv6Srv::assignIA_NA function resets the context values so your setting to "false" will be overriden anyway if there is more than one IA_NA being processed.

Sorry, for a little lengthy commentary. This is because I am still uncertain how this all should work in case there are multiple IA options, but I suspect that the current behavior is broken and we should have a unit test that covers the case of at least two IA_NAs.

I am thinking that the FQDN information within the ClientContext6 should really be a per-IA context information (namely belong to ClientContext6::IAContext()).

As you and I discussed, DDNS for mulitple addresses A: is explicitly barred in createNameChangeRequests and B: to properly supported requires a good deal of work in kea-dhcp6, the allocation engine, the dhcp_ddns library, and D2. DNS supports multiple addresses for single FQDN but they must be updated within the same DNS request. This work should be done under another ticket, starting with a mini-design.

We should also document as a known issue limitations with our current scheme of only creating DNS entries for the first address IA we find in a given response. What this implies is client requesting mulitple address IAs would have to always do so in the same order, for this to reliably work. Simply put, for Kea 1.1 anyone doing mulitple IAs should probably not attempt to use DDNS.

As to the work on this ticket, I dug a bit more and have revised the solution to work essentially as follows:

  1. When, during the allocation process an exisitng lease is reused/extended, the original lease is pushed onto the context's changed_leases_ list.
  1. In Dhcp6::createNameChangeRequest() we look for each address IA in the changed_leases_ list. If it is there and the FQDN on that lease is the same as the

response's FQDN domain name, then we skip that IA and try the next address. That way,
we will only create an NCR if address's changed_leases_ entry has different FQDN. It is
We test the FQDN as changed leases may also have changed only by lifetime and currently we do not update DNS for that. For new leases the changed_leases_ list should be empty and we should end up using the first address IA we find.

So this approach is less hackish then manipulating the context flags in that it at least attempts to use the context has it was intended but we must acknowledge we have some work to do in this area in general.

It should give anyone using single address IAs pretty workable DDNS.

Please re-review.

comment:10 follow-up: Changed 3 years ago by marcin

  • Add Hours to Ticket changed from 2 to 1
  • Owner changed from marcin to tmark
  • Total Hours changed from 8 to 9

Reviewed commit 1a986cf434453bdf6ccbcef5d707a23a23305093

src/bin/dhcp6/dhcp6_srv.cc
The following new piece of code:

// see if the lease for iaadr is in changed_leases, and if so
// if the FQDN is different, if not continue
bool extended_only = false;
for (Lease6Collection::const_iterator l = ctx.currentIA().changed_leases_.begin();
     l != ctx.currentIA().changed_leases_.end(); ++l) {
    if ((*l)->addr_ == iaaddr->getAddress()) {
        if ((*l)->hostname_ == opt_fqdn->getDomainName()) {
            extended_only = true;
            break;
        }
    }
}

if (extended_only) {
    continue;
}

The comment is unclear. Specifically, "and if so if the FQDN is different".

I wonder if instead of doing this:

if ((*l)->hostname_ == opt_fqdn->getDomainName()) {
...

you should also check fwd and rev flags. Assuming that the FQDN hasn't changed in the extended lease, but previously only a reverse flag was set, and now the server has been reconfigured to also perform forward update, you need to generate the NCR to perform fwd update?

comment:11 in reply to: ↑ 10 Changed 3 years ago by tmark

  • Add Hours to Ticket changed from 1 to 2
  • Owner changed from tmark to marcin
  • Total Hours changed from 9 to 11

Replying to marcin:

Reviewed commit 1a986cf434453bdf6ccbcef5d707a23a23305093

src/bin/dhcp6/dhcp6_srv.cc
The following new piece of code:

// see if the lease for iaadr is in changed_leases, and if so
// if the FQDN is different, if not continue
bool extended_only = false;
for (Lease6Collection::const_iterator l = ctx.currentIA().changed_leases_.begin();
     l != ctx.currentIA().changed_leases_.end(); ++l) {
    if ((*l)->addr_ == iaaddr->getAddress()) {
        if ((*l)->hostname_ == opt_fqdn->getDomainName()) {
            extended_only = true;
            break;
        }
    }
}

if (extended_only) {
    continue;
}

The comment is unclear. Specifically, "and if so if the FQDN is different".

I have revised the commentary.

I wonder if instead of doing this:

if ((*l)->hostname_ == opt_fqdn->getDomainName()) {
...

you should also check fwd and rev flags. Assuming that the FQDN hasn't changed in the extended lease, but previously only a reverse flag was set, and now the server has been reconfigured to also perform forward update, you need to generate the NCR to perform fwd update?

Yes, I agree and I have revised the logic accordingly. I also added a unit test to verify the permutations of the flags.

Ready for re-re-review.

comment:12 Changed 3 years ago by marcin

  • Owner changed from marcin to tmark

Your changes look good. You're ready to go.

comment:13 Changed 3 years ago by tmark

  • Resolution set to fixed
  • Status changed from reviewing to closed

Changes merged with git # 05ea3a5eb75c06cf9814c63a1a54261bf58a954b
Added ChangeLog? entry 1174.

Ticket is closed.

Note: See TracTickets for help on using tickets.