#5566 closed defect (fixed)

fix make distcheck with premium

Reported by: fdupont Owned by: UnAssigned
Priority: high Milestone: Kea1.4
Component: build system Version: git
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: DHCP Feature Depending on Ticket:
Estimated Difficulty: 0 Add Hours to Ticket: 2
Total Hours: 0 Internal?: no

Description

For instance install legal_log schemas...

Subtickets

Attachments (1)

d5566 (3.6 KB) - added by fdupont 21 months ago.
Fix for legal log problems

Download all attachments as: .zip

Change History (23)

comment:1 Changed 21 months ago by fdupont

Fixed a problem with radius when FreeRADIUS client is not available.
It seems there is no problem with legal log schemas?
BTW HA tests failed on my macOS (signaled some errors and stalled).

comment:2 follow-up: Changed 21 months ago by marcin

There are two more issues beyond Radius.

The legal_log/tests/mysql_unittests.cc has this include:

#include <testutils/mysql_legl_schema.h>

which includes the file in legal_log/testutils. This file is not included in the distribution when you do "make dist". As a result, compilation of the tarball (with unit tests included) fails due to lack of this file. I wouldn't rely so much on he distcheck as it is possible that the configuration parameters are not correctly propagated from configure step to make dist. Instead, please try "make dist".

Another problem I want to highlight is with how we handle tier1. The tier1.m4 contains AC_CONFIG_FILES with included Makefiles under premium/src/share. If tier1 is not built the tier1.m4 is not run, but the Makefile.am under premium/src still contains "share" subdirectory. This causes an error looking for a Makefile in "share"subdirectory. The "share" subdirectory should be conditionally included in this Makefile.am.

comment:3 Changed 21 months ago by marcin

I tried to force create the tarball that includes only tier2 hooks. I commented out "share" subdir in the premium/src/Makefile.am and created a tarball. While building the tarball I got the following error:

MacBook-Pro-Marcin:kea-1.3.0-git marcin$ build_kea_clang
configure.ac:101: error: required file 'premium/src/hooks/dhcp/flex_id/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/flex_id/tests/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/host_cmds/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/host_cmds/tests/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/legal_log/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/legal_log/libloadtests/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/hooks/dhcp/legal_log/tests/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/database/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/database/scripts/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/database/scripts/cql/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/database/scripts/mysql/Makefile.in' not found
configure.ac:101: error: required file 'premium/src/share/database/scripts/pgsql/Makefile.in' not found
autoreconf: automake failed with exit status: 1
make: *** No targets specified and no makefile found.  Stop.

being a result of the following lines in the configure.ac

AC_DEFUN([AX_PREMIUM],[])
# m4_sinclude includes the file if it exists at autoreconf time
m4_sinclude(premium/config.m4)
m4_sinclude(premium/tier1.m4)
m4_sinclude(premium/tier2.m4)
AC_SUBST(PREMIUM_DIR)
AC_SUBST(DISTCHECK_PREMIUM_CONFIGURE_FLAG)
AX_PREMIUM

So, we try to include tier1.m4 and tier2.m4 even when tier1 and/or tier2 are not built (not included in the distribution).

It seems that our premium "dist" is currently more than broken.

In addition, I'd like to know how (given the current model implemented with #5400) I should build be building a premium tarball, without nesting it within the whole Kea?

comment:4 in reply to: ↑ 2 Changed 21 months ago by fdupont

I have a patch of these 2 problems. I am verifying it does not break something else...

comment:5 Changed 21 months ago by fdupont

about the second message: it seems there was a tier1.m4 file when it should be none. The m4_sinclude(<file>) includes a file only if it exists and does nothing if it does not. And it is applied very early so you do not have to rely on conditionals (which BTW do not work).
So either the tarball got the file when it should not or you did not run autoreconf? I'll try to reproduce this because even tier2 only not critical (distributed sets are tier1 and tier1+tier2) there is no reason it does not work.

For the last question there is no answer: to avoid the chained configures which worked very badly now it is required to explicitly nest the whole Kea...

Changed 21 months ago by fdupont

Fix for legal log problems

comment:6 follow-up: Changed 21 months ago by fdupont

Trying a ./configure --without-tier1 --with-tier2, make all, make distcheck. At least the list phase in distcheck worked well...

comment:7 in reply to: ↑ 6 Changed 21 months ago by marcin

Replying to fdupont:

Trying a ./configure --without-tier1 --with-tier2, make all, make distcheck. At least the list phase in distcheck worked well...

I repeat what I said earlier today in this ticket. It is better to try "make dist" instead/in addition to "make distcheck". I recall that configuration switches are not propagated to the make distcheck properly, which may cause the distcheck to pass but with a wrong configuration (e.g. the --with-tier1 which is a default instead of what you explicitly set in configure).

comment:8 Changed 21 months ago by fdupont

The make distcheck worked well so I am retrying with make dist.
Can you look at the fix I attached?

comment:9 Changed 21 months ago by fdupont

  • Owner set to fdupont
  • Status changed from new to accepted

I tried the make list, untar, go to the new directory, configure with same arguments, make: fine.
So I looked for the detail which explains your concern: untar, go to the new directory, autoreconf: as the tar includes premium/tier1.m4 it bugs. So now I know the problem: not only the first config must be saved into premium/config.setup but also unused premium/tierX.m4 files must not be included.
Will work on this before the IETF.

comment:10 Changed 21 months ago by fdupont

  • Add Hours to Ticket changed from 0 to 2
  • Milestone changed from Kea-proposed to Kea1.4
  • Priority changed from medium to high

Done!!! Checked with --without-tier1 --with-tier2 on:
make dist; untar; cd; ./configure <same arguments>; make
make dist; untar; cd; autoreconf; ./configure <same arguments>; make
make distcheck

Ready for review. As I am answering to Wlodek question about this I dumped it to 1.4 high.

comment:11 Changed 20 months ago by fdupont

  • Owner changed from fdupont to UnAssigned
  • Status changed from accepted to reviewing

comment:12 Changed 20 months ago by wlodekwencel

On all systems we are currently using in the lab distcheck fails.

On BSD systems two errors:

https://jenkins.isc.org/job/kea-multiconfig-distcheck/BOOST=boost,TIER1=without-tier1,TIER2=without-tier2,slaves=freebsd104-64/15/console

  • NULL value (was already fixed on the premium master branch): ../../../../../../../../premium/src/hooks/dhcp/high_availability/tests/communication_state_unittest.cc:106:43: note: in instantiation of function template specialization 'boost::function<void ()>::function<nullptr_t>' requested here

EXPECT_THROW(state_.startHeartbeat(1, NULL), BadValue?);


https://jenkins.isc.org/job/kea-multiconfig-distcheck/BOOST=boost,TIER1=without-tier1,TIER2=with-tier2,slaves=freebsd104-64/15/console

Non-BSD systems:

make[3]: Leaving directory `/home/jenkins/workspace/kea-multiconfig-distcheck/BOOST/boost/TIER1/with-tier1/TIER2/with-tier2/slaves/debian7-amd64/premium/src'
make  \
	  top_distdir="../kea-1.3.0-git" distdir="../kea-1.3.0-git/premium" \
	  dist-hook
make[3]: Entering directory `/home/jenkins/workspace/kea-multiconfig-distcheck/BOOST/boost/TIER1/with-tier1/TIER2/with-tier2/slaves/debian7-amd64/premium'
sed -i '' /include_tier1/s/no/yes/ ../kea-1.3.0-git/premium/config.setup
sed: can't read /include_tier1/s/no/yes/: No such file or directory
make[3]: *** [dist-hook] Error 2

comment:13 Changed 20 months ago by fdupont

First problem is not a premium one so it requires its own ticket.
Second problem is a premium one but not in the build system: if it is enough to put 0 in place of the C++ incorrect NULL I believe it can be fixed by jabber.
The last problem is premium build so is in the scope of this ticket. I think I have enough about what systems show a sed incompatibility. Unfortunately my iMac with Linux VMs is at the other side of the Channel so it should have to wait if nobody takes the ticket and fixes it.

BTW thanks to have added premium checks in Jenkins!

comment:14 Changed 20 months ago by fdupont

Googling sed on linux sed -i works better than sed -i '' so if it is too hard to find an universal way to do the edit in place the last resort is to check it in configure and substitute the adapted command.

comment:15 Changed 20 months ago by fdupont

  • Owner changed from UnAssigned to fdupont
  • Status changed from reviewing to accepted

Trying with perl as inplace sed is not standard...

comment:16 Changed 20 months ago by fdupont

Seems to work... I shall put this ticket (premium trac5566 branch) under review when my Ubuntu VM will finish.

comment:17 Changed 20 months ago by fdupont

Got a problem with error on Ubuntu 14.10: ASYNC_CALLOUT_LIBRARY in src/lib/hooks/tests defined but not used: it is in a header file included in 3 unit test files but used only by one. Simplest fix is to split the file.
Not it is not related to 5566 itself but introduced by parking code merged to master...

comment:18 Changed 20 months ago by fdupont

Got the legal log crash: creating a ticket about it.

comment:19 Changed 20 months ago by fdupont

  • Owner changed from fdupont to UnAssigned
  • Status changed from accepted to reviewing

I think I fixed all the problems in the scope of this ticket. Unfortunately a few out of scope problems do not help to pass all checks.
Nevertheless I believe this ticket is ready again for review.

comment:20 Changed 20 months ago by fdupont

Run make distcheck with last trac5566 Kea and premium branches with trac5577 patch applied on my Ubuntu 17.10 64 bit VM with success at the second attempt (first crashed in CtrlAgentCommandMgrTest.failForwardToServer)!

comment:21 Changed 20 months ago by fdupont

Note since I merged 5528-5533a in Kea core the host cmds needs to be updated in premium too. I propose to include this in 5566 merge even it is not strictly a part of it.

comment:22 Changed 20 months ago by fdupont

  • Resolution set to fixed
  • Status changed from reviewing to closed

Merged. Closing.

Note: See TracTickets for help on using tickets.