Opened 8 years ago

Closed 6 years ago

#1501 closed defect (fixed)

[kean] Make install fails with relinking of sqlite3 datasource backend

Reported by: vorner Owned by: kean
Priority: high Milestone: Sprint-20131015
Component: build system Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: DNS Feature Depending on Ticket: none
Estimated Difficulty: 5 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no

Description

I do make install as part of my „try-it-all“ script. It started to fail recently (I'm not entirely sure when it happened, I lived on older branches for some time) for me with following output:

libtool: install: warning: relinking `sqlite3_ds.la'
libtool: install: (cd /home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/datasrc; /bin/sh /home/vorner/work/bind10/bind10-devel-20111129/_build/libtool  --tag CXX --mode=relink g++ -Wall -Wextra -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Werror -fPIC -g -O2 -module -no-undefined -version-info 1:0:0 -o sqlite3_ds.la -rpath /home/vorner/work/bind10/bind10-devel-20111129/_inst/libexec/bind10-devel/backends sqlite3_accessor.lo ../../../src/lib/exceptions/libexceptions.la libdatasrc.la -lsqlite3 )
libtool: relink: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/crtbeginS.o  .libs/data_source.o .libs/static_datasrc.o .libs/sqlite3_datasrc.o .libs/query.o .libs/cache.o .libs/zonetable.o .libs/logger.o .libs/database.o .libs/factory.o .libs/datasrc_messages.o   -Wl,-rpath -Wl,/home/vorner/work/bind10/bind10-devel-20111129/_inst/lib -Wl,-rpath -Wl,/opt/log4cplus/lib -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/exceptions/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/cryptolink/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/log/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/util/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_inst/lib -ldns++ -lcryptolink -lbotan -L/opt/log4cplus/lib -lcc -llog -llog4cplus -lutil -lexceptions -lsqlite3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64/crtn.o  -O2 -pthread   -pthread -Wl,-soname -Wl,libdatasrc.so.0 -o .libs/libdatasrc.so.0.0.0
libtool: relink: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/crtbeginS.o  .libs/sqlite3_accessor.o   -Wl,-rpath -Wl,/home/vorner/work/bind10/bind10-devel-20111129/_inst/lib -Wl,-rpath -Wl,/opt/log4cplus/lib -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/exceptions/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/dns/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/cryptolink/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/cc/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/log/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_build/src/lib/util/.libs -L/home/vorner/work/bind10/bind10-devel-20111129/_inst/lib -ldatasrc -L/opt/log4cplus/lib -ldns++ -lcryptolink -lbotan -lcc -llog -llog4cplus -lutil -lexceptions -lsqlite3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../x86_64-pc-linux-gnu/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../lib64/crtn.o  -O2 -pthread   -pthread -Wl,-soname -Wl,sqlite3_ds.so.1 -o .libs/sqlite3_ds.so.1.0.0
/usr/lib/gcc/x86_64-pc-linux-gnu/4.5.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -ldatasrc
collect2: ld returned 1 exit status
libtool: install: error: relink `sqlite3_ds.la' with the above command before installing it
make[7]: *** [install-pkglibexecLTLIBRARIES] Error 1
make[7]: *** Waiting for unfinished jobs....

Subtickets

Change History (18)

comment:1 Changed 8 years ago by shane

  • Milestone changed from New Tasks to Next-Sprint-Proposed

comment:2 Changed 8 years ago by jelte

  • Estimated Difficulty changed from 0 to 5

comment:3 Changed 8 years ago by jelte

  • Milestone changed from Next-Sprint-Proposed to Sprint-20120110

comment:4 Changed 8 years ago by jelte

  • Priority changed from major to critical

comment:5 Changed 8 years ago by shane

  • Feature Depending on Ticket set to none

comment:6 Changed 8 years ago by vorner

It turns out make -j1 install solves the problem for me (I've default of -j6). So the problem might be ordering inside the same directory.

comment:7 follow-up: Changed 8 years ago by vorner

Ups, sorry, it doesn't.

But, if the directory where the instalation goes is clean or doesn't exist, it fails. The second time the installation is attempted, it works. Maybe really some ordering problems or something. Should the relinking search for something in the directory where the stuff is being installed?

comment:8 in reply to: ↑ 7 Changed 8 years ago by jinmei

Replying to vorner:

Ups, sorry, it doesn't.

But, if the directory where the instalation goes is clean or doesn't exist, it fails. The second time the installation is attempted, it works. Maybe really some ordering problems or something. Should the relinking search for something in the directory where the stuff is being installed?

FYI, this didn't happen on my local environment using FreeBSD 7.4.

comment:9 Changed 8 years ago by jelte

  • Milestone changed from Sprint-20120110 to Year 3 Task Backlog

comment:10 Changed 7 years ago by bkublik

I wonder if this is still an issue, i tried a much newer version of bind10-devel (20121115), and install flawlessly on my scientific linux 6.2.

if this is no longer the case, it may be worth to either close or abandon this ticket.

comment:11 Changed 6 years ago by shane

Michal does this still happen for you?

comment:12 Changed 6 years ago by vorner

Yes, it does.

make[5]: Nothing to be done for `install-data-am'.
 /bin/mkdir -p '/home/vorner/testing/bind10/lib64/python3.3/site-packages'
 /bin/bash ../../../../libtool   --mode=install /usr/bin/install -c   libutil_io_python.la '/home/vorner/testing/bind10/lib64/python3.3/site-packages'

libtool: install: warning: relinking `libb10-util-io.la'
libtool: install: (cd /home/vorner/work/bind10/src/lib/util/io; /bin/bash /home/vorner/work/bind10/libtool  --silent --tag CXX --mode=relink g++ -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Werror -fPIC -g -O0 -Wl,-R/usr/lib -o libb10-util-io.la -rpath /home/vorner/testing/bind10/lib fd.lo fd_share.lo socketsession.lo ../../../../src/lib/exceptions/libb10-exceptions.la -ldl )

libtool: install: warning: relinking `libutil_io_python.la'
libtool: install: (cd /home/vorner/work/bind10/src/lib/util/io; /bin/bash /home/vorner/work/bind10/libtool  --silent --tag CXX --mode=relink g++ -Wall -Wextra -Wnon-virtual-dtor -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -Werror -fPIC -g -O0 -module -avoid-version -Wl,-R/usr/lib -o libutil_io_python.la -rpath /home/vorner/testing/bind10/lib64/python3.3/site-packages libutil_io_python_la-fdshare_python.lo libb10-util-io.la -ldl )

/usr/lib/gcc/x86_64-pc-linux-gnu/4.7.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lb10-util-io
collect2: error: ld returned 1 exit status
libtool: install: error: relink `libutil_io_python.la' with the above command before installing it
make[5]: *** [install-pyexecLTLIBRARIES] Error 1
make[5]: *** Waiting for unfinished jobs....

More concrete description of what is happening:

  • When the target directory does not exist and I make install -j<a lot>, it fails.
  • When the target directory does exit and I make install -j<a lot>, it succeeds.
  • When the target directory does not exist and I make install -j, it succeeds.

My guess is, it talks about relinking, I think it tries to take the library and link it against libraries already installed. But if there are more than one library in the same directory and they link against each other, they try to relink in parallel and failing. I don't know if there's a way to force automake to do it's job properly (other than putting it to directories, but that prevents parallel compilation too, which is bad).

comment:13 Changed 6 years ago by shane

  • Milestone set to Sprint-20131015
  • Summary changed from Make install fails with relinking of sqlite3 datasource backend to [kean] Make install fails with relinking of sqlite3 datasource backend

Please see if this can be reproduced. If so, and it can be quickly fixed, then it makes sense. :)

comment:14 follow-up: Changed 6 years ago by kean

  • Status changed from new to reviewing

This is very easily reproduced and very difficult to fix. It is a fundamental problem with automake. The problem is that "make install" only has dependencies on the target it is installing, not on other components. I am not aware of any mechanism for introducing dependencies for the install rule for generated makefiles. This is further complicated by libtool which when installing does a relink to get the correct "final" SONAME entries into the shared abject. Doing parallel installs is bad for other reasons too. You can end up with corrupted archives if you have multiple sub-makes all adding members to a ranlib archive.

However, it is worth noting that using the jobserver for "make install" is highly unconventional for the above reasons, and this problem and be very easily avoided by simply not doing that. At best this warrants some documentation in a README or the building instructions.

comment:15 in reply to: ↑ 14 Changed 6 years ago by muks

  • Owner changed from UnAssigned to kean

Replying to kean:

However, it is worth noting that using the jobserver for "make install" is highly unconventional for the above reasons, and this problem and be very easily avoided by simply not doing that. At best this warrants some documentation in a README or the building instructions.

Can you add this documentation to the installation instructions in the BIND 10 guide? Also it would be nice to add a sentence about it to the message that configure prints at the end of execution.

comment:16 Changed 6 years ago by kean

  • Owner changed from kean to muks

Done.

comment:17 Changed 6 years ago by muks

  • Owner changed from muks to kean

I've committed a minor update to say "it may cause errors" instead of using definitive language.

If you are ok with this commit, please go ahead and merge.

comment:18 Changed 6 years ago by kean

  • Resolution set to fixed
  • Status changed from reviewing to closed

Merged and closed.

Note: See TracTickets for help on using tickets.