Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#1702 closed defect (duplicate)

src/bin/stats/tests/test_utils.py can't find isc.config.cfgmgr

Reported by: jreed Owned by:
Priority: medium Milestone:
Component: Unclassified Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: Core Feature Depending on Ticket:
Estimated Difficulty: 0 Add Hours to Ticket: 0
Total Hours: 0 Internal?: no

Description

http://git.bind10.isc.org/~tester/2011/10/20111003063457-85e4dfa/make-coverage.out failed starting October 3. No notification so the coverage reports weren't updated since then.

This happens when ran using python coverage (via make coverage).

The error is:

make[5]: Entering directory `/home/tester/git-bind10/bind10/src/bin/stats/tests'
touch /home/tester/git-bind10/bind10/.coverage
rm -f .coverage
ln -s /home/tester/git-bind10/bind10/.coverage .coverage
for pytest in b10-stats_test.py b10-stats-httpd_test.py ; do \
	echo Running test: $pytest ; \
	 \
	PYTHONPATH=/home/tester/git-bind10/bind10/src/lib/python/isc/log_messages:/home/tester/git-bind10/bind10/src/lib/python:/home/tester/git-bind10/bind10/src/lib/python:/home/tester/git-bind10/bind10/src/bin/stats:/home/tester/git-bind10/bind10/src/bin/stats/tests:/home/tester/git-bind10/bind10/src/bin/msgq:/home/tester/git-bind10/bind10/src/lib/python/isc/config \
	B10_FROM_SOURCE=/home/tester/git-bind10/bind10 \
	CONFIG_TESTDATA_PATH=/home/tester/git-bind10/bind10/src/lib/config/tests/testdata \
	/opt/pkg/bin/coverage run --branch --append /home/tester/git-bind10/bind10/src/bin/stats/tests/$pytest || exit ; \
	done
Running test: b10-stats_test.py
Traceback (most recent call last):
  File "/opt/pkg/bin/coverage", line 9, in <module>
    load_entry_point('coverage==3.2', 'console_scripts', 'coverage')()
  File "/opt/pkg/lib/python3.1/site-packages/coverage/cmdline.py", line 590, in main
    status = CoverageScript().command_line(sys.argv[1:])
  File "/opt/pkg/lib/python3.1/site-packages/coverage/cmdline.py", line 468, in command_line
    self.run_python_file(args[0], args)
  File "/opt/pkg/lib/python3.1/site-packages/coverage/execfile.py", line 43, in run_python_file
    exec_function(source, filename, main_mod.__dict__)
  File "/opt/pkg/lib/python3.1/site-packages/coverage/backward.py", line 62, in exec_function
    exec(compile(source, filename, "exec"), global_map)
  File "/home/tester/git-bind10/bind10/src/bin/stats/tests/b10-stats_test.py", line 33, in <module>
    from test_utils import BaseModules, ThreadingServerManager, MyStats, SignalHandler, send_command, send_shutdown
  File "/home/tester/git-bind10/bind10/src/bin/stats/tests/test_utils.py", line 15, in <module>
    import isc.config.cfgmgr
ImportError: No module named cfgmgr
make[5]: *** [check-local] Error 1
make[5]: Leaving directory `/home/tester/git-bind10/bind10/src/bin/stats/tests'

This happened here:

git diff f9cb0d1..85e4dfa

Which was introduced here:

291.    [func]          naokikambe
       Statistics items are specified by each module's spec file.
       Stats module can read these through the config manager. Stats
       module and stats httpd report statistics data and statistics
       schema by each module via both bindctl and HTTP/XML.
       (Trac #928,#929,#930,#1175, git 054699635affd9c9ecbe7a108d880829f3ba229e)

You can also see this failure is in latest report:
http://git.bind10.isc.org/~tester/2012/02/20120221190111-18eda22/make-coverage.out

Subtickets

Change History (6)

comment:1 Changed 8 years ago by jreed

This can be repeated by doing "make check". See:

sh-3.2$ cat /home/tester/git-bind10/bind10/src/lib/python/isc/config/__init__.py
from isc.config.ccsession import *
from isc.config.config_data import *
from isc.config.module_spec import *

sh-3.2$ ls -l /home/tester/git-bind10/bind10/src/lib/python/isc/config/cfgmgr.py*
-rw-r--r-- 1 tester tester 24840 2012-02-09 23:00 /home/tester/git-bind10/bind10/src/lib/python/isc/config/cfgmgr.py
-rw-r--r-- 1 tester tester 23264 2012-02-10 00:27 /home/tester/git-bind10/bind10/src/lib/python/isc/config/cfgmgr.pyc

comment:2 Changed 8 years ago by naokikambe

The PYTHONPATH might be wrong.

What can we see if the following patch is applied?

  • src/bin/stats/tests/Makefile.am

    diff --git a/src/bin/stats/tests/Makefile.am b/src/bin/stats/tests/Makefile.am
    index 01254d4..87d0d5c 100644
    a b endif 
    2424       B10_FROM_SOURCE=$(abs_top_srcdir) \
    2525       BIND10_MSGQ_SOCKET_FILE=$(abs_top_builddir)/msgq_socket \
    2626       CONFIG_TESTDATA_PATH=$(abs_top_srcdir)/src/lib/config/tests/testdata \
    27        $(PYCOVERAGE_RUN) $(abs_srcdir)/$$pytest || exit ; \
     27       $(PYTHON) -v -c 'import sys;print(sys.path);import isc.config.cfgmgr' || exit ; \
    2828       done
    2929
    3030CLEANDIRS = __pycache__

Otherwise, can the following patch fix it?

  • src/bin/stats/tests/Makefile.am

    diff --git a/src/bin/stats/tests/Makefile.am b/src/bin/stats/tests/Makefile.am
    index 01254d4..17f9236 100644
    a b endif 
    2020       for pytest in $(PYTESTS) ; do \
    2121       echo Running test: $$pytest ; \
    2222       $(LIBRARY_PATH_PLACEHOLDER) \
    23        PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/stats:$(abs_top_builddir)/src/bin/stats/tests:$(abs_top_builddir)/src/bin/msgq:$(abs_top_builddir)/src/lib/python/isc/config \
     23       PYTHONPATH=$(COMMON_PYTHON_PATH):$(abs_top_builddir)/src/bin/stats:$(abs_top_builddir)/src/bin/stats/tests:$(abs_top_builddir)/src/bin/msgq:$(abs_top_builddir)/src/lib/python \
    2424       B10_FROM_SOURCE=$(abs_top_srcdir) \
    2525       BIND10_MSGQ_SOCKET_FILE=$(abs_top_builddir)/msgq_socket \
    2626       CONFIG_TESTDATA_PATH=$(abs_top_srcdir)/src/lib/config/tests/testdata \

comment:3 Changed 8 years ago by jreed

Running test: b10-stats_test.py
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
import encodings # directory /opt/pkg/lib/python3.1/encodings
# /opt/pkg/lib/python3.1/encodings/__init__.pyc matches /opt/pkg/lib/python3.1/encodings/__init__.py
import encodings # precompiled from /opt/pkg/lib/python3.1/encodings/__init__.pyc
# /opt/pkg/lib/python3.1/codecs.pyc matches /opt/pkg/lib/python3.1/codecs.py
import codecs # precompiled from /opt/pkg/lib/python3.1/codecs.pyc
import _codecs # builtin
# /opt/pkg/lib/python3.1/encodings/aliases.pyc matches /opt/pkg/lib/python3.1/encodings/aliases.py
import encodings.aliases # precompiled from /opt/pkg/lib/python3.1/encodings/aliases.pyc
# /opt/pkg/lib/python3.1/encodings/utf_8.pyc matches /opt/pkg/lib/python3.1/encodings/utf_8.py
import encodings.utf_8 # precompiled from /opt/pkg/lib/python3.1/encodings/utf_8.pyc
# /opt/pkg/lib/python3.1/encodings/latin_1.pyc matches /opt/pkg/lib/python3.1/encodings/latin_1.py
import encodings.latin_1 # precompiled from /opt/pkg/lib/python3.1/encodings/latin_1.pyc
# /opt/pkg/lib/python3.1/io.pyc matches /opt/pkg/lib/python3.1/io.py
import io # precompiled from /opt/pkg/lib/python3.1/io.pyc
import _io # builtin
# /opt/pkg/lib/python3.1/os.pyc matches /opt/pkg/lib/python3.1/os.py
import os # precompiled from /opt/pkg/lib/python3.1/os.pyc
import errno # builtin
import posix # builtin
# /opt/pkg/lib/python3.1/posixpath.pyc matches /opt/pkg/lib/python3.1/posixpath.py
import posixpath # precompiled from /opt/pkg/lib/python3.1/posixpath.pyc
# /opt/pkg/lib/python3.1/stat.pyc matches /opt/pkg/lib/python3.1/stat.py
import stat # precompiled from /opt/pkg/lib/python3.1/stat.pyc
# /opt/pkg/lib/python3.1/genericpath.pyc matches /opt/pkg/lib/python3.1/genericpath.py
import genericpath # precompiled from /opt/pkg/lib/python3.1/genericpath.pyc
# /opt/pkg/lib/python3.1/_abcoll.pyc matches /opt/pkg/lib/python3.1/_abcoll.py
import _abcoll # precompiled from /opt/pkg/lib/python3.1/_abcoll.pyc
# /opt/pkg/lib/python3.1/abc.pyc matches /opt/pkg/lib/python3.1/abc.py
import abc # precompiled from /opt/pkg/lib/python3.1/abc.pyc
# /opt/pkg/lib/python3.1/_weakrefset.pyc matches /opt/pkg/lib/python3.1/_weakrefset.py
import _weakrefset # precompiled from /opt/pkg/lib/python3.1/_weakrefset.pyc
import _weakref # builtin
# /opt/pkg/lib/python3.1/copyreg.pyc matches /opt/pkg/lib/python3.1/copyreg.py
import copyreg # precompiled from /opt/pkg/lib/python3.1/copyreg.pyc
# /opt/pkg/lib/python3.1/site.pyc matches /opt/pkg/lib/python3.1/site.py
import site # precompiled from /opt/pkg/lib/python3.1/site.pyc
# /opt/pkg/lib/python3.1/locale.pyc matches /opt/pkg/lib/python3.1/locale.py
import locale # precompiled from /opt/pkg/lib/python3.1/locale.pyc
# /opt/pkg/lib/python3.1/re.pyc matches /opt/pkg/lib/python3.1/re.py
import re # precompiled from /opt/pkg/lib/python3.1/re.pyc
# /opt/pkg/lib/python3.1/sre_compile.pyc matches /opt/pkg/lib/python3.1/sre_compile.py
import sre_compile # precompiled from /opt/pkg/lib/python3.1/sre_compile.pyc
import _sre # builtin
# /opt/pkg/lib/python3.1/sre_parse.pyc matches /opt/pkg/lib/python3.1/sre_parse.py
import sre_parse # precompiled from /opt/pkg/lib/python3.1/sre_parse.pyc
# /opt/pkg/lib/python3.1/sre_constants.pyc matches /opt/pkg/lib/python3.1/sre_constants.py
import sre_constants # precompiled from /opt/pkg/lib/python3.1/sre_constants.pyc
# /opt/pkg/lib/python3.1/collections.pyc matches /opt/pkg/lib/python3.1/collections.py
import collections # precompiled from /opt/pkg/lib/python3.1/collections.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/_collections.so", 2);
import _collections # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/_collections.so
dlopen("/opt/pkg/lib/python3.1/lib-dynload/operator.so", 2);
import operator # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/operator.so
# /opt/pkg/lib/python3.1/keyword.pyc matches /opt/pkg/lib/python3.1/keyword.py
import keyword # precompiled from /opt/pkg/lib/python3.1/keyword.pyc
# /opt/pkg/lib/python3.1/heapq.pyc matches /opt/pkg/lib/python3.1/heapq.py
import heapq # precompiled from /opt/pkg/lib/python3.1/heapq.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/itertools.so", 2);
import itertools # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/itertools.so
# /opt/pkg/lib/python3.1/bisect.pyc matches /opt/pkg/lib/python3.1/bisect.py
import bisect # precompiled from /opt/pkg/lib/python3.1/bisect.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/_bisect.so", 2);
import _bisect # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/_bisect.so
dlopen("/opt/pkg/lib/python3.1/lib-dynload/_heapq.so", 2);
import _heapq # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/_heapq.so
# /opt/pkg/lib/python3.1/weakref.pyc matches /opt/pkg/lib/python3.1/weakref.py
import weakref # precompiled from /opt/pkg/lib/python3.1/weakref.pyc
# /opt/pkg/lib/python3.1/functools.pyc matches /opt/pkg/lib/python3.1/functools.py
import functools # precompiled from /opt/pkg/lib/python3.1/functools.pyc
import _functools # builtin
import _locale # builtin
Python 3.1.1 (r311:74480, Jan 27 2010, 18:13:57) 
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
['', '/opt/pkg/lib/python3.1/site-packages/distribute-0.6.10-py3.1.egg', '/opt/pkg/lib/python3.1/site-packages/setproctitle-1.1-py3.1-linux-i686.egg', '/home/tester/git-bind10/bind10/src/lib/python/isc/log_messages', '/home/tester/git-bind10/bind10/src/lib/python', '/home/tester/git-bind10/bind10/src/bin/stats', '/home/tester/git-bind10/bind10/src/bin/stats/tests', '/home/tester/git-bind10/bind10/src/bin/msgq', '/home/tester/git-bind10/bind10/src/lib/python/isc/config', '/opt/pkg/lib/python31.zip', '/opt/pkg/lib/python3.1', '/opt/pkg/lib/python3.1/plat-linux2', '/opt/pkg/lib/python3.1/lib-dynload', '/opt/pkg/lib/python3.1/site-packages']
import isc # directory isc
import isc # precompiled from isc/__init__.pyc
import isc.config # directory isc/config
import isc.config # precompiled from isc/config/__init__.pyc
import isc.config.ccsession # precompiled from isc/config/ccsession.pyc
import json # directory /opt/pkg/lib/python3.1/json
# /opt/pkg/lib/python3.1/json/__init__.pyc matches /opt/pkg/lib/python3.1/json/__init__.py
import json # precompiled from /opt/pkg/lib/python3.1/json/__init__.pyc
# /opt/pkg/lib/python3.1/json/decoder.pyc matches /opt/pkg/lib/python3.1/json/decoder.py
import json.decoder # precompiled from /opt/pkg/lib/python3.1/json/decoder.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/binascii.so", 2);
import binascii # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/binascii.so
# /opt/pkg/lib/python3.1/struct.pyc matches /opt/pkg/lib/python3.1/struct.py
import struct # precompiled from /opt/pkg/lib/python3.1/struct.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/_struct.so", 2);
import _struct # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/_struct.so
# /opt/pkg/lib/python3.1/json/scanner.pyc matches /opt/pkg/lib/python3.1/json/scanner.py
import json.scanner # precompiled from /opt/pkg/lib/python3.1/json/scanner.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/_json.so", 2);
import _json # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/_json.so
# /opt/pkg/lib/python3.1/json/encoder.pyc matches /opt/pkg/lib/python3.1/json/encoder.py
import json.encoder # precompiled from /opt/pkg/lib/python3.1/json/encoder.pyc
dlopen("/opt/pkg/lib/python3.1/lib-dynload/time.so", 2);
import time # dynamically loaded from /opt/pkg/lib/python3.1/lib-dynload/time.so
import isc.cc # directory isc/cc
import isc.cc # precompiled from isc/cc/__init__.pyc
import isc.cc.session # precompiled from isc/cc/session.pyc
import fake_socket # precompiled from fake_socket.pyc
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named cfgmgr
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] site
# cleanup[1] io
# cleanup[1] abc
# cleanup[1] sre_constants
# cleanup[1] keyword
# cleanup[1] weakref
# cleanup[1] _weakrefset
# cleanup[1] isc
# cleanup[1] isc.config
# cleanup[1] isc.cc
# cleanup[1] isc.cc.session
# cleanup[1] fake_socket
# cleanup[1] isc.config.ccsession
# cleanup[1] json
# cleanup[1] json.scanner
# cleanup[1] json.encoder
# cleanup[1] json.decoder
# cleanup[1] struct
# cleanup[2] heapq
# cleanup[2] sre_compile
# cleanup[2] _collections
# cleanup[2] locale
# cleanup[2] _sre
# cleanup[2] functools
# cleanup[2] operator
# cleanup[2] encodings
# cleanup[2] copyreg
# cleanup[2] _heapq
...

comment:4 Changed 8 years ago by jreed

The patch doesn't appear to fix it. Also the COMMON_PYTHON_PATH already had that path in it.

comment:5 Changed 8 years ago by jreed

  • Resolution set to duplicate
  • Status changed from new to closed

This is same problem as http://bind10.isc.org/ticket/1243

I am closing this.

My workaround was:

  mv isc isc-MOVED
  mv http http-MOVED

I don't know where that http directory got there. I see on my personal system I also moved: src/bin/stats/tests/http/init.pyc-MOVED due to same problem. Sorry I forgot about this. I think the lesson is if we have issues like this we should have the build detect it and complain as temporary hacks such as removed in #1392 today.

comment:6 Changed 8 years ago by shane

  • Milestone New Tasks deleted
Note: See TracTickets for help on using tickets.