Skip to content
  1. Jun 28, 2013
  2. Jun 27, 2013
    • Aurélien Dunand's avatar
      Only remove used modules instead of clearing sys.modules · 8c4bc60b
      Aurélien Dunand authored
      As added to documentation [1], delete essential items from sys.modules may
      cause Python to fail. Import machinery is complicated [2] and not well
      documented before Python 3.3 [3] and the import system rework.
      
      In our case, at least two issues was caused by clearing sys.modules.
      - The first one impact import mechanism in our libs. If a lib import a function
      (or other) with the 'from module import function' syntax, this function will be loaded,
      but the module will not be properly loaded, this will result to TypeError.
      For example if in our lib we use:
      
      >>> from re import search
      >>> search('(?<=-)\w+', 'spam-egg')
      
      This will raise the following error:
      
        File "/usr/lib/python2.7/re.py", line 142, in search
          return _compile(pattern, flags).search(string)
      TypeError: 'NoneType' object is not callable
      
      - The second one is raised by importing a gzipped tarball in a script and store
      them in 'namespace', namespace.tar for example. If we try to extract a file
      from namespace.tar in subsequent scripts, we've got this error:
      
        File "/usr/lib/python2.7/tarfile.py", line 2112, in extractfile
          tarinfo = self.getmember(member)
        File "/usr/lib/python2.7/tarfile.py", line 1794, in getmember
          tarinfo = self._getmember(name)
        File "/usr/lib/python2.7/tarfile.py", line 2357, in _getmember
          members = self.getmembers()
        File "/usr/lib/python2.7/tarfile.py", line 1805, in getmembers
          self._load()        # all members, we first have to
        File "/usr/lib/python2.7/tarfile.py", line 2380, in _load
          tarinfo = self.next()
        File "/usr/lib/python2.7/tarfile.py", line 2319, in next
          tarinfo = self.tarinfo.fromtarfile(self)
        File "/usr/lib/python2.7/tarfile.py", line 1239, in fromtarfile
          buf = tarfile.fileobj.read(BLOCKSIZE)
        File "/usr/lib/python2.7/gzip.py", line 240, in read
          raise IOError(errno.EBADF, "read() on write-only GzipFile object")
      IOError: [Errno 9] read() on write-only GzipFile object
      
      [1] http://hg.python.org/cpython/rev/4f8160e45cb7
      [2] http://python-notes.boredomandlaziness.org/en/latest/python_concepts/import_traps.html
      [3] http://docs.python.org/dev/reference/import.html
      8c4bc60b
    • Sébastien Luttringer's avatar
      Deep refactoring · 084ed9b5
      Sébastien Luttringer authored
      - 2 new packages (image and repository).
      - Split big modules into little one under packages.
      - Replace full module import by needed objects
      - Avoid wildcard imports
      - Fix a lot of pylint error and warnings on modules
      084ed9b5