Почему при импорте создаются файлы .pyc?

Я видел несколько ресурсов, описывающих файлы .pyc и когда они создаются. Но теперь мне интересно, почему они созданы, когда импортируются файлы .py?

Кроме того, почему бы не создать файл .pyc для основного файла Python, выполняющего импорт?

Я предполагаю, что это связано с оптимизацией производительности и обучением, это побудило меня вырвать мои файлы, так как встроенная компиляция кажется приятной в использовании. Но я не уверен, если это так, и мне также интересно, есть ли у кого-нибудь статистика относительно разницы между запуском программ с файлами .pyc и без них, если это действительно для скорости.

Я бы запускал их сам, но у меня нет хорошего, большого кода на Python, чтобы проверить его.: (

Ответ 1

Исходный код Python скомпилирован в байт-код и запускается байт-код. Файл .pyc содержит копию этого байт-кода и путем кэширования того, что Python не должен повторно компилировать код Python каждый раз, когда ему нужно загрузить модуль.

Вы можете получить представление о том, сколько времени было сохранено путем выбора функции compile():

>>> import urllib2
>>> import timeit
>>> urllib2_source = open(urllib2.__file__.rstrip('c')).read()
>>> timeit.timeit("compile(source, '', 'exec')", 'from __main__ import urllib2_source as source', number=1000)
6.977046966552734
>>> _ / 1000.0
0.006977046966552734

Таким образом, для компиляции исходного кода urllib2.py требуется 7 миллисекунд. Это не похоже на много, но это быстро складывается, поскольку Python загружает много модулей за всю свою жизнь. Просто запустите средний script с помощью -v командной строки; здесь я запускаю вывод справки для pydoc tool:

$ bin/python -v -m pydoc -h
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.py
import site # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/site.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.py
import os # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.py
import posixpath # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.py
import stat # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/stat.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.py
import genericpath # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/genericpath.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.py
import warnings # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/warnings.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.py
import linecache # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/linecache.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.py
import types # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/types.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.py
import UserDict # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/UserDict.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.py
import _abcoll # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_abcoll.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.py
import abc # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/abc.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.py
import _weakrefset # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/_weakrefset.pyc
import _weakref # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.py
import copy_reg # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/copy_reg.pyc
import encodings # directory /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.py
import encodings # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/__init__.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.py
import codecs # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/codecs.pyc
import _codecs # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/aliases.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.py
import encodings.utf_8 # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/utf_8.pyc
Python 2.7.8 (default, Sep  9 2014, 11:33:29) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.py
import runpy # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/runpy.pyc
import imp # builtin
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.py
import pkgutil # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/pkgutil.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.py
import re # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/re.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.py
import sre_compile # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_compile.pyc
import _sre # builtin
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.py
import sre_parse # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_parse.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.py
import sre_constants # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/sre_constants.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_locale.so", 2);
import _locale # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_locale.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.py
import inspect # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/inspect.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.py
import string # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/string.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/strop.so", 2);
import strop # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/strop.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.py
import dis # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/dis.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.py
import opcode # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/opcode.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.py
import tokenize # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/tokenize.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/itertools.so", 2);
import itertools # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/itertools.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.py
import token # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/token.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/operator.so", 2);
import operator # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/operator.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.py
import collections # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/collections.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_collections.so", 2);
import _collections # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_collections.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.py
import keyword # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/keyword.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.py
import heapq # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/heapq.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_heapq.so", 2);
import _heapq # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_heapq.so
import thread # builtin
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.py
import repr # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/repr.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.py
import traceback # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/traceback.pyc
# /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.pyc matches /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.py
import locale # precompiled from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/locale.pyc
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.py
import functools # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/functools.pyc
dlopen("/Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_functools.so", 2);
import _functools # dynamically loaded from /Users/mpieters/Development/venvs/stackoverflow-2.7/lib/python2.7/lib-dynload/_functools.so
# /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.pyc matches /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.py
import getopt # precompiled from /Users/mpietre/Development/Library/buildout.python/parts/opt/lib/python2.7/getopt.pyc
pydoc - the Python documentation tool

pydoc.py <name> ...
    Show text documentation on something.  <name> may be the name of a
    Python keyword, topic, function, module, or package, or a dotted
    reference to a class or function within a module or module in a
    package.  If <name> contains a '/', it is used as the path to a
    Python source file to document. If name is 'keywords', 'topics',
    or 'modules', a listing of these things is displayed.

pydoc.py -k <keyword>
    Search for a keyword in the synopsis lines of all available modules.

pydoc.py -p <port>
    Start an HTTP server on the given port on the local machine.

pydoc.py -g
    Pop up a graphical interface for finding and serving documentation.

pydoc.py -w <name> ...
    Write out the HTML documentation for a module to a file in the current
    directory.  If <name> contains a '/', it is treated as a filename; if
    it names a directory, documentation is written for all the contents.

# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] _collections
# cleanup[1] locale
# cleanup[1] functools
# cleanup[1] encodings
# cleanup[1] site
# cleanup[1] runpy
# cleanup[1] operator
# cleanup[1] supervisor
# cleanup[1] _heapq
# cleanup[1] abc
# cleanup[1] _weakrefset
# cleanup[1] sre_constants
# cleanup[1] collections
# cleanup[1] _codecs
# cleanup[1] opcode
# cleanup[1] _warnings
# cleanup[1] mpl_toolkits
# cleanup[1] inspect
# cleanup[1] encodings.utf_8
# cleanup[1] repr
# cleanup[1] codecs
# cleanup[1] getopt
# cleanup[1] pkgutil
# cleanup[1] _functools
# cleanup[1] thread
# cleanup[1] keyword
# cleanup[1] strop
# cleanup[1] signal
# cleanup[1] traceback
# cleanup[1] itertools
# cleanup[1] posix
# cleanup[1] encodings.aliases
# cleanup[1] exceptions
# cleanup[1] _weakref
# cleanup[1] token
# cleanup[1] dis
# cleanup[1] tokenize
# cleanup[1] heapq
# cleanup[1] string
# cleanup[1] imp
# cleanup[1] zipimport
# cleanup[1] re
# cleanup[1] _locale
# cleanup[1] sre_compile
# cleanup[1] _sre
# cleanup[1] sre_parse
# cleanup[2] copy_reg
# cleanup[2] posixpath
# cleanup[2] errno
# cleanup[2] _abcoll
# cleanup[2] types
# cleanup[2] genericpath
# cleanup[2] stat
# cleanup[2] warnings
# cleanup[2] UserDict
# cleanup[2] os.path
# cleanup[2] linecache
# cleanup[2] os
# cleanup sys
# cleanup __builtin__
# cleanup ints: 21 unfreed ints
# cleanup floats

Это 53 импорта:

$ bin/python -v -m pydoc -h 2>&1 | egrep ^import | wc -l
      53

вместо того, чтобы загружать (больший) исходный файл каждый раз и компилировать его, можно сразу прочитать и использовать меньший файл байт-кода. Это легко добавляет до 1/3 или 1/2 секунды, чтобы распечатать некоторую справочную информацию для инструмента командной строки.

Python не создает файл кэша для основного script; потому что это загромождает ваш каталог скриптов файлами, которые не будут загружаться почти так же часто, как загружаются модули.

Если вы запустите script, что часто время компиляции для этого одного файла влияет на вас, вы всегда можете либо переместить большую часть кода в модуль (и избегать компиляции большого script), либо вы может использовать инструмент compileall для создания файла кэша .pyc для script, а затем запустить этот .pyc файл напрямую. Обратите внимание, что Python не будет перекомпилировать этот файл, если вы изменили script!