How Does Python Locate Modules And Packages

The aim of this page📝is to show how python locates modules.

• Q: So, what does python do when asked to import a module?
• A: it looks for a corresponding source file and loads that code
• Q: how?
• A: it checks the built-in sys.path variable or PYTHONPATH environmental variable
• This complements my previous Why Import Of Sibling Packages Does Not Always Work In Python — Medium

1. sys.path

• this is a list of directories
• they are searched sequentially when import is called
• the first match provides the module
• else ImportError is thrown when there is no match
• sys.path it can be long — depending on what 3rd-party packages are installed and how they were installed
• this is my sys.path from ptpython

>>> from pprint import pprint as pp 
>>> sys.path
['',
 'C:\\Users\\Admin\\AppData\\Local\\Programs\\Python\\Python38-32\\Scripts\\ptpython.exe',
 'c:\\users\\admin\\appdata\\local\\programs\\python\\python38-32\\python38.zip',
 'c:\\users\\admin\\appdata\\local\\programs\\python\\python38-32\\DLLs',
 'c:\\users\\admin\\appdata\\local\\programs\\python\\python38-32\\lib',
 'c:\\users\\admin\\appdata\\local\\programs\\python\\python38-32',
 'C:\\Users\\Admin\\AppData\\Roaming\\Python\\Python38\\site-packages',
 'C:\\Users\\Admin\\AppData\\Roaming\\Python\\Python38\\site-packages\\win32',
 'C:\\Users\\Admin\\AppData\\Roaming\\Python\\Python38\\site-packages\\win32\\lib',
 'C:\\Users\\Admin\\AppData\\Roaming\\Python\\Python38\\site-packages\\Pythonwin',
 'c:\\users\\admin\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages']

• look at the first entry — sys.path[0]

>>> sys.path[0]
''

• this means that import first looks for module in the current working directory from which the script is called
• the rest is python standard library and the site-packages folder where you can install 3rd party modules
• sometimes: the best way to make code available in python is to manually modify sys.path
• you do this by appending the path to that list

import sys
sys.path.append("<folder_with_code>")

1.1 SYS.PATH VS CWD

• when you call a script in a sub-folder such as python /foo/bar.py, the cwd is still one folder above
• → ./ NOT /foo
• however, the sys.path which is a point of orientation for the module, is the subfolder itself
• → IS ./foo — this matters greatly and can be confusing for the debugging of particular module and proper reference between them

2. PYTHONPATH

• another way to add entries to sys.path that does not require direct manipulation
• the env variable is a list of paths added to sys.path when python starts
• the format of PYTHONPATH is the same as any other path of a given platform
• win: semicolon-separated list of folders set PYTHONPATH = path1;path2;path3
• mac/linux: colon-separated list of folders export PYTHONPATH=path1:path2:path3