Modules, Packages, and The Search Path

Last updated on 2026-02-10 | Edit this page

Overview

Questions

  • How does Python know where to find the libraries you import?
  • What distinguishes a python “script” from a python “package”?
  • What is an __init__.py file?

Objectives

  • Inspect the sys.path variable to understand import resolution.
  • Differentiate between built-in modules, installed packages, and local code.
  • Create a minimal local package structure.

From Scripts to Reusable Code

You have likely written Python scripts before: single files ending in .py that perform a specific analysis or task. While scripts are excellent for execution (running a calculation once), they are often poor at facilitating reuse.

Imagine you wrote a useful function to calculate the center of mass of a molecule in analysis.py. A month later, you start a new project and need that same function. You have two options:

  1. Copy and Paste: You copy the function into your new script.
    • Problem: If you find a bug in the original function, you have to remember to fix it in every copy you made.
  2. Importing: You tell Python to load the code from the original file.

Option 2 is the foundation of Python packaging. To do this effectively, we must first understand how Python finds the code you ask for.

How Python Finds Code

When you type import numpy, Python does not magically know where that code lives. It follows a deterministic search procedure. We can see this procedure in action using the built-in sys module.

PYTHON 

import sys
from pprint import pprint

pprint(sys.path)

PYTHON 

['',
 '/usr/lib/python314.zip',
 '/usr/lib/python3.14',
 '/usr/lib/python3.14/lib-dynload',
 '/usr/lib/python3.14/site-packages']

The variable sys.path is a list of directory strings. When you import a module, Python scans these directories in order. The first match wins.

  1. The Empty String (’’): This represents the current working directory. This is why you can always import a helper.py file if it is sitting right next to your script.
  2. Standard Library: Locations like /usr/lib/python3.* contain built-ins like os, math, and pathlib.
  3. Site Packages: Directories like site-packages or dist-packages are where tools like pip, conda, or pixi place third-party libraries.
Challenge

Challenge: Shadowing the Standard Library

What happens if you create a file named math.py in your current folder with the following content:

PYTHON 

# math.py
print("This is my math!")
def sqrt(x):
    return "No square roots here."

And then run python and type import math?

Python will import your local file instead of the standard library math module.

Why? Because the current working directory (represented by '' in sys.path) is usually at the top of the list. It finds your math.py before scanning the standard library paths. This is called “Shadowing” and is a common source of bugs!

Search order for packages
Search order for packages

The Anatomy of a Package

A Module
Is simply a single file ending in .py.
A Package
Is a directory containing modules and a special file: __init__.py.

Let’s create a very simple local package to handle some basic chemistry geometry. We will call it chemlib.

BASH 

mkdir chemlib
touch chemlib/__init__.py

Now, create a module inside this directory called geometry.py:

SH 

def center_of_mass(atoms):
    print("Calculating Center of Mass...")
    return [0.0, 0.0, 0.0]

Your directory structure should look like this:

project_folder/
├── script.py
└── chemlib/
    ├── __init__.py
    └── geometry.py

The Role of __init__.py

The __init__.py file tells Python: “Treat this directory as a package.” It is the first file executed when you import the package. It can be empty, but it is often used to expose functions to the top level.

Open `chemlib/_init__.py` and add:

PYTHON 

print("Loading chemlib package...")
from .geometry import center_of_mass

Now, from the project_folder (the parent directory), launch Python:

PYTHON 

import chemlib

chemlib.center_of_mass([])
Loading chemlib package...
Calculating Center of Mass...
[0.0, 0.0, 0.0]

The “It Works on My Machine” Problem

We have created a package, but it is fragile. It relies entirely on the Current Working Directory being in sys.path.

Challenge

Challenge: Moving Directories

  1. Exit your python session.
  2. Change your directory to go one level up (outside your project folder): cd ..
  3. Start Python and try to run import chemlib.

What happens and why?

Output:

ModuleNotFoundError: No module named 'chemlib'

Reason: You moved out of the folder containing chemlib. Since the package is not installed in the global site-packages, and the current directory no longer contains it, Python’s search through sys.path fails to find it.

Directory structure for current setup
Directory structure for current setup

To solve this, we need a standard way to tell Python “This package exists, please add it to your search path permanently.” This is the job of Packaging and Installation.

Key Points
  • sys.path is the list of directories Python searches for imports.
  • The order of search is: Current Directory -> Standard Library -> Installed Packages.
  • A Package is a directory containing an __init__.py file.
  • Code that works locally because of the current directory will fail when shared unless properly installed.