Table of contents
- Creating a local Python package in a virtual environment
- What are Python packages and why use/create them
- How to create a local Python package
- Extra (optional but recommended): Virtual environment setup and usage
- What are virtual environments and why use/create them
- Virtual environment setup and usage
- 1. Create virtual environment
- 2. Activate created virtual environment
- 3. Create a file containing a list of items to be installed to use the package (requirements)
- Installing the package requirements (if not already installed) given requirements.txt
- Deactivating the virtual environment once done using it
- References
- Appendix
Creating a local Python package in a virtual environment
What are Python packages and why use/create them
Packages enable modularization that's particularly important as Python programs grow in size and complexity.
Modularization is enabled by structuring module namespaces using "dotted module names." This facilitates importing stuff (classes, attributes, methods, variables, and/or functions) from one module to another.
Modules are like lego bricks.
Packages are like a lego set with various lego bricks you can arrange to build something. "Dotted module names" are how a specific lego brick is referenced.
Photo by Xavi Cabrera from Unsplash
Lego brick studs and anti-studs are a way to identify lego bricks; therefore analogous to "dotted module names".
The import system can be viewed as the system that picks up lego bricks from where they're to where they'll be interlocked using their studs and anti-studs.
Photo by Kelly Sikkema from Unsplash
- Python's official documentation tutorials contains more details on modules and packages
How to create a local Python package
1. Make Python treat directories containing files as packages
Create an __init__.py in each directory of the package.
2. [optional] Import stuff into the __init__.py file
Within the __init__.py file, import stuff (classes, attributes, methods, variables, and/or functions) from modules in the same directory.
from .<file_name> import <what_importing>
Step 2 is optional if using setuptools in setup.py covered in step 3 below.
3. Setup package configuration
Create a setup.py file at the project's root directory.
To facilitate packaging use setuptool's setup and find_packages.
from setuptools import setup, find_packages
- A sample setup.py by Python Packaging Authority (pypa) can be found here
- More details on configuring Python packages can be found here
4. Install the local package
At the project's root directory, run the terminal command below to install the package in the current working directory (.) in editable mode (-e).
pip install -e .
- Pip's documentation has more details on pip install.
5. Use the local package
You can now use the local package.
- This reference contains more details on importing modules and intra-package references.
Extra (optional but recommended): Virtual environment setup and usage
What are virtual environments and why use/create them
Different Python projects may have different dependencies. Virtual environments facilitate dependency management by creating isolated environments for each project (or group of projects).
Virtual environments can be thought of as vivariums in a building.
A building can contain an aquarium and terrarium without flooding the building or placing soil on the floor. The vivariums have their own isolated environments with specific conditions for their specific needs.
Photo by Fallon Michael on Unsplash
Using a Star Trek analogy, virtual environments can also be thought of as different holoprograms in a holodeck. Tools such as venv, virtualenv, and conda that manage virtual environments can be thought of as holodecks.
The system can then be considered a spaceship/facility containing holodecks.
Virtual environment setup and usage
1. Create virtual environment
Before installing the local package, run the terminal command below to create a virtual environment named venv.
The -m interface option locates the venv module on the Python module path and executes it as a script.
python -m venv venv
2. Activate created virtual environment
Run the terminal command below to activate the environment (named venv created using the venv module)
Unix OS:
.\venv\bin\activate
Windows OS:
.\venv\Scripts\activate
Ideally creating/importing and activating a virtual environment should occur before creating scripts and modules.
3. Create a file containing a list of items to be installed to use the package (requirements)
Run the following command in the project's root directory to create a requirements.txt file listing the project's dependencies.
pip freeze > requirements.txt
Ideally, the requirements.txt file is created using the method above after the virtual environment is activated and third-party dependencies are installed in it.
Installing the package requirements (if not already installed) given requirements.txt
To install packages in the requirements.txt, run the terminal command below:
pip install -r requirements.txt
- The requirements.txt documentation gives more information about its usage.
Deactivating the virtual environment once done using it
Run the terminal command below:
deactivate
- The venv module documentation provides more details on the module here
References
[1] 6. Modules — Python 3.10.2 documentation
[2] 2. Writing the Setup Script — Python 3.10.2 documentation
[3] Packaging Python Projects — Python Packaging User Guide
[4] Packaging and distributing projects — Python Packaging User Guide
[5] Package Discovery and Namespace Package
[7] User Guide - pip documentation v22.0.3
[8] A Practical Guide to Using Setup.py
[9] pip install - pip documentation v22.0.3
[10] 1. Command line and environment — Python 3.10.2 documentation
Appendix
For more details on Python and pip in the command line, run the following terminal commands.
python -h
pip -h
Check if you have Python and pip installed by running the following terminal commands.
python -V
pip -V
For the Python terminal commands in this article, replace python with the name of your Python installation.