- pyenv for versioning Python itself
- pyenv-virtualenv for managing virtual environments
- poetry for handling package installation and dependencies
The world of Python versioning (and the downstream package versioning) is wild. This StackOverflow thread gives you a sense of some of the core issues at play. (As an indication of the importance of the issue, even BDFL Guido van Rossum himself has the current second most upvoted answer.)
For a really vanilla and close-to-core-python setup, a combination of
pip seem to be the way to go.
venv is part of the standard library and as such is pretty close to a default option.
For something a bit more involved, that handles dependencies and package installation in a slightly more deft manner, the combination of
poetry works really well. I’ll detail some of the setup gotchas and usage patterns below.
pyenv for versioning Python itself
pyenv lets you install multiple versions of Python on the same machine. The interface to switch between local versions and whatever you’ve decided will be your
global option is pretty intuitive.
Visit the pyenv github page for more on installation. (If you’re on a Mac you can simply do a
brew install pyenv.)
To see which versions of Python you have installed locally:
To see versions of Python which are available for installation:
pyenv install —list
Note that, as I understand it, these versions are not dynamically updated. You get an updated list of new Python versions by updating
pyenv, in other words.
To install a specific version of Python, and to make it available for use:
pyenv install 3.9.1
To set that version of Python as the global version (i.e. running
python will use this version by default):
pyenv global 3.9.1
If you are in a project directory and wish to only use a particular version of Python in that directory (and its subdirectories):
pyenv local 3.8.2
This creates a
.python-version file in that directory with the desired local version.
pyenv-virtualenv for managing virtual environments
pyenv-virtualenv is a plugin that connects the work of selecting which version of Python to use (through
pyenv, which we’ve previously installed) to the work of creating and running virtual environments to keep code contained in quasi-sandbox environments. When you install packages in virtual environments they don’t conflict with other locations where you might have conflicting versions of those same packages installed.
Read installation instructions and the docs here. (If you installed
pyenv with homebrew, be sure to do the same with
To create a virtual environment for the Python version used with
pyenv virtualenv, specifying the Python version you want and the name of the virtual environment directory:
pyenv virtualenv 3.8.2 my-virtual-env-3.8.2
This will create a virtual environment based on Python 3.8.2 under
$(pyenv root)/versions in a folder called
To list what virtual environments have been created and are available to use:
As a common workflow pattern, you’d create your directory and
cd into it, and then you can set the virtual environment you just created as the one to use for that directory:
mkdir test-project && cd test-project pyenv local my-virtual-env-3.8.2
This should change the prompt in your terminal window and you’ll thus know that you’re now working out of that virtual environment. Any time you return to that folder you’ll automatically switch to that environment.
The manual way of turning on and off virtual environments is:
pyenv activate env-name pyenv deactivate env-name
To remove a virtual environment from your system:
pyenv uninstall my-virtual-env
(This is the functional equivalent of removing the directories in
$(pyenv root)/versions and
poetry for handling package installation and dependencies
python-poetry is the latest standard tool for handling package installations and dependency management.
poetry self update
poetry is one of those tools that’s able to update itself.
For basic usage for a new project, you can follow the following workflow. There are two ways to start a new project using
init. For example:
poetry new some-project-name
This will kickstart your new project by creating a bunch of files and a directory structure suitable for most projects, like so:
some-project-name ├── pyproject.toml ├── README.rst ├── some-project-name │ └── __init__.py └── tests ├── __init__.py └── test_some-project-name.py
You might want to use a
src folder (above the
some-project-name in our example) which is fairly commonly used, in which case amend the command as follows:
poetry new --src some-project-name
poetry init doesn’t do all the extra work of creating a directory and file structure. It merely creates a
pyproject.toml file interactively, using some smart defaults. For a minimal use of
poetry, this is definitely the way to go.
add command adds required packages to your
pyproject.toml and installs them (along with all their dependencies). It does a lot under the hood to make sure that dependencies are correctly resolving before installing. For example:
poetry add zenml
To add packages only to be used in the development environment:
poetry add --dev zenml
To list all installed packages in your current environment / project:
To uninstall a package and remove it (and its dependencies) from the project:
poetry remove zenml
To install all relevant packages and dependencies of a project that you’ve newly cloned into:
Note that it is possibly worth creating some custom scripts to handle some of the overhead of using these tools, depending on your common development workflows.