by Alex Becker

Using non-PyPI Package Indices

By default, Python tools like pip install packages from PyPI, the Python Package Index. However, most tools allow you to use any server that implements the same API as PyPI (usually but not always the Simple Repository API). There are two main reasons you might want to use an alternative package index:

How to configure your tools depends on whether you are using an alternative index in place of or in addition to PyPI. Mirrors can be used either way, while a private index is usually used in addition to PyPI. Note that PyDist or a self-hosted devpi instance can act as both a mirror and a private index.

Unfortunately, you need to configure each of your tools individually.

Configuring pip

There are two configuration options for pip that allow it to use alternative indices:

If you are using a mirror, you can either set it as the index to install all packages through the mirror, or you can set it as an extra index to only use it as a backup. If you are using a private mirror to compliment PyPI, it is tempting to use it as an extra index. However, if a package by the same name as one of your private packages is published on PyPI, this will cause pip to install that package instead. Thus it is safest to set the private index's URL as the index and use PyPI as an extra index.

You can configure pip in a number of different ways; which you should choose depends on what is easiest to set in your infrastructure and how broadly you want the configuration to apply.

Configuring twine

The standard tool to upload Python packages is twine. However, only some rivate python indices support uploading with twine—to my knowledge only PyDist and Gemfury do among hosted solutions. Other indices will generally offer their own python packages for deployment.

There are two ways to configure twine to upload to a repository other than PyPI:

The .pypirc file should look like:

[distutils]
index-servers =
    pydist

[pydist]
repository: <index-url>
username: <username>
password: <password>

If you include multiple index-servers in .pypirc, you can pass the name you gave the index server (pydist in the example above) to the --repository flag when uploading. The .pypirc file is convenient because you will not be prompted for a username/password.

Configuring pipenv

Pipenv is probably the most popular dependency-locking tool for Python. Some guides suggest using the PIP_INDEX_URL and PIP_EXTRA_INDEX_URL environment variables to configure the pipenv like you would pip, but this is not handled correctly and the maintainer told me not to use it.

Instead, you can use the [[source]] section at the top of your Pipfile:

[[source]]
url = '<index-url>'
verify_ssl = true
name = 'pydist'

If you want to use multiple package indices, you can include multiple [[source]] sections—when pipenv finds packages it tries them in the order they are specified, or if you declare a package like mypackage = { version="*", index="pydist" } it will try the specified index first. However, pipenv's handling of multiple indices is currently buggy.