Why use a requirements.txt file?
Together, we have now created two sample projects and a virtual environment for each project. In each virtual environment we installed one or two Python packages. We can view the packages that we have installed in our virtual environment using the command pip freeze while the environment is active.
What happens, however, if we want another developer to work on one of our projects? How will they know what packages to install inside their virtual environment?
In order to ensure that all developers working on a project are using the same virtual environment we use a requirements.txt file. This is essentially a list of the Python packages that are ‘required’ to be installed inside a virtual environment for the associated application to run successfully.
As an example, this would be a valid requirements.txt file:
matplotlib numpy requests
If the above requirements.txt file was stored as part of a project, it would basically say to any new user working on the project: ”When you first start work on this project, create and activate a virtual environment and then install these packages.” This allows the developer to start working on the project immediately, while being confident that they have the same local development environment as other developers working on the project.
You may have already noticed that in the example, package versions are not specified. It is better practice to specify an exact version, or a range of versions of packages, in your requirements.txt file. For example, an alternative requirements.txt file might be as follows:
matplotlib==3.2.2 numpy>1.12 requests>2.0,<3.0
This specifies an exact version to be used for matplotlib and a range of versions that can be used for numpy and requests .
There are a variety of situations which arise in software development where in your requirements.txt file you might:
Not specify package versions.
Specify exact package versions.
Specify a range of package versions for each package.
The difference between these scenarios is well beyond the scope of this course, so we'll go forward using the second option for the remainder of this chapter! 😉
Create and store a requirements file
Let’s go back to the demo-app-2 project that we worked on earlier. First of all, change directory so that you are in the appropriate folder on the command line:
~/projects/demo-app-2 →
Running ls should show that you have a file called demo.py and a virtual environment folder called env :
~/projects/demo-app-2 → ls demo.py env ~/projects/demo-app-2 →
As before, we can run our demo by activating the virtual environment and using Python to run the application:
~/projects/demo-app-2 → source env/bin/activate (env) ~/projects/demo-app-2 → python demo.py
As before, this will generate a graph as an output!
Now, we want to create and store a requirements.txt file so that we can share our project (and the specification for the virtual environment) with other developers. We have two ways of doing this.
Create the requirements.txt manually
So far, we have only installed matplotlib in our virtual environment. We can therefore create a requirements.txt file with just these two packages manually. First, check with versions of the packages you have installed by running pip freeze :
(env) ~/projects/demo-app-2 → pip freeze cycler==0.10.0 kiwisolver==1.2.0 matplotlib==3.2.2 numpy==1.19.0 pyparsing==2.4.7 python-dateutil==2.8.1 six==1.15.0
You might note at this point that there are other packages listed alongside matplotlib . These are dependencies of matplotlib and we don’t explicitly need to list these in our requirements.txt file.
Now we are ready. Create a requirements.txt file and add the following contents:
matplotlib==3.2.2
Save the file! Your project should now look as follows:
(env) ~/projects/demo-app-2 → ls demo.py env requirements.txt (env) ~/projects/demo-app-2 → cat requirements.txt matplotlib==3.2.2
Great! 😁 You’ve successfully created a requirements.txt file, and if you were to share the demo-app-2 project with another developer they would be able to create a virtual environment and run the application as required.
Create the requirements.txt automatically using pip
The alternative option for creating a requirements.txt file is to use the pip freeze command. For example, we could run:
(env) ~/projects/demo-app-2 → pip freeze > requirements.txt (env) ~/projects/demo-app-2 →
This outputs the content of pip freeze to a requirements.txt file. Let’s take a peek inside the newly created file:
(env) ~/projects/demo-app-2 → cat requirements.txt cycler==0.10.0 kiwisolver==1.2.0 matplotlib==3.2.2 numpy==1.19.0 pyparsing==2.4.7 python-dateutil==2.8.1 six==1.15.0 (env) ~/projects/demo-app-2 →
For some extra help on creating requirements files, check out this screencast:
Install packages in your virtual environment using a requirements file
Now, let’s imagine that you are inheriting a project from another developer. The project should already have a requirements.txt file which you can use to install packages in your virtual environment. Ideally, it will have some documentation too!
First of all, download this script and this requirements.txt file. Put it into a newly-created demo-app-3 project in your projects folder, change directory into it and check the contents:
~/projects → ls demo-app demo-app-2demo-app-3 ~/projects → cd demo-app-3/ ~/projects/demo-app-3 → ls demo.py requirements.txt ~/projects/demo-app-3 →
Now, let’s check the contents of both the files:
~/projects/demo-app-3
→ cat demo.py
import requests
from bs4 import BeautifulSoup
r = requests.get('http://www.example.com')
soup = BeautifulSoup(r.text, features="html.parser")
print(soup.text)
~/projects/demo-app-3
→ cat requirements.txt
beautifulsoup4==4.9.1
requests==2.24.0Note that in demo.py we make use of two packages: requests and bs4 . Both of these packages, along with their versions, are listed explicitly in requirements.txt . At this point, you could try and run demo.py using Python. If you have requests and bs4 installed globally (ideally you won’t) then the script will quite possibly run successfully. However, even if this is the case you can’t be sure that the script is running in exactly the way that the repository author intended because you might be using different versions of requests and bs4 to the versions specified in requirements.txt .
To ensure that you have an identical setup to the other developers working on the project, we use a virtual environment. First, create and activate a virtual environment:
~/projects/demo-app-3 → python -m venv env ~/projects/demo-app-3 → ls demo.py env requirements.txt ~/projects/demo-app-3 → source env/bin/activate (env) ~/projects/demo-app-3 →
Next, we need to install the Python packages listed in the requirements.txt file. We do this using the following command:
(env) ~/projects/demo-app-3 → pip install -r requirements.txt Collecting beautifulsoup4==4.9.1 (from -r requirements.txt (line1)) Using cached https://files.pythonhosted.org/packages/66/25/ff030e2437265616a1e9b25ccc864e0371a0bc3adb7c5a404fd661c6f4f6/beautifulsoup4-4.9.1-py3-none-any.whl Collecting requests==2.24.0 (from -r requirements.txt (line2)) Using cached https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl Collecting soupsieve>1.2 (from beautifulsoup4==4.9.1->-rrequirements.txt (line1)) Using cached https://files.pythonhosted.org/packages/6f/8f/457f4a5390eeae1cc3aeab89deb7724c965be841ffca6cfca9197482e470/soupsieve-2.0.1-py3-none-any.whl Collecting idna<3,>=2.5 (from requests==2.24.0->-rrequirements.txt (line2)) Using cached https://files.pythonhosted.org/packages/89/e3/afebe61c546d18fb1709a61bee788254b40e736cff7271c7de5de2dc4128/idna-2.9-py2.py3-none-any.whl Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests==2.24.0->-rrequirements.txt (line2)) Using cached https://files.pythonhosted.org/packages/e1/e5/df302e8017440f111c11cc41a6b432838672f5a70aa29227bf58149dc72f/urllib3-1.25.9-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests==2.24.0->-rrequirements.txt (line2)) Using cached https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl Collecting chardet<4,>=3.0.2 (from requests==2.24.0->-rrequirements.txt (line2)) Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl Installing collected packages: soupsieve, beautifulsoup4, idna, urllib3, certifi, chardet, requests Successfully installed beautifulsoup4-4.9.1 certifi-2020.6.20 chardet-3.0.4 idna-2.9 requests-2.24.0 soupsieve-2.0.1 urllib3-1.25.9
Now, when you run pip freeze you will see that you have successfully installed requests and bs4 , along with some of their dependencies:
(env) ~/projects/demo-app-3 → pip freeze beautifulsoup4==4.9.1 certifi==2020.6.20 chardet==3.0.4 idna==2.9 requests==2.24.0 soupsieve==2.0.1 urllib3==1.25.9
Finally, you can run the demo.py script successfully:
(env) ~/projects/demo-app-3 → python demo.py Example Domain Example Domain This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission. More information...
Great! You have successfully created a virtual environment and installed packages listed in a requirements.txt file. Kudos! 🥳
If you are looking for a more interactive demonstration of the code snippets above, check out the screencast below:
Exercise
Task 1
You work in sales and a member of your team has been working on a Python project that they are keen to share with you.
First, download the repository here: p2c5s3_task_1.zip.
Your task is to create a virtual environment, install the required packages listed in the requirements.txt file and run the application.
Watch this screencast for a solution to the exercise:
Task 2
Another member of your team has been working on a separate Python project, but they aren’t as experienced as you. Their project hasn’t got a requirements.txt file included.
Download their project here: p2c5s3_task_2.zip.
See if you can get it to run inside a virtual environment. You may need to take a look through their code to see what packages you need to install. Once you have successfully run the application, create a requirements.txt file so that someone else can work on the project if needed. Once you are finished, you can test that your requirements.txt file works by deleting your virtual environment, creating a new one, using your requirements.txt file to install Python packages and then running the application.
Watch this screencast for a solution to the exercise:
Let’s Recap!
You should always create a
requirements.txtfile when you are using a virtual environment for a project.If a project you inherit doesn’t have a
requirements.txtfile then ask the author why not - it isn’t a rude question and there may be a perfectly good answer!
Now that you know how to work with requirements files we will move on to deleting virtual environments.
