Building Query Parameters

00:00 To download files with the requests library, first you need to import requests, and then you need to define your URL from which you want to download the file.

00:09 Unlike urllib, you don’t need to add query parameters in the URL itself. Instead, you can create a dictionary that contains all your query parameters. Once you have your URL and the query parameters ready, you just call the requests.get() function and pass the URL and query parameters, which returns a response object.

00:29 You can check response.url to see the full URL that the requests library has created.

00:37 The response object also contains various other information, like status code, and if the request was successful or not. You can check the status code of the request using response.status_code. You can also check if the request was successful or not using response.ok, which returns True if the request was successful.

00:58 Now let’s go ahead and see how you can actually save the downloaded file to your local machine.

01:04 As I mentioned before, requests is an external library, so you’ll need to install it. So go ahead and activate your virtual environment. If you want to learn more about virtual environments, I’ll add the appropriate link in the additional resources section. Now to install requests, just enter this command: python -m pip install requests.

01:33 Once requests is installed, start the Python REPL.

01:38 Now the first step is to import the requests module, so just type import requests.

01:45 And then, just like in urllib, you need to define the URL from which you want to download the file.

01:53 Note that when you used urllib, you had the whole URL with the query strings, that is the part after the question mark, as a string. requests allows you to handle it cleanly.

02:04 So you have your URL, and then you can create a dictionary with all your query parameters. Let’s create a variable called query_parameters. This is a dictionary in which you can define all your parameters.

02:16 In the last example, the query parameter was "downloadformat", so let’s define "downloadformat" and set it to "csv". And once you have your query_parameters dictionary and the URL ready, you can just call the requests.get() function.

02:34 So let’s type response = requests.get(), and the first parameter is the URL, and the second parameter is a named parameter called params, and we’ll pass query_parameters as the argument.

02:52 Now, unlike urlretrieve(), which just saved the downloaded file on your local machine, requests won’t directly do that. Instead, requests gives you an object that you can inspect and later on save the file as you’d like. Now let’s see some of the things that this response object has.

03:12 First, let’s check the URL that requests has created for us. You can just enter response.url to get the full URL that was used.

03:21 Now as you can note, requests has automatically added the question mark, the equals sign, and if you had multiple query parameters, it’ll also add the ampersand symbols in the right places.

03:33 Now requests also has a variable called the status code, which you can check using response.status_code to get the status code of the response.

03:42 It also contains a variable called .ok, which you can check by just entering response.ok to check if the request was successful or not.

03:51 Till now, you’ve successfully retrieved the contents from the URL, but strictly speaking, you haven’t saved anything yet. The data is just sitting inside your response variable.

04:02 To actually make this a permanent file on your hard drive, you need to write that data to the disk. In the next lesson, you’ll see how you can save the downloaded file to your local machine.