Simplify Request-Promise requests

Request-promise is a great library for making web requests with nodejs, though the code format can be inefficient, using the await syntax we can simplify our code when used inside of an async function.

Traditionally the documentation for request-promise is to use a format as follows:

But if you are making lots of requests while web scraping, this can get very clunky make your code very complex with curly braces all over the place and redundant code that really could be simplified. You might need to load the login page, then get a list of products from another page, and then scrape each product page, and nesting your requests using this format is not always the most effective solution as it affects code readability.

I'm surprised that their isn't any examples of this usage in the official request-promise documentation but here I will demonstrate how to simplify your request-promise requests.

The following shows how you can simplify a request made with request-promise.

Simplify Request Promise Requests

There is simply two steps to make a request and simplify your code with request-promise.

1. Build the request - specify the URL to load and any additional options you would like to pass to the request.
   
   let req = {
    method: "GET",
    uri: "https://www.google.com.au",
     port: 443,
     resolveWithFullResponse: true
};
2. Make the request - Using the following code inside of an async function means you can simplify the code into one single line of code, instead of expanding out and creating functions to handle the response.
   
   The execution of the code will not continue until the response has been received, once the response is received the resp variable hold the returned object and you can process the request from there.
   
   let resp = await rp(req);

The following example demonstrates a working example using this method to scrape company information from Yahoo Finance.