Concurrent HTTP requests in Go

For a small project I am using Go, a free and open source programming language created at Google.
I need to run multiple HTTP requests (APIs) concurrently. Go’s concurrency primitives (goroutines and channels) provide a simple and efficient means of expressing concurrent execution.
If the sentence “Concurrency is not parallelism” or “goroutines and channels” means nothing to you, I suggest you to watch this video: Rob Pike – ‘Concurrency Is Not Parallelism’ and read these resources: ‘How Goroutines Work’ and ‘concurrency in Go’.

So, assuming you are familiar with the concurrency/parallelism and goroutines/channel topics and that you have installed Go, we are going now to see ho to run concurrent HTTP requests.

I am using Go on a Windows 10 machine, go version go1.7.5 windows/amd64 and I am using Visual Studio Code IDE. If you want to use VS Code for Go (it works pretty well), check out the vscode-go extension.

I defined a struct called HTTPResponse with two fields:

  • status: the HTTP response status
  • body: the HTTP response body

We use a channel to connect concurrent goroutines. The channel we are using is typed to HTTPResponse (the previously created struct).
Given an array of HTTP urls (in the following example I am using an array of two urls) we invoke the function DoHTTPGet or DoHTTPPost in a goroutine (notice the go keyword before the call). This new goroutine will execute concurrently with the calling one.

The DoHTTPGet function makes the HTTP GET call to the given url and send the response (the struct HTTPResponse) back to the channel.

The DoHTTPPost function makes the HTTP POST call to the given url and send the response (the struct HTTPResponse) back to the channel.

The last for loop reads from the channel and prints the responses (here you should handle each response and do your stuff with it).

To benchmark the execution of the program we can use the Measure-commands{} command (if you are using Powershell) or the time command if you are using Ubuntu (or similar).

go_run_time