In Part 1 and Part 2 of this article, we saw what concurrency is and how it's different from parallelism. But how does concurrency help us? In one line - concurrency helps us improve the efficiency (throughput) of web servers, assuming that the app performs a lot of I/O based operations. Before digging deeper, lets look at scenarios where concurrency cannot help us.
Not Helpful
Let’s assume that our web server can handle 5,000 users at any given point of time, but our app has only 1,000 users.
Concurrency may not be very helpful here because the server has more bandwidth and efficiency is not really a concern for us.
Another scenario where concurrency can’t help is if our tasks are CPU based. For instance, let’s take the task we discussed earlier in Part 2. Instead of querying the database in step 2, lets assume that our code is doing some mathematical calculation that takes 1 second to complete.
In this is case, the work is being done by the code that is executed within the web server's thread. There is no point in introducing concurrency here because there is no idle time for the web server to reuse. Concurrency can’t help! So, when will it help?
Helpful
Let’s take example 1 again but this time, lets assume that our app has 10,000 users. There is a good chance that our web server might receive request # 5,001 when it’s already handling 5,000 requests.
The web server doesn't have any more threads to process the new request. The request has to wait for 2 + 2 seconds. Concurrency can help reduce the wait time here. Or, of course, you can simply add another server to handle the additional requests. But since writing concurrent code is relatively easy now-a-days, why not save the cost of the additional server? Maybe you can ask your boss for a raise since you have saved the cost of a web server.. :)
Let’s look at a real world example. One of the clients that I worked with had a web app that was handling millions of users. They had multiple web servers behind a load balancer to handle all the traffic.
Now the client wanted to create a mobile app. But due to various practical reasons like budget & timeline, the client wanted us to reuse the web app’s (API) service. But the mobile couldn’t directly connect to the web API because of certain constraints. So, the end architecture looked something like this:
We had mobile API servers which received requests from mobile devices, then made relevant calls to the web app’s servers. The client organization expected to have as many mobile users as there are web users. This means, we’ll need as many mobile API servers as there are web API servers. This is because, once the mobile API server receives a request, it has to wait for the web server to respond with the relevant data. It’s sitting idle while the web API servers did most of the heavy-lifting.
If we write concurrent code, that is, if we write the entire mobile API layer as asynchronous service methods, the mobile API server would be able to handle more mobile requests while the web API server does the chunk of the work.
This was a substantial cost-saving for the client.
Summary
- Concurrency can help us write efficient server-side code, when there is a need for high throughput in the web servers.
- The only pre-requisite being that the code should be I/O based rather than CPU intensive.
Comments
Post a Comment