Can somebody explain to me why async IO is so important and why it is better than using the operating system scheduler?
If process A is blocked because of IO, then then thing that needs to be done will need to wait for the IO anyways.
Of course, in a server context, process A cannot handle new server requests while it is blocked. But luckily we can run more than one process, so process B will be free to pick it up. I will need to run a few more worker processes than there are CPU cores, but is there a problem with that?
I'm thinking now the problem is maybe that running more workers than there are cores will mean that the server accepts more concurrent connections than it can handle? If I use async code and run exacly as many workers as I have cores, the workers will never blocked. But then, I have the scenario where multiple async callbacks resolve in short sequence, but cannot be picked up by a worker because all workers are busy.
So, in both scenarios (no async but more workers than cores VS async with as many workers as cores) it can happen that the server puts too much on its plate and accepts more than it can handle.
I have a feeling that this is a fundamental problem that manifests itself differently in both paradigms, but exists notheless?
There is also a drop-in replacement for asyncio called uvloop . It claims to be faster than asyncio, gevent, node.js, etc. and comparable to golang.
> In my case, there was a ~40% speed increase compared to sequential processing. Once a code runs in parallel, the difference in speed performance between the parallel methods is very low.
But they didn't actually present the total processing time for all of the methods - I assume all of the parallel methods were about 17 seconds? (Compared to the sequential baseline of 29 seconds.) And how were the threaded frameworks configured? How many threads were they told to use (or just the default?), how many threads can they use, and what kind of parallel hardware did they run on?
This blog post presents the decision as one-dimensional; it claims all parallelization methods are the same, so the only dimension to choose on is memory efficiency. But I'm skeptical that all parallelization methods are the same, and the experimental design gives me no information on that front.
Title should refer to concurrent IO, not parallel IO.
Something seems off here - they mention using 100 workers (a worker for every request). I would expect that to perform way more than 40% faster unless there's a ton of overhead in creating those workers.
I use gevent + gunicorn, it improved by API's latency and throughput by 10x.
If you have a lot of RAM I highly recommend you look at Plasma + PyArrow
Sounds like a good job for Go :)
If you want to parallelize network I/O, use async. Otherwise, don't.
 Technically: not parallelize, but overlap.