Parallel Concurrent Requests in Python
You’d think that the fastest way to make parallel network requests would be to use asyncio, but it’s actually
I’ve known ThreadPools before as I worked with them in Java 6+ months ago, but I couldn’t find something similar in Python until yesterday.
There is also a
ProcessPoolExecutor, but I’d only suggest you use that for CPU-bound tasks.
ThreadPoolExecutor is better since requests are network bound (and also because I encountered a jumbled error when I tried to use
Here is a quick proof of concept using
I hope that the comments in the code above were enough but if not, please let me know.