parallel – OpenSourceOptions

Use Python to Download Multiple Files (or URLs) in Parallel

ByKonrad Hafen April 12, 2022August 23, 2022

Automating file downloads can save a lot of time. There are several ways for automating file downloads in Python. The easiest way to download files is using a simple Python loop to iterate through a list of URLs to download. This serial approach can work well with a few small files, but if you are…

Parallel Programming in Python with Message Passing Interface (mpi4py)

ByKonrad Hafen June 26, 2020December 28, 2020

Did you know you can write parallel Python code that will run on your laptop and on a super computer? You can, and it’s not as difficult as you might expect. If you already write code for asynchronous parallelization then you won’t even have to do much restructuring. High Performance Computing (HPC) distributes pieces of jobs across…