This worked well for me because I managed to keep the computation over output ratio quite high - so scalability would be good for well over 200 CPUs. In the meantime, other players wait for the writer to finish. I gather the whole thing on one CPU and do the serial output. What I am doing currently is not quite optimal, but works for now. Parallel input is rather simple - what I do is each CPU opens the input NetCDF file and reads the part of the array that belongs to its tile (there might be a limit to how many readers can open the same NetCDF file but I am not sure). I assume your scalability issue is related to output, and not to input. But if that's a feature you don't need, it's faster to get started in NetCDF4.Īt our centre we have an afternoon-long and a day-long class on parallel I/O where we talk about the basic concepts, MPI-IO, HDF5, and NetCDF4 the slides can be found here. HDF5 is more complicated, and in exchange for that complexity allows very complicated data models. Note that if your data is relatively simple - eg, the big data structures are mainly n-dimensional arrays or vectors - I recommend NetCDF4 (which is also parallel, and based on HDF5) rather than HDF5 it's siginificantly simpler to use. MPI-IO is very low-level it's worth understanding something about it to know what's going on "under the hood" with parallel HDF5, NetCDF4, or ADIOS, but using it yourself is really only well suited for raw binary data where the structure is well known at compile time. Still, it could help to a point, especially since it sounds like the bottleneck is reading rather than writing, and it will be a big improvement if your system gets upgraded to a fully parallel filesystem. Parallel I/O can help you in this case, but if you're using (inherently pretty serial) NFS to serve your files, then it's not going to have the full effect you might want - there's going to be a serial bottleneck at the fileserver and having hundreds of processes making requests of the single server is not going to give you factors of hundreds speedup of doing it through a single process.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |