High-performance scientific computing in a modern age uses parallel techniques at a scale of hundreds of thousands of processors. These large-scale applications have I/O system workloads that are primarily driven by small, sparse I/O operations. While parallel file systems have provided application developers with scalable peak I/O bandwidth for large, contiguous I/O operations, the noncontiguous I/O access patterns common to scientific applications have remained a serious performance problem. Our work in this area has contributed several solutions toward improving the gap in I/O performance and other system components through new list I/O and datatype I/O methods. Detailed studies on basic I/O characteristics and application-level I/O strategies have led to performance-oriented suggestions for application programmers on how to best use these new I/O methods. In this paper, we also discuss our optimized DLM and versioning approaches for implementing atomic noncontiguous I/O operations that are the building blocks for handling challenging problems in redundant storage, consistent distributed data layouts for parallel processing, and other producer-consumer problems.

Last modified

• 08/31/2018

Creator

• Avery Ching

DOI

• https://doi.org/10.21985/N2243N

Subject

• Computer Science

Keyword

• Computer Science

Date created

• 2007-12-12

Resource type

• Dissertation

Rights statement

• In Copyright