File#

Overview#

Parallel file operations are implemented via the mpl::file class. It offers various read and write modalities (collective and non-collective, blocking and non-blocking etc.) by closely following the MPI standard. See the MPI standard for a detailed description of the semantics of the various i/o operations.

Class documentation#

File class#

class file#

Class implementing parallel file i/o.

Public Types

enum class access_mode : int#

file access mode

Values:

enumerator read_only#: read-only file access

enumerator read_write#: read and write file access

enumerator write_only#: write-only file access

enumerator create#: create file it it does not exist

enumerator no_replace#: raises an error when file to create already exists

enumerator delete_on_close#: delete file when closed

enumerator unique_open#: file not opened concurrently

enumerator sequential#: file will be accessed sequentially

enumerator append#: set initial file position to end of file

enum class whence_mode : int#

file pointer positioning mode

Values:

enumerator set#: pointer positioning relative to the file’s beginning (absolute pointer positioning)

enumerator current#: pointer positioning relative to current position

enumerator end#: pointer positioning relative to the file’s end

Public Functions

file() = default#: default constructor

inline explicit file(const communicator &comm, const char *name, access_mode mode, const info &i = info{})#

constructs and opens a file

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

inline explicit file(const communicator &comm, const std::string &name, access_mode mode, const info &i = info{})#

constructs and opens a file

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

inline explicit file(const communicator &comm, const std::filesystem::path &name, access_mode mode, const info &i = info{})#

constructs and opens a file

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

file(const file&) = delete#: deleted copy constructor

inline file(file &&other) noexcept#

move constructor

Parameters:: other – file to move from

inline ~file()#: destructor

file &operator=(const file&) = delete#: deleted copy-assignment operator

inline file &operator=(file &&other) noexcept#

move-assignment operator

Parameters:: other – file to move from

inline void open(const communicator &comm, const char *name, access_mode mode, const info &i = info{})#

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

inline void open(const communicator &comm, const std::string &name, access_mode mode, const info &i = info{})#

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

inline void open(const communicator &comm, const std::filesystem::path &name, access_mode mode, const info &i = info{})#

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

comm – communicator
name – file name
mode – file open-mode
i – hints

inline void close()#: close a file

inline void resize(ssize_t size)#

resize file (shrink or grow as required)

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:: size – file size in bytes

inline void preallocate(ssize_t size)#

resize file (grow as required)

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:: size – file size in bytes

inline ssize_t size() const#

get file size

Returns:: file size in bytes

inline access_mode mode() const#

get file open-mode

Returns:: file open-mode

inline void sync()#: flush write buffers and write pending data to device

Note

This is a collective operation and must be called by all processes in the communicator.

template<typename T> inline void set_view(const char *representation, ssize_t displacement = 0, const info &i = info{})#

set the process’s file view

Template Parameters:

T – elementary read/write data type

Parameters:

representation – data representation, e.g., “native”, “internal” or “external32”
displacement – beginning of the view in bytes from the beginning of the file
i – hints

Returns:

status of performed i/o operation

template<typename T> inline void set_view(const char *representation, const layout<T> &l, ssize_t displacement = 0, const info &i = info{})#

set the process’s file view

Template Parameters:

T – elementary read/write data type

Parameters:

representation – data representation, e.g., “native”, “internal” or “external32”
l – layout defining the file view
displacement – beginning of the view in bytes from the beginning of the file
i – hints

Returns:

status of performed i/o operation

inline void seek(ssize_t offset, whence_mode whence)#

update current individual file pointer

Parameters:

offset – file pointer offset
whence – file pointer positioning mode

inline ssize_t position() const#

get current individual file pointer

Returns:: current individual file pointer

inline ssize_t byte_offset(ssize_t offset) const#

get absolute byte position in file

Parameters:: offset – file pointer offset
Returns:: absolute byte position in file that corresponds to the given view-relative offset

inline void set_info(info &i)#

set file hint

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:: i – hint

inline info get_info() const#

get file hint

Returns:: file hint

inline MPI_File native_handle() const#

Get the underlying MPI handle of the file.

Note

This function returns a non-owning handle to the underlying MPI file, which may be useful when refactoring legacy MPI applications to MPL.

Warning

This method will be removed in a future version.

Returns:: MPI handle of the file

template<typename T> inline status_t read_at(ssize_t offset, T &data)#

read data from file, blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – value to read

Returns:

status of performed i/o operation

template<typename T> inline status_t read_at(ssize_t offset, T *data, const layout<T> &l)#

read data from file, blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write_at(ssize_t offset, const T &data)#

write data to file, blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – value to write

Returns:

status of performed i/o operation

template<typename T> inline status_t write_at(ssize_t offset, const T *data, const layout<T> &l)#

write data to file, blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline irequest iread_at(ssize_t offset, T &data)#

read data from file, non-blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iread_at(ssize_t offset, T *data, const layout<T> &l)#

read data from file, non-blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_at(ssize_t offset, const T &data)#

write data to file, non-blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_at(ssize_t offset, const T *data, const layout<T> &l)#

write data to file, non-blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline status_t read(T &data)#

read data from file, blocking, non-collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read(T *data, const layout<T> &l)#

read data from file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write(const T &data)#

write data to file, blocking, non-collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write(const T *data, const layout<T> &l)#

write data to file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline irequest iread(T &data)#

read data from file, non-blocking, non-collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iread(T *data, const layout<T> &l)#

read data from file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite(const T &data)#

write data to file, non-blocking, non-collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iwrite(const T *data, const layout<T> &l)#

write data to file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline status_t read_shared(T &data)#

read data from file, blocking, non-collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_shared(T *data, const layout<T> &l)#

read data from file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write_shared(const T &data)#

write data to file, blocking, non-collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_shared(const T *data, const layout<T> &l)#

write data to file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline irequest iread_shared(T &data)#

read data from file, non-blocking, non-collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iread_shared(T *data, const layout<T> &l)#

read data from file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_shared(const T &data)#

write data to file, non-blocking, non-collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_shared(const T *data, const layout<T> &l)#

write data to file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline status_t read_at_all(ssize_t offset, T &data)#

read data from file, blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – value to read

Returns:

status of performed i/o operation

template<typename T> inline status_t read_at_all(ssize_t offset, T *data, const layout<T> &l)#

read data from file, blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write_at_all(ssize_t offset, const T &data)#

write data to file, blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – value to write

Returns:

status of performed i/o operation

template<typename T> inline status_t write_at_all(ssize_t offset, const T *data, const layout<T> &l)#

write data to file, blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline irequest iread_at_all(ssize_t offset, T &data)#

read data from file, non-blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iread_at_all(ssize_t offset, T *data, const layout<T> &l)#

read data from file, non-blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_at_all(ssize_t offset, const T &data)#

write data to file, non-blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_at_all(ssize_t offset, const T *data, const layout<T> &l)#

write data to file, non-blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline status_t read_all(T &data)#

read data from file, blocking, collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_all(T *data, const layout<T> &l)#

read data from file, blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write_all(const T &data)#

write data to file, blocking, collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_all(const T *data, const layout<T> &l)#

write data to file, blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline irequest iread_all(T &data)#

read data from file, non-blocking, collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iread_all(T *data, const layout<T> &l)#

read data from file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_all(const T &data)#

write data to file, non-blocking, collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: request representing the ongoing i/o operation

template<typename T> inline irequest iwrite_all(const T *data, const layout<T> &l)#

write data to file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T> inline status_t read_ordered(T &data)#

read data from file, blocking, collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_ordered(T *data, const layout<T> &l)#

read data from file, blocking, collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline status_t write_ordered(const T &data)#

write data to file, blocking, collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_ordered(const T *data, const layout<T> &l)#

write data to file, blocking, collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T> inline void read_at_all_begin(ssize_t offset, T &data)#

read data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – value to read

template<typename T> inline void read_at_all_begin(ssize_t offset, T *data, const layout<T> &l)#

read data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

offset – file offset in bytes
data – pointer to the data to read
l – layout used in associated i/o operation

template<typename T> inline status_t read_at_all_end(T &data)#

finish reading data from file, blocking, split-collective, explicit offset

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_at_all_end(T *data)#

finish reading data from file, blocking, split-collective, explicit offset

Template Parameters:: T – read data type
Parameters:: data – pointer to the data to read
Returns:: status of performed i/o operation

template<typename T> inline void write_at_all_begin(ssize_t offset, const T &data)#

write data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – value to write

template<typename T> inline void write_at_all_begin(ssize_t offset, const T *data, const layout<T> &l)#

write data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

offset – file offset in bytes
data – pointer to the data to write
l – layout used in associated i/o operation

template<typename T> inline status_t write_at_all_end(const T &data)#

finish writing data to file, blocking, split-collective, explicit offset

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_at_all_end(const T *data)#

finish writing data to file, blocking, split-collective, explicit offset

Template Parameters:: T – write data type
Parameters:: data – pointer to the data to write
Returns:: status of performed i/o operation

template<typename T> inline void read_all_begin(T &data)#

read data from file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read

template<typename T> inline void read_all_begin(T *data, const layout<T> &l)#

read data from file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

template<typename T> inline status_t read_all_end(T &data)#

finish reading data from file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_all_end(T *data)#

finish reading data from file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – read data type
Parameters:: data – pointer to the data to read
Returns:: status of performed i/o operation

template<typename T> inline void write_all_begin(const T &data)#

write data to file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write

template<typename T> inline void write_all_begin(const T *data, const layout<T> &l)#

write data to file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

template<typename T> inline status_t write_all_end(const T &data)#

finish writing data to file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_all_end(const T *data)#

finish writing data to file, blocking, split-collective, individual file-pointer based

Template Parameters:: T – write data type
Parameters:: data – pointer to the data to write
Returns:: status of performed i/o operation

template<typename T> inline void read_ordered_begin(T &data)#

read data from file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read

template<typename T> inline void read_ordered_begin(T *data, const layout<T> &l)#

read data from file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read
l – layout used in associated i/o operation

template<typename T> inline status_t read_ordered_end(T &data)#

finish reading data from file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – value to read
Returns:: status of performed i/o operation

template<typename T> inline status_t read_ordered_end(T *data)#

finish reading data from file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – read data type
Parameters:: data – pointer to the data to read
Returns:: status of performed i/o operation

template<typename T> inline void write_ordered_begin(const T &data)#

write data to file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write

template<typename T> inline void write_ordered_begin(const T *data, const layout<T> &l)#

write data to file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write
l – layout used in associated i/o operation

template<typename T> inline status_t write_ordered_end(const T &data)#

finish writing data to file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – value to write
Returns:: status of performed i/o operation

template<typename T> inline status_t write_ordered_end(const T *data)#

finish writing data to file, blocking, split-collective, shared file-pointer based

Template Parameters:: T – write data type
Parameters:: data – pointer to the data to write
Returns:: status of performed i/o operation

Friends

friend class group

File access mode operations#

inline file::access_mode mpl::operator|(file::access_mode mode1, file::access_mode mode2)#

bit-wise disjunction operator for file access modes

Parameters:

mode1 – 1st access mode
mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode &mpl::operator|=(file::access_mode &mode1, file::access_mode mode2)#

bit-wise disjunction assignment operator for file access modes

Parameters:

mode1 – 1st access mode
mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode mpl::operator&(file::access_mode mode1, file::access_mode mode2)#

bit-wise conjunction operator for file access modes

Parameters:

mode1 – 1st access mode
mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode &mpl::operator&=(file::access_mode &mode1, file::access_mode mode2)#

bit-wise conjunction assignment operator for file access modes

Parameters:

mode1 – 1st access mode
mode2 – 2nd access mode

Returns:

combined file access mode

Error handling#

Methods of mpl::file class may throw an exception of the type mpl::io_failure in the case of run-time i/o failures. Thus, file operations should be wrapped into a try block and possible exceptions should be caught in a matching catch clause as demonstrated in the following example:

try {
  mpl::file file;
  file.open(comm_world, "file_name.bin",
            mpl::file::access_mode::create | mpl::file::access_mode::read_write);
  // further file operations
  file.close();
} catch (mpl::io_failure &error) {
  // catch and handle i/o failures
  std::cerr << error.what() << '\n';
}