File

Overview

Parallel file operations are implemented via the mpl::file class. It offers various read and write modalities (collective and non-collective, blocking and non-blocking etc.) by closely following the MPI standard. See the MPI standard for a detailed description of the semantics of the various i/o operations.

Class documentation

File class

class file

Class implementing parallel file i/o.

Public Types

enum class access_mode : int

file access mode

Values:

enumerator read_only

read-only file access

enumerator read_write

read and write file access

enumerator write_only

write-only file access

enumerator create

create file if it does not exist

enumerator no_replace

raises an error when file to create already exists

enumerator delete_on_close

delete file when closed

enumerator unique_open

file not opened concurrently

enumerator sequential

file will be accessed sequentially

enumerator append

set initial file position to end of file

enum class whence_mode : int

file pointer positioning mode

Values:

enumerator set

pointer positioning relative to the file’s beginning (absolute pointer positioning)

enumerator current

pointer positioning relative to current position

enumerator end

pointer positioning relative to the file’s end

Public Functions

file() = default

default constructor

inline explicit file(const communicator &comm, const char *name, access_mode mode, const info &i = info{})

constructs and opens a file

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

inline explicit file(const communicator &comm, const std::string &name, access_mode mode, const info &i = info{})

constructs and opens a file

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

inline explicit file(const communicator &comm, const std::filesystem::path &name, access_mode mode, const info &i = info{})

constructs and opens a file

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

file(const file&) = delete

deleted copy constructor

inline file(file &&other) noexcept

move constructor

Parameters:

other – file to move from

inline ~file()

destructor

file &operator=(const file&) = delete

deleted copy-assignment operator

inline file &operator=(file &&other) noexcept

move-assignment operator

Parameters:

other – file to move from

inline void open(const communicator &comm, const char *name, access_mode mode, const info &i = info{})

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

inline void open(const communicator &comm, const std::string &name, access_mode mode, const info &i = info{})

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

inline void open(const communicator &comm, const std::filesystem::path &name, access_mode mode, const info &i = info{})

open a file

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

  • comm – communicator

  • name – file name

  • mode – file open-mode

  • i – hints

inline void close()

close a file

inline void resize(ssize_t size)

resize file (shrink or grow as required)

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

size – file size in bytes

inline void preallocate(ssize_t size)

resize file (grow as required)

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

size – file size in bytes

inline ssize_t size() const

get file size

Returns:

file size in bytes

inline access_mode mode() const

get file open-mode

Returns:

file open-mode

inline void sync()

flush write buffers and write pending data to device

Note

This is a collective operation and must be called by all processes in the communicator.

template<typename T>
inline void set_view(const char *representation, ssize_t displacement = 0, const info &i = info{})

set the process’s file view

Template Parameters:

T – elementary read/write data type

Parameters:

  • representation – data representation, e.g., “native”, “internal” or “external32”

  • displacement – beginning of the view in bytes from the beginning of the file

  • i – hints

Returns:

status of performed i/o operation

template<typename T>
inline void set_view(const char *representation, const layout<T> &l, ssize_t displacement = 0, const info &i = info{})

set the process’s file view

Template Parameters:

T – elementary read/write data type

Parameters:

  • representation – data representation, e.g., “native”, “internal” or “external32”

  • l – layout defining the file view

  • displacement – beginning of the view in bytes from the beginning of the file

  • i – hints

Returns:

status of performed i/o operation

inline void seek(ssize_t offset, whence_mode whence)

update current individual file pointer

Parameters:

  • offset – file pointer offset

  • whence – file pointer positioning mode

inline ssize_t position() const

get current individual file pointer

Returns:

current individual file pointer

inline ssize_t byte_offset(ssize_t offset) const

get absolute byte position in file

Parameters:

offset – file pointer offset

Returns:

absolute byte position in file that corresponds to the given view-relative offset

inline void set_info(info &i)

set file hint

Note

This is a collective operation and must be called by all processes in the communicator.

Parameters:

i – hint

inline info get_info() const

get file hint

Returns:

file hint

inline MPI_File native_handle() const

Get the underlying MPI handle of the file.

Note

This function returns a non-owning handle to the underlying MPI file, which may be useful when refactoring legacy MPI applications to MPL.

Returns:

MPI handle of the file

template<typename T>
inline status_t read_at(ssize_t offset, T &data)

read data from file, blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_at(ssize_t offset, T *data, const layout<T> &l)

read data from file, blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_at(ssize_t offset, const T &data)

write data to file, blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_at(ssize_t offset, const T *data, const layout<T> &l)

write data to file, blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline irequest iread_at(ssize_t offset, T &data)

read data from file, non-blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iread_at(ssize_t offset, T *data, const layout<T> &l)

read data from file, non-blocking, non-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_at(ssize_t offset, const T &data)

write data to file, non-blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_at(ssize_t offset, const T *data, const layout<T> &l)

write data to file, non-blocking, non-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline status_t read(T &data)

read data from file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read(T *data, const layout<T> &l)

read data from file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write(const T &data)

write data to file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write(const T *data, const layout<T> &l)

write data to file, blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline irequest iread(T &data)

read data from file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iread(T *data, const layout<T> &l)

read data from file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite(const T &data)

write data to file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite(const T *data, const layout<T> &l)

write data to file, non-blocking, non-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline status_t read_shared(T &data)

read data from file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_shared(T *data, const layout<T> &l)

read data from file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_shared(const T &data)

write data to file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_shared(const T *data, const layout<T> &l)

write data to file, blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline irequest iread_shared(T &data)

read data from file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iread_shared(T *data, const layout<T> &l)

read data from file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_shared(const T &data)

write data to file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_shared(const T *data, const layout<T> &l)

write data to file, non-blocking, non-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline status_t read_at_all(ssize_t offset, T &data)

read data from file, blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_at_all(ssize_t offset, T *data, const layout<T> &l)

read data from file, blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_at_all(ssize_t offset, const T &data)

write data to file, blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_at_all(ssize_t offset, const T *data, const layout<T> &l)

write data to file, blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline irequest iread_at_all(ssize_t offset, T &data)

read data from file, non-blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iread_at_all(ssize_t offset, T *data, const layout<T> &l)

read data from file, non-blocking, collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_at_all(ssize_t offset, const T &data)

write data to file, non-blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_at_all(ssize_t offset, const T *data, const layout<T> &l)

write data to file, non-blocking, collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline status_t read_all(T &data)

read data from file, blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_all(T *data, const layout<T> &l)

read data from file, blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_all(const T &data)

write data to file, blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_all(const T *data, const layout<T> &l)

write data to file, blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline irequest iread_all(T &data)

read data from file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iread_all(T *data, const layout<T> &l)

read data from file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_all(const T &data)

write data to file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

request representing the ongoing i/o operation

template<typename T>
inline irequest iwrite_all(const T *data, const layout<T> &l)

write data to file, non-blocking, collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

request representing the ongoing i/o operation

template<typename T>
inline status_t read_ordered(T &data)

read data from file, blocking, collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_ordered(T *data, const layout<T> &l)

read data from file, blocking, collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_ordered(const T &data)

write data to file, blocking, collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_ordered(const T *data, const layout<T> &l)

write data to file, blocking, collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

Returns:

status of performed i/o operation

template<typename T>
inline void read_at_all_begin(ssize_t offset, T &data)

read data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – value to read

template<typename T>
inline void read_at_all_begin(ssize_t offset, T *data, const layout<T> &l)

read data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

template<typename T>
inline status_t read_at_all_end(T &data)

finish reading data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_at_all_end(T *data)

finish reading data from file, blocking, split-collective, explicit offset

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read

Returns:

status of performed i/o operation

template<typename T>
inline void write_at_all_begin(ssize_t offset, const T &data)

write data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – value to write

template<typename T>
inline void write_at_all_begin(ssize_t offset, const T *data, const layout<T> &l)

write data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

  • offset – file offset in bytes

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

template<typename T>
inline status_t write_at_all_end(const T &data)

finish writing data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_at_all_end(const T *data)

finish writing data to file, blocking, split-collective, explicit offset

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write

Returns:

status of performed i/o operation

template<typename T>
inline void read_all_begin(T &data)

read data from file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

template<typename T>
inline void read_all_begin(T *data, const layout<T> &l)

read data from file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

template<typename T>
inline status_t read_all_end(T &data)

finish reading data from file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_all_end(T *data)

finish reading data from file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read

Returns:

status of performed i/o operation

template<typename T>
inline void write_all_begin(const T &data)

write data to file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

template<typename T>
inline void write_all_begin(const T *data, const layout<T> &l)

write data to file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

template<typename T>
inline status_t write_all_end(const T &data)

finish writing data to file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_all_end(const T *data)

finish writing data to file, blocking, split-collective, individual file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write

Returns:

status of performed i/o operation

template<typename T>
inline void read_ordered_begin(T &data)

read data from file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

template<typename T>
inline void read_ordered_begin(T *data, const layout<T> &l)

read data from file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

  • data – pointer to the data to read

  • l – layout used in associated i/o operation

template<typename T>
inline status_t read_ordered_end(T &data)

finish reading data from file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – value to read

Returns:

status of performed i/o operation

template<typename T>
inline status_t read_ordered_end(T *data)

finish reading data from file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – read data type

Parameters:

data – pointer to the data to read

Returns:

status of performed i/o operation

template<typename T>
inline void write_ordered_begin(const T &data)

write data to file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

template<typename T>
inline void write_ordered_begin(const T *data, const layout<T> &l)

write data to file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

  • data – pointer to the data to write

  • l – layout used in associated i/o operation

template<typename T>
inline status_t write_ordered_end(const T &data)

finish writing data to file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – value to write

Returns:

status of performed i/o operation

template<typename T>
inline status_t write_ordered_end(const T *data)

finish writing data to file, blocking, split-collective, shared file-pointer based

Template Parameters:

T – write data type

Parameters:

data – pointer to the data to write

Returns:

status of performed i/o operation

Friends

friend class group

File access mode operations

inline file::access_mode mpl::operator|(file::access_mode mode1, file::access_mode mode2)

bit-wise disjunction operator for file access modes

Parameters:

  • mode1 – 1st access mode

  • mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode &mpl::operator|=(file::access_mode &mode1, file::access_mode mode2)

bit-wise disjunction assignment operator for file access modes

Parameters:

  • mode1 – 1st access mode

  • mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode mpl::operator&(file::access_mode mode1, file::access_mode mode2)

bit-wise conjunction operator for file access modes

Parameters:

  • mode1 – 1st access mode

  • mode2 – 2nd access mode

Returns:

combined file access mode

inline file::access_mode &mpl::operator&=(file::access_mode &mode1, file::access_mode mode2)

bit-wise conjunction assignment operator for file access modes

Parameters:

  • mode1 – 1st access mode

  • mode2 – 2nd access mode

Returns:

combined file access mode

Error handling

Methods of mpl::file class may throw an exception of the type mpl::io_failure in the case of run-time i/o failures. Thus, file operations should be wrapped into a try block and possible exceptions should be caught in a matching catch clause as demonstrated in the following example:

try {
  mpl::file file;
  file.open(comm_world, "file_name.bin",
            mpl::file::access_mode::create | mpl::file::access_mode::read_write);
  // further file operations
  file.close();
} catch (mpl::io_failure &error) {
  // catch and handle i/o failures
  std::cerr << error.what() << '\n';
}