File System Interfaces

NuttX File System Overview

Overview. NuttX includes an optional, scalable file system. This file-system may be omitted altogether; NuttX does not depend on the presence of any file system.

Pseudo Root File System. A simple in-memory, pseudo file system can be enabled by default. This is an in-memory file system because it does not require any storage medium or block driver support. Rather, file system contents are generated on-the-fly as referenced via standard file system operations (open, close, read, write, etc.). In this sense, the file system is pseudo file system (in the same sense that the Linux /proc file system is also referred to as a pseudo file system).

Any user supplied data or logic can be accessed via the pseudo-file system. Built in support is provided for character and block driver nodes in the any pseudo file system directory. (By convention, however, all driver nodes should be in the /dev pseudo file system directory).

Mounted File Systems The simple in-memory file system can be extended my mounting block devices that provide access to true file systems backed up via some mass storage device. NuttX supports the standard mount() command that allows a block driver to be bound to a mount-point within the pseudo file system and to a a file system. At present, NuttX supports only the VFAT file system.

Comparison to Linux From a programming perspective, the NuttX file system appears very similar to a Linux file system. However, there is a fundamental difference: The NuttX root file system is a pseudo file system and true file systems may be mounted in the pseudo file system. In the typical Linux installation by comparison, the Linux root file system is a true file system and pseudo file systems may be mounted in the true, root file system. The approach selected by NuttX is intended to support greater scalability from the very tiny platform to the moderate platform.

File System Interfaces. The NuttX file system simply supports a set of standard, file system APIs (open(), close(), read(), write, etc.) and a registration mechanism that allows devices drivers to a associated with nodes in a file-system-like name space.

Driver Operations

fcntl.h

int creat(FAR const char *path, mode_t mode);
int open(FAR const char *path, int oflag, ...);
int fcntl(int fd, int cmd, ...);

unistd.h

int close(int fd);
int dup(int fd);
int dup2(int fd1, int fd2);
off_t lseek(int fd, off_t offset, int whence);
ssize_t pread(int fd, FAR void *buf, size_t nbytes, off_t offset);
ssize_t pwrite(int fd, FAR const void *buf, size_t nbytes, off_t offset);
ssize_t read(int fd, void *buf, size_t nbytes);
ssize_t write(int fd, FAR const void *buf, size_t nbytes);

sys/ioctl.h

int ioctl(int fd, int req, ...)

poll.h

int poll(FAR struct pollfd *fds, nfds_t nfds, int timeout)

Waits for one of a set of file descriptors to become ready to perform I/O. If none of the events requested (and no error) has occurred for any of the file descriptors, then poll() blocks until one of the events occurs.

Configuration Settings. In order to use the select with TCP/IP sockets test, you must have the following things selected in your NuttX configuration file:

  • CONFIG_NET Defined for general network support

  • CONFIG_NET_TCP Defined for TCP/IP support

In order to for select to work with incoming connections, you must also select:

  • CONFIG_NET_TCPBACKLOG Incoming connections pend in a backlog until accept() is called. The size of the backlog is selected when listen() is called.

Parameters:

  • fds – List of structures describing file descriptors to be monitored.

  • nfds – The number of entries in the list.

  • timeout – Specifies an upper limit on the time for which poll() will block in milliseconds. A negative value of timeout means an infinite timeout.

Returns:

On success, the number of structures that have nonzero revents fields. A value of 0 indicates that the call timed out and no file descriptors were ready. On error, -1 is returned, and errno is set appropriately:

  • EBADF. An invalid file descriptor was given in one of the sets.

  • EFAULT. The fds address is invalid

  • EINTR. A signal occurred before any requested event.

  • EINVAL. The nfds value exceeds a system limit.

  • ENOMEM. There was no space to allocate internal data structures.

  • ENOSYS. One or more of the drivers supporting the file descriptor does not support the poll method.

sys/select.h

int select(int nfds, FAR fd_set *readfds, FAR fd_set *writefds, FAR fd_set *exceptfds, FAR struct timeval *timeout)

Allows a program to monitor multiple file descriptors, waiting until one or more of the file descriptors become “ready” for some class of I/O operation (e.g., input possible). A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g., read(2)) without blocking.

NOTE: `poll() <#poll>`__ is the fundamental API for performing such monitoring operation under NuttX. select() is provided for compatibility and is simply a layer of added logic on top of poll(). As such, select() is more wasteful of resources and `poll() <#poll>`__ is the recommended API to be used.

Parameters:

  • nfds – the maximum file descriptor number (+1) of any descriptor in any of the three sets.

  • readfds – the set of descriptions to monitor for read-ready events

  • writefds – the set of descriptions to monitor for write-ready events

  • exceptfds – the set of descriptions to monitor for error events

  • timeout – Return at this time if none of these events of interest occur.

Returns:

  • 0: Timer expired

  • >0: The number of bits set in the three sets of descriptors

  • -1: An error occurred (errno will be set appropriately, see `poll() <#poll>`__).

Directory Operations (dirent.h)

int closedir(DIR *dirp);
FAR DIR *opendir(const char *path);
FAR struct dirent *readdir(FAR DIR *dirp);
int readdir_r(FAR DIR *dirp, FAR struct dirent *entry, FAR struct dirent **result);
void rewinddir(FAR DIR *dirp);
void seekdir(FAR DIR *dirp, int loc);
int telldir(FAR DIR *dirp);

UNIX Standard Operations (unistd.h)

#include <unistd.h>

/* Task Control Interfaces */

pid_t   vfork(void);
pid_t   getpid(void);
void    _exit(int status) noreturn_function;
unsigned int sleep(unsigned int seconds);
void    usleep(unsigned long usec);
int     pause(void);

/* File descriptor operations */

int     close(int fd);
int     dup(int fd);
int     dup2(int fd1, int fd2);
int     fsync(int fd);
off_t   lseek(int fd, off_t offset, int whence);
ssize_t read(int fd, FAR void *buf, size_t nbytes);
ssize_t write(int fd, FAR const void *buf, size_t nbytes);
ssize_t pread(int fd, FAR void *buf, size_t nbytes, off_t offset);
ssize_t pwrite(int fd, FAR const void *buf, size_t nbytes, off_t offset);

/* Check if a file descriptor corresponds to a terminal I/O file */

int     isatty(int fd);

/* Memory management */

#if defined(CONFIG_ARCH_ADDRENV) && defined(CONFIG_MM_PGALLOC) && \
    defined(CONFIG_ARCH_USE_MMU)
FAR void *sbrk(intptr_t incr);
#endif

/* Special devices */

int     pipe(int fd[2]);

/* Working directory operations */

int     chdir(FAR const char *path);
FAR char *getcwd(FAR char *buf, size_t size);

/* File path operations */

int     access(FAR const char *path, int amode);
int     rmdir(FAR const char *pathname);
int     unlink(FAR const char *pathname);

#ifdef CONFIG_PSEUDOFS_SOFTLINKS
int     link(FAR const char *path1, FAR const char *path2);
ssize_t readlink(FAR const char *path, FAR char *buf, size_t bufsize);
#endif

/* Execution of programs from files */

#ifdef CONFIG_LIBC_EXECFUNCS
int     execl(FAR const char *path, ...);
int     execv(FAR const char *path, FAR char * const argv[]);
#endif

/* Networking */

#ifdef CONFIG_NET
int     gethostname(FAR char *name, size_t size);
int     sethostname(FAR const char *name, size_t size);
#endif

/* Other */

int     getopt(int argc, FAR char * const argv[], FAR const char *optstring);

Standard I/O

#include <stdio.h>

/* Operations on streams (FILE) */

void   clearerr(FAR FILE *stream);
int    fclose(FAR FILE *stream);
int    fflush(FAR FILE *stream);
int    feof(FAR FILE *stream);
int    ferror(FAR FILE *stream);
int    fileno(FAR FILE *stream);
int    fgetc(FAR FILE *stream);
int    fgetpos(FAR FILE *stream, FAR fpos_t *pos);
FAR char *fgets(FAR char *s, int n, FAR FILE *stream);
void   flockfile(FAR FILE *stream);
FAR FILE *fopen(FAR const char *path, FAR const char *type);
int    fprintf(FAR FILE *stream, FAR const char *format, ...);
int    fputc(int c, FAR FILE *stream);
int    fputs(FAR const char *s, FAR FILE *stream);
size_t fread(FAR void *ptr, size_t size, size_t n_items, FAR FILE *stream);
FAR FILE *freopen(FAR const char *path, FAR const char *mode,
         FAR FILE *stream);
int    fseek(FAR FILE *stream, long int offset, int whence);
int    fsetpos(FAR FILE *stream, FAR fpos_t *pos);
long   ftell(FAR FILE *stream);
int    ftrylockfile(FAR FILE *stream);
void   funlockfile(FAR FILE *stream);
size_t fwrite(FAR const void *ptr, size_t size, size_t n_items, FAR FILE *stream);
FAR char *gets(FAR char *s);
FAR char *gets_s(FAR char *s, rsize_t n);
void   setbuf(FAR FILE *stream, FAR char *buf);
int    setvbuf(FAR FILE *stream, FAR char *buffer, int mode, size_t size);
int    ungetc(int c, FAR FILE *stream);

/* Operations on the stdout stream, buffers, paths, and the whole printf-family *    /

int    printf(FAR const char *format, ...);
int    puts(FAR const char *s);
int    rename(FAR const char *source, FAR const char *target);
int    sprintf(FAR char *dest, FAR const char *format, ...);
int    asprintf(FAR char **ptr, FAR const char *fmt, ...);
int    snprintf(FAR char *buf, size_t size, FAR const char *format, ...);
int    sscanf(FAR const char *buf, FAR const char *fmt, ...);
void   perror(FAR const char *s);

int    vprintf(FAR const char *s, va_list ap);
int    vfprintf(FAR FILE *stream, FAR const char *s, va_list ap);
int    vsprintf(FAR char *buf, FAR const char *s, va_list ap);
int    vasprintf(FAR char **ptr, FAR const char *fmt, va_list ap);
int    vsnprintf(FAR char *buf, size_t size, FAR const char *format, va_list ap);
int    vsscanf(FAR char *buf, FAR const char *s, va_list ap);

/* Operations on file descriptors including:
 *
 * POSIX-like File System Interfaces (fdopen), and
 * Extensions from the Open Group Technical Standard, 2006, Extended API Set
 *   Part 1 (dprintf and vdprintf)
 */

FAR FILE *fdopen(int fd, FAR const char *type);
int    dprintf(int fd, FAR const char *fmt, ...);
int    vdprintf(int fd, FAR const char *fmt, va_list ap);

/* Operations on paths */

FAR char *tmpnam(FAR char *s);
FAR char *tempnam(FAR const char *dir, FAR const char *pfx);
int       remove(FAR const char *path);

#include <sys/stat.h>

int mkdir(FAR const char *pathname, mode_t mode);
int mkfifo(FAR const char *pathname, mode_t mode);
int stat(FAR const char *path, FAR struct stat *buf);
int fstat(int fd, FAR struct stat *buf);

#include <sys/statfs.h>

int statfs(FAR const char *path, FAR struct statfs *buf);
int fstatfs(int fd, FAR struct statfs *buf);

Standard Library (stdlib.h)

Generally addresses other operating system interfaces. However, the following may also be considered as file system interfaces:

int mktemp(FAR char *template);
int mkstemp(FAR char *template);

Asynchronous I/O

#include <aio.h>

int aio_cancel(int, FAR struct aiocb *aiocbp);
int aio_error(FAR const struct aiocb *aiocbp);
int aio_fsync(int, FAR struct aiocb *aiocbp);
int aio_read(FAR struct aiocb *aiocbp);
ssize_t aio_return(FAR struct aiocb *aiocbp);
int aio_suspend(FAR const struct aiocb * const list[], int nent,
                FAR const struct timespec *timeout);
int aio_write(FAR struct aiocb *aiocbp);
int lio_listio(int mode, FAR struct aiocb * const list[], int nent,
               FAR struct sigevent *sig);

Standard String Operations

#include <string.h>

char  *strchr(const char *s, int c);
FAR char *strdup(const char *s);
const char *strerror(int);
size_t strlen(const char *);
size_t strnlen(const char *, size_t);
char  *strcat(char *, const char *);
char  *strncat(char *, const char *, size_t);
int    strcmp(const char *, const char *);
int    strncmp(const char *, const char *, size_t);
int    strcasecmp(const char *, const char *);
int    strncasecmp(const char *, const char *, size_t);
char  *strcpy(char *dest, const char *src);
char  *strncpy(char *, const char *, size_t);
char  *strpbrk(const char *, const char *);
char  *strchr(const char *, int);
char  *strrchr(const char *, int);
size_t strspn(const char *, const char *);
size_t strcspn(const char *, const char *);
char  *strstr(const char *, const char *);
char  *strtok(char *, const char *);
char  *strtok_r(char *, const char *, char **);

void  *memset(void *s, int c, size_t n);
void  *memcpy(void *dest, const void *src, size_t n);
int    memcmp(const void *s1, const void *s2, size_t n);
void  *memmove(void *dest, const void *src, size_t count);

#include <strings.h>

#define bcmp(b1,b2,len)  memcmp(b1,b2,(size_t)len)
#define bcopy(b1,b2,len) memmove(b2,b1,len)
#define bzero(s,n)       memset(s,0,n)
#define index(s,c)       strchr(s,c)
#define rindex(s,c)      strrchr(s,c)

int    ffs(int j);
int    strcasecmp(const char *, const char *);
int    strncasecmp(const char *, const char *, size_t);

Pipes and FIFOs

int pipe(int fd[2])

Creates a pair of file descriptors, pointing to a pipe inode, and places them in the array pointed to by fd.

Parameters:

  • fd – The user provided array in which to catch the pipe file descriptors. fd[0] is for reading, fd[1] is for writing.

Returns:

0 is returned on success; otherwise, -1 is returned with errno set appropriately.

int mkfifo(FAR const char *pathname, mode_t mode);

mkfifo() makes a FIFO device driver file with name pathname. Unlike Linux, a NuttX FIFO is not a special file type but simply a device driver instance. mode specifies the FIFO’s permissions (but is ignored in the current implementation).

Once the FIFO has been created by mkfifo(), any thread can open it for reading or writing, in the same way as an ordinary file. However, it must have been opened from both reading and writing before input or output can be performed. This FIFO implementation will block all attempts to open a FIFO read-only until at least one thread has opened the FIFO for writing.

If all threads that write to the FIFO have closed, subsequent calls to read() on the FIFO will return 0 (end-of-file).

Parameters:

  • pathname – The full path to the FIFO instance to attach to or to create (if not already created).

  • mode – Ignored for now

Returns:

0 is returned on success; otherwise, -1 is returned with errno set appropriately.

mmap() and eXecute In Place (XIP)

NuttX operates in a flat open address space and is focused on MCUs that do support Memory Management Units (MMUs). Therefore, NuttX generally does not require mmap() functionality and the MCUs generally cannot support true memory-mapped files.

However, memory mapping of files is the mechanism used by NXFLAT, the NuttX tiny binary format, to get files into memory in order to execute them. mmap() support is therefore required to support NXFLAT. There are two conditions where mmap() can be supported:

  1. mmap() can be used to support eXecute In Place (XIP) on random access media under the following very restrictive conditions:

    1. Any file system that maps files contiguously on the media should implement the mmap file operation. By comparison, most file system scatter files over the media in non-contiguous sectors. As of this writing, ROMFS is the only file system that meets this requirement.

    2. The underlying block driver supports the BIOC_XIPBASE ioctl command that maps the underlying media to a randomly accessible address. At present, only the RAM/ROM disk driver does this.

    Some limitations of this approach are as follows:

    1. Since no real mapping occurs, all of the file contents are “mapped” into memory.

    2. All mapped files are read-only.

    3. There are no access privileges.

  2. If CONFIG_FS_RAMMAP is defined in the configuration, then mmap() will support simulation of memory mapped files by copying files whole into RAM. These copied files have some of the properties of standard memory mapped files. There are many, many exceptions exceptions, however. Some of these include:

    1. The goal is to have a single region of memory that represents a single file and can be shared by many threads. That is, given a filename a thread should be able to open the file, get a file descriptor, and call mmap() to get a memory region. Different file descriptors opened with the same file path should get the same memory region when mapped.

      The limitation in the current design is that there is insufficient knowledge to know that these different file descriptors correspond to the same file. So, for the time being, a new memory region is created each time that rammmap() is called. Not very useful!

    2. The entire mapped portion of the file must be present in memory. Since it is assumed that the MCU does not have an MMU, on-demanding paging in of file blocks cannot be supported. Since the while mapped portion of the file must be present in memory, there are limitations in the size of files that may be memory mapped (especially on MCUs with no significant RAM resources).

    3. All mapped files are read-only. You can write to the in-memory image, but the file contents will not change.

    4. There are no access privileges.

    5. Since there are no processes in NuttX, all mmap() and munmap() operations have immediate, global effects. Under Linux, for example, munmap() would eliminate only the mapping with a process; the mappings to the same file in other processes would not be effected.

    6. Like true mapped file, the region will persist after closing the file descriptor. However, at present, these ram copied file regions are not automatically “unmapped” (i.e., freed) when a thread is terminated. This is primarily because it is not possible to know how many users of the mapped region there are and, therefore, when would be the appropriate time to free the region (other than when munmap is called).

      NOTE: Note, if the design limitation of a) were solved, then it would be easy to solve exception d) as well.

FAR void *mmap(FAR void *start, size_t length, int prot, int flags, int fd, off_t offset);

Provides minimal mmap() as needed to support eXecute In Place (XIP) operation (as described above).

Parameters:

  • start – A hint at where to map the memory – ignored. The address of the underlying media is fixed and cannot be re-mapped without MMU support.

  • length – The length of the mapping – ignored. The entire underlying media is always accessible.

  • prot –

    See the PROT_* definitions in sys/mman.h.

    • PROT_NONE - Will cause an error.

    • PROT_READ - PROT_WRITE and PROT_EXEC also assumed.

    • PROT_WRITE - PROT_READ and PROT_EXEC also assumed.

    • PROT_EXEC - PROT_READ and PROT_WRITE also assumed.

  • flags –

    See the MAP_* definitions in sys/mman.h.

    • MAP_SHARED - Required

    • MAP_PRIVATE - Will cause an error

    • MAP_FIXED - Will cause an error

    • MAP_FILE - Ignored

    • MAP_ANONYMOUS - Will cause an error

    • MAP_ANON - Will cause an error

    • MAP_GROWSDOWN - Ignored

    • MAP_DENYWRITE - Will cause an error

    • MAP_EXECUTABLE - Ignored

    • MAP_LOCKED - Ignored

    • MAP_NORESERVE - Ignored

    • MAP_POPULATE - Ignored

    • AP_NONBLOCK - Ignored

  • fd – file descriptor of the backing file – required.

  • offset – The offset into the file to map.

Returns:

On success, mmap() returns a pointer to the mapped area. On error, the value MAP_FAILED is returned, and errno is set appropriately.

  • ENOSYS - Returned if any of the unsupported mmap() features are attempted.

  • EBADF - fd is not a valid file descriptor.

  • EINVAL - Length is 0. flags contained neither MAP_PRIVATE or MAP_SHARED, or contained both of these values.

  • ENODEV - The underlying file-system of the specified file does not support memory mapping.

Fdsan

FD (file descriptor) is widely used in system software development, and almost all implementations of posix os (including nuttx) use FD as an index. the value of fd needs to be allocated starting from the minimum available value of 3, and each process has a copy, so the same fd value is very easy to reuse in the program.

In multi threaded or multi process environments without address isolation, If the ownership, global variables, and competition relationships of fd are not properly handled, there may be issues with fd duplication or accidental closure. Further leading to the following issues, which are difficult to troubleshoot.

  1. Security vulnerability: the fd we wrote is not the expected fd and will be accessed by hackers to obtain data

  2. Program exceptions or crashes: write or read fd failures, and program logic errors

  3. The structured file XML or database is damaged: the data format written to the database is not the expected format.

The implementation principle of fdsan is based on the implementation of Android https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md

uint64_t android_fdsan_create_owner_tag(enum android_fdsan_owner_type type, uint64_t tag);

Create an owner tag with the specified type and least significant 56 bits of tag.

Parameters:

  • type –

    See the ANDROID_* definitions in include/android/fdsan.h.

    • ANDROID_FDSAN_OWNER_TYPE_FILE - FILE

    • ANDROID_FDSAN_OWNER_TYPE_DIR - DIR

    • ANDROID_FDSAN_OWNER_TYPE_UNIQUE_FD - android::base::unique_fd

    • ANDROID_FDSAN_OWNER_TYPE_SQLITE - sqlite-owned file descriptors

    • ANDROID_FDSAN_OWNER_TYPE_FILEINPUTSTREAM - java.io.FileInputStream

    • ANDROID_FDSAN_OWNER_TYPE_FILEOUTPUTSTREAM - java.io.FileOutputStream

    • ANDROID_FDSAN_OWNER_TYPE_RANDOMACCESSFILE - java.io.RandomAccessFile

    • ANDROID_FDSAN_OWNER_TYPE_PARCELFILEDESCRIPTOR - android.os.ParcelFileDescriptor

    • ANDROID_FDSAN_OWNER_TYPE_ART_FDFILE - ART FdFile

    • ANDROID_FDSAN_OWNER_TYPE_DATAGRAMSOCKETIMPL - java.net.DatagramSocketImpl

    • ANDROID_FDSAN_OWNER_TYPE_SOCKETIMPL - java.net.SocketImpl

    • ANDROID_FDSAN_OWNER_TYPE_ZIPARCHIVE - libziparchive’s ZipArchive

  • tag – least significant 56 bits of tag. Typically, it is a pointer to the object/structure where fd is located

void android_fdsan_exchange_owner_tag(int fd, uint64_t expected_tag, uint64_t new_tag);

Exchange a file descriptor’s tag. Logs and aborts if the fd’s tag does not match expected_tag.

Parameters:

  • fd – The fd we want to protect

  • expected_tag – The tag corresponding to the current fd

  • new_tag – A new tag for fd binding

int android_fdsan_close_with_tag(int fd, uint64_t tag);

Close a file descriptor with a tag, and resets the tag to 0. Logs and aborts if the tag is incorrect.

Parameters:

  • fd – The fd we want to protect.

  • tag – The tag corresponding to the current fd.

Returns:

Consistent with the return value of close.