DIO

Diagnosing applications I/O behavior
through system call observability

DIO is a generic tool for observing and diagnosing applications' storage I/O.

It is designed to be used by applications developers and users to understand how applications interact with storage systems. By combining system call tracing with a customizable data analysis and visualization pipeline, DIO provide non-intrusive and comprehensive I/O diagnosis for applications using in-kernel POSIX storage systems (e.g., ext4, linux block device).

DIO is an open-source project available at https://github.com/dsrhaslab/dio.

Features

• Generic and non-intrusive

  DIO can be used to observe I/O system calls made by any application interacting with in-kernel storage systems without requiring any modification to the application code.

• Flexible

  DIO can be configured to collect only the information that is relevant to the user. Namely, DIO allows users to filter events based on:

  • process name (command)
  • process/thread IDs
  • system call types
  • file paths
  
  

• Comprehensive

  DIO collects a wide range of information about the I/O system calls made by applications. Namely, DIO collects:

  • system call type, arguments and return value
  • process name (command), process ID, thread ID
  • start and end timestamps
  • additional context from the kernel:
  • file type
  • file offset
  • file tag
  
  

• Pratical and timely analysis

  DIO provides a full pipeline to capture, analyze and visualize collected data in a timely manner. Namely, DIO provides:

  • a tracer component that intercepts system calls made by applications.
  • a backend component that stores collected data in a database and enables users to query traced data, apply filters to analyze specific information, and correlate different types of data.
  • a visualizer component that allows users to query the database and visualize the results in a web interface and build customized visualizations.

Publications

About DIO

• Diagnosing applications' I/O behavior through system call observability.
  Tânia Esteves, Ricardo Macedo, Rui Carlos Oliveira and João Paulo.
  53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), 2023.
  

• Toward a Practical and Timely Diagnosing of Applications' I/O Behavior..
  Tânia Esteves, Ricardo Macedo, Rui Carlos Oliveira and João Paulo.
  IEEE Access. 2023.
  

Using DIO

• CRIBA: A Tool for Comprehensive Analysis of Cryptographic Ransomware's I/O Behavior.
  Tânia Esteves, Bruno Pereira, Rui Pedro Oliveira, João Marco and João Paulo.
  42nd International Symposium on Reliable Distributed Systems (SRDS), 2023.
  

DIO Use cases

DIO was testes and validated with two production-level systems.

• Inefficient I/O behavior of Redis.
  Diagnosis of inefficient use of system calls that lead to poor storage performance.

• Top-down exploration and diagnosis of Elasticsearch.
  Observation of unexpected file access patterns caused by the usage of high-level libraries that lead to redundant I/O calls.

• Identifying Fluent Bit's erroneous actions that lead to data loss.
  Root cause analysis of data loss caused by erroneous file accesses.

• Finding the Root Cause of RocksDB's Performance Anomalies.
  Root cause analysis of resource contention in multi threaded I/O that leads to high tail latency for user workloads.