Elasticsearch use case

How to reproduce

This page describes how to reproduce the results presented in the paper for four setups:

• Vanilla: without tracing I/O events
• Strace: tracing system calls with Strace tool
• Sysdig: tracing system calls with Sysdig tool
• DIO: tracing and analyzing system calls with DIO

Elasticsearch configuration

• Source code: https://github.com/elastic/elasticsearch/releases/tag/v8.3.0
• Version: 8.3.0
• Configurations (elasticsearch.yml):
  • xpack.security.enabled: false
  • discovery.type: single-node
  • network.host: 0.0.0.0

Benchmark configuration

• Benchmark: Rally
• Repository: https://esrally.readthedocs.io/en/stable/install.html
• Docker image: elastic/rally
• Configurations:
  • workload: geonames (--track=geonames)
  • pipeline: benchmark-only (--pipeline=benchmark-only)
  • target host: local elasticsearch instance (--target-hosts=localhost:9200)

DIO configuration

• Tracer Configurations:
  • Filter events by file paths (target_paths):
  • Elasticsearch folder: /usr/share/elasticsearch-8.3.0-SNAPSHOT

Docker images

• Docker image: taniaesteves/elasticsearch_dio:v1.0.0
• Run Elasticsearch server:
  • Vanilla:
  docker run -it -d --name es830 --pid=host --privileged --cap-add=ALL --net=host -v /lib/modules:/lib/modules -v /usr/src:/usr/src -v /sys/kernel/debug/:/sys/kernel/debug/ taniaesteves/elasticsearch_dio:v1.0.0
  • strace:
  docker run -it -d --name es830 --pid=host --privileged --cap-add=ALL --net=host -v /lib/modules:/lib/modules -v /usr/src:/usr/src -v /sys/kernel/debug/:/sys/kernel/debug/ -v /tmp/strace_data:/strace_data taniaesteves/elasticsearch_dio:v1.0.0 strace
  • Sysdig (run along the vanilla command):
  docker run -it -d --name sysdig --privileged -v /var/run/docker.sock:/host/var/run/docker.sock -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro -v /tmp/sysdig_data:/home --net=host -e SYSDIG_BPF_PROBE="" sysdig/sysdig:0.31.4 sysdig -B -t a -p "*%evt.num %evt.outputtime %evt.cpu %proc.name (%thread.tid) %evt.dir %evt.type %evt.rawres %evt.args" container.name=es830 and "evt.type in ('"'open','openat','creat','read','pread','readv','write','pwrite','writev','lseek','truncate','ftruncate','rename','renameat','renameat2','close','unlink','unlinkat','stat','fstat','lstat','fstatfs','newfstatat','setxattr','getxattr','listxattr','removexattr','lsetxattr','lgetxattr','llistxattr','lremovexattr','fsetxattr','fgetxattr','flistxattr','fsync','fdatasync','readlink','readlinkat','mknod','mknodat'"')" -s 1 -w /home/sysdig_trace.scap
  • DIO:
  docker run -it -d --name es830 --pid=host --privileged --cap-add=ALL --net=host -v /lib/modules:/lib/modules -v /usr/src:/usr/src -v /sys/kernel/debug/:/sys/kernel/debug/ -v /tmp/dio_data:/dio_data -e CORRELATE_PATHS=true -e ES_URL=<ES_URL> taniaesteves/elasticsearch_dio:v1.0.0 dio --target_paths /usr/share/elasticsearch-8.3.0-SNAPSHOT
• Run Elasticsearch benchmark:
• docker run --rm --net=host elastic/rally race --track=geonames --pipeline=benchmark-only --target-hosts=localhost:9200