Topics

Benchmark science and engineering across multi-disciplines:

• The formulation of problems or challenges in emerging and future computing
• The benchmarks, datasets, and indexes in multidisciplinary applications, e.g., medical, finance, education, management, psychology, etc.
• Benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges, industry best practices

Benchmark and standard specifications, implementations, and validations:

• Big Data, Artificial intelligence (AI), High performance computing (HPC)
• Machine learning, Big scientific data, Datacenter, Cloud, Warehouse-scale computing
• Mobile robotics, Edge and fog computing, Internet of Things (IoT), Blockchain
• Data management and storage, Financial, Education, Medical or other application domains

Datasets:

• Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements
• Analyses or meta-analyses of existing data and original articles on systems, technologies, and techniques that advance data sharing and reuse to support reproducible research
• Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description
• Tools that can generate large-scale data while preserving their original characteristics

Workload characterization, quantitative measurement, design, and evaluation studies:

• Computer and communication networks, protocols and algorithms
• Wireless, mobile, ad-hoc and sensor networks, IoT applications
• Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks
• HPC systems; Operating systems, file systems and databases; Virtualization, data centers, distributed and cloud computing, fog and edge computing
• Mobile and personal computing systems, energy-efficient computing systems; real-time and fault-tolerant systems, security and privacy of computing and networked systems, software systems and services, and enterprise applications, social networks, multimedia systems, web services, cyber-physical systems, including the smart grid

Methodologies, metrics, abstractions, algorithms, and tools:

• Analytical modeling techniques and model validation
• Workload characterization and benchmarking
• Performance, scalability, power and reliability analysis
• Sustainability analysis and power management
• System measurement, performance monitoring and forecasting
• Anomaly detection, problem diagnosis and troubleshooting
• Capacity planning, resource allocation, run time management and scheduling
• Experimental design, statistical analysis, and simulation

Measurement and evaluation:

• Measurement standards, evaluation methodologies and metrics, testbed methodologies and systems
• Instrumentation, sampling, tracing and profiling of large-scale, real-world applications and systems
• Measurement-based modeling (e.g., workloads, scaling behavior, assessment of performance bottlenecks)
• Methods and tools to monitor and visualize measurement and evaluation data
• Systems and algorithms that build on measurement-based findings
• Advances in data collection, analysis and storage (e.g., anonymization, querying, sharing)
• Reappraisal of previous empirical measurements and measurement-based conclusions
• Descriptions of challenges and future directions that the measurement and evaluation community should pursue