Skip navigation

BenchCouncil: International Open Benchmark Council

 

BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) is an open-access multi-disciplinary journal dedicated to benchmarks, standards, evaluations, optimizations, and data sets. It will take a hybrid publication mode with the Bench Conference. Both TBench and Bench use a double-blind review process. Papers that are accepted to present at the Bench conference will appear in the issue of TBench immediately following acceptance. Meanwhile, the accepted TBench papers will be encouraged but not mandatory to register and present at the Bench conference. TBench will be OA without any charge. It seeks a fast-track publication with an average turnaround time of two months. Submit your paper at https://www.editorialmanager.com/tbench/default.aspx.

TBench Editorial Board

Co-EIC

Prof. Dr. Jianfeng Zhan, ICT, Chinese Academy of Sciences and BenchCouncil

Prof. Dr. Tony Hey, Rutherford Appleton Laboratory STFC, UK

Assistant EIC

Dr. Wanling Gao, ICT, Chinese Academy of Sciences and BenchCouncil

Chunjie Luo, University of Chinese Academy of Sciences, China

Advisory Board

Prof. Jack Dongarra, University of Tennessee, USA

Prof. Geoffrey Fox, Indiana University, USA

Prof. D. K. Panda, The Ohio State University, USA

Founding Editor

Prof. H. Peter Hofstee, IBM Systems, USA and Delft University of Technology, Netherlands

Dr. Zhen Jia, Amazon, USA

Prof. Blesson Varghese, Queen's University Belfast, UK

Prof. Raghu Nambiar, AMD,USA

Prof. Jidong Zhai, Tsinghua University, China

Prof. Francisco Vilar Brasileiro, Federal University of Campina Grande, Brazil

Prof. Jianwu Wang, University of Maryland, USA

Prof. David Kaeli, Northeastern University, USA

Prof. Bingshen He, National University of Singapore, Singapore

Dr. Lei Wang, Institute of Computing Technology, Chinese Academy of Sciences, China

Prof. Weining Qian, East China Normal University, China

Dr. Arne J. Berre, SINTEF, Norway

Prof. Zhifei Zhang, Capital Medical University

Dr. Yunyou Huang, Guangxi Normal University


If your feel interested in joining the TBench editorial board, please contact:

Editor-in-Chief

Dr. Jianfeng Zhan

Professor, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences, and BenchCouncil

zhanjianfeng (at) ict (dot) ac (dot) cn

Dr. Tony Hey

Rutherford Appleton Laboratory STFC, UK

Tony (dot) Hey (at) stfc (dot) ac (dot) uk


Aims and Scopes

It publishes a range of papers, including letters, research articles, survey articles, benchmark articles, standard articles, data articles, tool articles, practice articles, and comments on previously published papers. Particular areas of interest include, but are not limited to:

  • Benchmark and standard specifications, implementations, and validations of:
    • Big Data
    • Artificial intelligence (AI)
    • High performance computing (HPC)
    • Machine learning
    • Big scientific data
    • Datacenters
    • Cloud
    • Warehouse-scale computing
    • Mobile robotics
    • Edge and fog computing
    • Internet of Things (IoT)
    • Blockchain
    • Data management and storage
    • Science domains
    • Medical domains
    • Financial domains
    • Education domains
    • Other application domains
  • Open access data sets:
    • Detailed descriptions of research or industry data sets, including the methods used to collect the data and technical analyses supporting the measurements' quality.
    • Analyses or meta-analyses of existing data and original articles on systems, technologies, and techniques that advance data sharing and reuse to support reproducible research.
    • Evaluations of the rigor and quality of the experiments used to generate data and the completeness of the data's descriptions.
    • Tools generating large-scale data while preserving their original characteristics.
  • Workload characterization, quantitative measurement, design and evaluation studies of:
    • Computer and communication networks, protocols, and algorithms
    • Wireless, mobile, ad-hoc and sensor networks, IoT applications
    • Computer architectures, hardware accelerators, multi-core processors, memory systems, and storage networks
    • HPC
    • Operating systems, file systems, and databases
    • Virtualization, data centers, distributed and cloud computing, fog, and edge computing
    • Mobile and personal computing systems
    • Energy-efficient computing systems
    • Real-time and fault-tolerant systems
    • Security and privacy of computing and networked systems
    • Software systems and services, and enterprise applications
    • Social networks, multimedia systems, web services
    • Cyber-physical systems, including the smart grid
  • Methodologies, abstractions, metrics, algorithms and tools for:
    • Analytical modeling techniques and model validation
    • Workload characterization and benchmarking
    • Performance, scalability, power, and reliability analysis
    • Sustainability analysis and power management
    • System measurement, performance monitoring, and forecasting
    • Anomaly detection, problem diagnosis, and troubleshooting
    • Capacity planning, resource allocation, run time management, and scheduling
    • Experimental design, statistical analysis, and simulation
  • Measurement and evaluation:
    • Evaluation methodologies and metrics
    • Testbed methodologies and systems
    • Instrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systems
    • Collection and analysis of measurement data that yield new insights
    • Measurement-based modeling (e.g., workloads, scaling behavior, assessment of performance bottlenecks)
    • Methods and tools to monitor and visualize measurement and evaluation data
    • Systems and algorithms that build on measurement-based findings
    • Advances in data collection, analysis, and storage (e.g., anonymization, querying, sharing)
    • Reappraisal of previous empirical measurements and measurement-based conclusions
    • Descriptions of challenges and future directions that the measurement and evaluation community should pursue
  • Optimization methodologies and tools

Submission

Submission Site

https://www.editorialmanager.com/tbench/default.aspx

Peer Review

The articles in this journal are peer-reviewed according to the BenchCouncil Journals review rules. Before sending the paper to a specific Associate Editor (AE) for further handling, the Editor-in-Chief (EIC) team, consisting of one EIC, one or two associate EICs, and one assistant EIC, will guarantee each paper reaches the minimum publication standards. The one member of the EIC team without conflict of interest (COI) is responsible for checking COIs, while the other EIC and AE, who do not know the authors' identities, make a final decision. Each published article is reviewed by a minimum of three independent reviewers using a double-blind peer-review process. The identities of the reviewers are not known to the authors. Still, the reviewers also do not know the identities of the authors. Articles are screened for plagiarism before acceptance. For each paper, the EIC team makes a final decision within three months.

Cost

TBench is an open-access publication. To publish in TBench, authors are not required to pay an article-processing charge (APC).

Manuscript Types Accepted by TBench

TBench manuscript types and submission length guidelines are described below. All page limits do not include references and author biographies.

  • Research article – 12 double-column pages (All research article page limits do not include references and author biographies.)

  • Survey article – no page limits.

  • Benchmark article – 12 double-column pages (All benchmark article page limits do not include references and author biographies.)

  • Standard article – 12 double-column pages (All standard article page limits do not include references and author biographies.)

  • Data article – 8 double-column pages (All data article page limits do not include references and author biographies.)

  • Tool article – 8 double-column pages (All tool article page limits do not include references and author biographies.)

  • Practice article – 8 double-column pages (All practice article page limits do not include references and author biographies.)

  • Editor’s Letter– 4 double-column pages

  • Letter– 4 double-column pages (All letter page limits do not include references and author biographies.) After being peer-reviewed, the EIC team will make a final decision with one month.

  • Comment – 2 double-column pages. The EIC team will make a final decision with one week.


Review Rules

1. The articles in this journal are peer-reviewed according to the BenchCouncil Journal review rules. Before sending the paper to a specific AE for further handling, the EIC team, two EIC, three associate EIC, and one assistant EIC will guarantee each article reaching the minimum publication standards. One member of the EIC team without conflict of interest (COI) is responsible for checking COI, while the other EIC and AE who do not know the authors' identities made a final decision.

2. Each published article was reviewed by a minimum of three independent reviewers using a double-blind peer-review process. The authors' identities are unknown to the authors, and the reviewers also do not know the authors' identities.

3. Articles will be screened for plagiarism before acceptance.

4. When the reviewers' pointing out closeness to prior work that informs the reviewer’s decision to lower the novelty and contribution of a paper, they should provide a full citation to that previous work.

5. In the following cases, this comparison should not inform a lower score by the reviewer. The reviewers ask authors to draw a comparison with concurrent work published or appeared online after the paper submission deadline or with preliminary work, e.g., a poster or abstract that is not archival.

6. Provide useful and constructive feedback to the authors. Be respectful, professional, and positive in your reviews and provide suggestions for the authors to improve their work.

7. Reviewers must contact the AE or EIC if they feel there is an ethical violation of any sort (e.g., authors seeking support for a paper, authors seeking to identify who the reviewers are).

8. Do not actively look for author identities. Reviewers should judge a paper solely on its merits.

9. Reviewers should review the current submission. If you have reviewed a previous submission, make sure your review is based on the current submission.

10. Reviewers must not share the papers with students/colleagues.

11. Reviewers must compose the reviews themselves and provide unbiased reviews.

12. Do not solicit external reviews without consulting the EIC. If you regularly involve your students in the review process as part of their Ph.D. training, contact the EIC. You are still responsible for the reviews.

13. Do not discuss the content of a submitted paper/review with anyone from now until paper publication in any venue.

14. Do not reveal the name of paper authors if reviewers happen to be aware of the author's identity. (Author names of accepted papers will be revealed after being accepted; The editorial board will never reveal author names of rejected papers.)

15. Do not disclose a paper's outcome until the editorial board notifies its acceptance or rejection to its authors.

16. Do not download or acquire material from the review site you do not need access to.

17. Do not disclose the reviews' content, including the reviewers' identities or discussions about a paper.