General Co-Chairs
Rakesh Agrawal, Data Insights Laboratories, San Jose, CA, USA
Aoying Zhou, East China Normal University
Program Co-Chairs
Weining Qian, East China Normal University
Sascha Hunold, TU Wien, Austria
Program Vice-Chairs
Biwei Xie, Institute of Computing Technology, CAS
Kai Shu, Illinois Institute of Technology
Web Chair
Jiahui Dai, BenchCouncil
Publicity Chairs
Weicong Chen, University of California, Merced
Dipti Shankar, SAP Labs, Munich
Ahmad Ghazal, PingCAP
Bartlomiej Przybylski, Adam Mickiewicz University
Ben Blamey, Uppsala University
Benson Muite, University of Tartu
Bin Ren, College of William & Mary
Bo Wu, Colorado School of Mines
Chen Zheng, Institute of software, Chinese Academy of Sciences
Cheol-Ho Hong, Chung-Ang University
Chunjie Luo, Institute of Computing Technology, Chinese Academy of Sciences
David Boehme, LLNL
Florina Ciorba, University of Basel
Francieli Boito, University of Bordeaux
Guangli Li, Institute of Computing Technology, Chinese Academy of Sciences
Gwangsun Kim, POSTECH
Joseph Schuchart, University of Tennessee, Knoxville
K. Selçuk Candan, Arizona State University
Khaled Ibrahim, Lawrence Berkeley National Laboratory
Lucas Mello Schnorr, UFRGS
Mario Marino, Leeds Beckett University
Miaoqing Huan, University of Arkansas
Murali Emani, Argonne National Laboratory
Philippe Swartvagher, TU Wien
Piotr Luszczek, University of Tennessee Knoxville
Ryan Grant, Sandia National Laboratories
Salman Zubair Toor, Uppsala University
Shu Yin, ShanghaiTech University
Sophie Cerf, INRIA, France
Steven Farrell, Lawrence Berkeley National Laboratory
Todor Ivanov, DXC Technology
Woongki Baek, UNIST
Xiaoyi Lu, University of California, Merced
Zhen Jia, Amazon
Zhihui Du, New Jersey Institute of Technology
2023 BenchCouncil Achievement Award Committee:
D. K. Panda, the Ohio State University
Lizy Kurian John, the University of Texas at Austin
Geoffrey Fox, Indiana University
Jianfeng Zhan, University of Chinese Academy of Sciences
Tony Hey, Rutherford Appleton Laboratory STFC
David J. Lilja, University of Minnesota, Minneapolis
Jack J. Dongarra, University of Tennessee
John L. Henning, Oracle
2023 BenchCouncil Rising Star Award Committees:
D. K. Panda, the Ohio State University
Lizy Kurian John, the University of Texas at Austin
Geoffrey Fox, Indiana University
Jianfeng Zhan, University of Chinese Academy of Sciences
Torsten Hoefler, ETH Zürich
Vijay Janapa Reddi, Harvard University
Peter Mattson, Google, USA
Douwe Kiela, Stanford University
BenchCouncil Distinguished Doctoral Dissertation Award Committee in Other Areas:
Jack Dongarra, University of Tennessee
Xiaoyi Lu, The University of California
Jeyan Thiyagalingam, STFC-RAL
Lei Wang, ICT, Chinese Academy of Sciences
Spyros Blanas, The Ohio State University
BenchCouncil Distinguished Doctoral Dissertation Award Committee in Computer Architecture:
Resit Sendag, University of Rhode Island
Peter Mattson, Google
Vijay Janapa Reddi, Harvard University
Wanling Gao, Chinese Academy of Sciences
Jack Dongarra, University of Tennessee
Geoffrey Fox, Indiana University
D. K. Panda, The Ohio State University
Felix, Wolf, TU Darmstadt
Xiaoyi Lu, University of California
Resit Sendag, University of Rhode Island
Wanling Gao, ICT, Chinese Academy of Sciences & UCAS
Jianfeng Zhan, BenchCouncil
2023 BenchCouncil International Conference on Benchmarking, Measuring and Optimizing
The Bench conference encompasses a wide range of topics in benchmarking, measurement, evaluation methods and tools. Bench’s multi-disciplinary emphasis provides an ideal environment for developers and researchers from the architecture, system, algorithm, computing, and application communities to discuss practical and theoretical work covering workload characterization, benchmarks and tools, evaluation, measurement and optimization, and dataset generation.
Researchers both from academia and industry are invited to submit proposals for special sessions. A special session could consist of a set of individual presentations or a panel. Special sessions complement the regular technical program by highlighting emerging research topics. We welcome special sessions in all areas of Benchmarking and Optimizations (See https://www.benchcouncil.org/bench2023/cfp.html for topics). The participation of speakers from industry in a special session proposal is encouraged and will be positively evaluated when the proposal is assessed.
Proposal Submission
Please submit your proposals (maximum 3 pages) in PDF format by email to the Special Sessions Chair. Proposal should include the following information:
1. Title and Abstract: Propose a title and a brief abstract of no more than approximately 200 words that will allow conference attendees to understand the topic and the focus of the special session.
2. Rationale: Please explain why the topic of the special session is timely and compelling and why it is relevant to the Bench community.
3. Session Format: Please describe what the session format is (e.g., workshop, tutorial, panel, a series of invited talks, and so on).
4. Expected Length and/or Agenda: Please describe what the expected length of your session is (e.g., one hour, two hours, half day, and one day). Please feel free to include an expected agenda if needed.
5. Biographies: Please provide the list of organizers/speakers with a short bio, including the recent activity that is relevant to the topic of the special session.
6. Other Information: Please feel free to describe other important information about your proposed session. Note that the total proposal length should be within 3 pages.
* Proposals are due on or before August 31, 2023 11:59 pm (AoE) and should be submitted to the Special Sessions Chairs, Kai Shu and Reza Zafarani, by email ( kshu@iit.edu and/or reza@data.syr.edu ).
* Notification of acceptance will be sent to the organizers by September 15, 2023.
Sanya, also known as "China's Hawaii," is a breathtaking destination renowned for its stunning natural scenery, pleasant climate, and pristine air quality. Located on the southern coast of Hainan Island, Sanya boasts a perfect combination of picturesque landscapes and a tropical paradise. The climate in Sanya is pleasant, with a perpetual spring-like atmosphere, and one can escape the cold winter elsewhere and indulge in the warm sunshine, pristine beaches, crystal-clear waters, delectable cuisine, and lush tropical landscapes.
The Travel Grant is a compensatory award that is available to applicants who have registered for the conference. The conference will be held Dec.3 – Dec.6 in Sanya, Hainan, China, all registers are eligible to apply. Since the total grant is a fixed amount, the amount that can be awarded to each register will depend on the number of awardees and will cover only a portion of your expenses.
There are two levels of Travel Grant: 2100 RMB (300 USD) for up to 20 people, and 700 RMB (100 USD) for up to 100 people. The final list of recipients of the Travel Grant will be announced during the conference banquet. Please note that individuals who have already applied for the family meal ticket are not eligible to apply for the Travel Grant.
Important information about travel grants:
If you wish to apply for the grant, please contact student.travel.grant.bench@gmail.com.
Use the subject line "Bench 2023 Travel Grant Application." The travel grant application (a single consolidated pdf file) should include:
BenchCouncil has launched the Top100 initiative, which aims to recognize the top 100 achievements in the fields of chips, AI, open source, evaluations, and benchmarks. These achievements are known as Chip100, AI100, Open100, and Bench100. As a result, we have compiled both annual and centennial versions of four rankings, which can be accessed publicly at https://www.benchcouncil.org/evaluation/. These rankings have generated significant interest, comments, and suggestions from academic and industry practitioners worldwide.
For those that have been selected into annual version of the BenchCouncil Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100)
Academia complimentary Service:
Industry complimentary Service:
Free registration: https://eur.cvent.me/eAxmP
For every selected achievement, BenchCouncil will provide you with a standard-size exhibition space (2.5m*1m) if you need it. Please note that we try to provide tables and chairs if possible. However, we cannot guarantee that. If you need roll-up banners, please prepare them yourself, and the size should be 2m*0.8m. [Roll-up Banner Template]
Furthermore, we offer upgrade services for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. We can also provide customized services upon request.
If you need to register for upgrade service, please transfer the cash to the following bank account.
Meanwhile, please send an email to benchcouncil.evaluation@gmail.com. Also, you can upgrade services through the registration link https://eur.cvent.me/eAxmP.
Academia Upgrade Service:
Industry Upgrade Service:
To avoid missing important achievements, BenchCouncil calls for the submission of achievements in the fields of AI, Chips, Open source, benchmarks, and evaluations. After the preliminary evaluation, the submitters of the achievements will have the opportunity to present or showcase their work at the FICC 2023 conference and have a chance to be included in the final rankings.
Ten benefits
1. The submitted achievements will be included in the annual achievement report of the BenchCouncil and will be cited. The submitters need to provide valid citation formats, including paper publications, code repositories, etc. The compiled annual achievement report will be officially published in BenchCouncil Transactions.
2. The submitters of the achievements will receive the certificate of annual achievement Selection. For example, the certificate for the Chip category will be named "the certificate of 2023 BenchCouncil Chip achievement selection." The certificate will display the individual and institutional names.
3. The achievements will be featured on the official English webpages of the BenchCouncil's achievement evaluation (https://www.benchcouncil.org/evaluation/).
4. The official English webpages of the BenchCouncil will provide links to the achievements. The submitters should ensure that the linked content is authentic, non-deceptive, and accurate.
5. The submitters will have the opportunity to attend all four conferences: Chips 2023, OpenCS 2023, Bench 2023, and IC 2023. They will have the chance to engage in extensive exchanges with international peers.
6. The selected achievements will be showcased at the conference venue. The specific time period (generally not exceeding one day) and location need to be coordinated with the conference organizers.
7. The submitters of the achievements have the opportunity to present their work at the conference, subject to further evaluation and arrangement by the conference organizers.
8. Outstanding achievements invited to present at the conference may have the chance to be included in the annual version of the Top 100 achievements ranking (final version) after evaluation by the committee and recognition from peers. The committee has reserved 20 slots for selecting achievements from the submissions.
9. Exemplary achievements that have stood the test of time have the chance to be included in the annual version of the Top 100 achievements ranking (request for comments).
10. The submitters of the achievements can apply for travel funding.
Scope of achievements:
In the field of chips, including but not limited to as follows:
system design, logic design, physical design, timing design, verification and simulation, emulators, auxiliary design tools, emerging chips such as superconducting and quantum chips, emerging accelerators, semiconductor materials, lithography technology, circuits, packaging technology.
In the field of artificial intelligence, including but not limited to as follows:
machine learning, reinforcement learning, natural language processing, image analysis, video analysis, data mining, recommendation systems, knowledge representation and reasoning, medical image processing, multimodal information fusion, collective intelligence, intelligent robots and systems, intelligent control, intelligent healthcare, artificial intelligence and astronomy, artificial intelligence and high-energy physics, artificial intelligence and space science, artificial intelligence and transportation, artificial intelligence and oceanography, artificial intelligence and security, artificial intelligence and law, artificial intelligence and finance, artificial intelligence and traditional industries, ethics and governance of artificial intelligence, artificial intelligence and big data, artificial intelligence and materials, artificial intelligence and civil aviation applications, artificial intelligence and business management.
In the field of open source, including but not limited to as follows:
chips, operating systems, containerization and virtualization, compilers, data management, networks, programming languages and compilers, systems and frameworks, basic libraries, monitoring and optimization tools, big data, AI, cloud computing, HPC, blockchain, LLM, autonomous driving, digital twins, Internet of Things, privacy and security, datasets, education.
In the field of benchmark and evaluation, including but not limited to as follows:
econometrics, clinical medical evaluation, drug evaluation, business and financial benchmarks, AI evaluation, HPC benchmarks, database benchmarks, memory benchmarks, network benchmarks, hardware and disk benchmarks, graph benchmarks, human resource evaluation, education evaluation, indicator and index systems.
Process of submitting achievements:
Evaluation criteria:
After being selected, you must register the conference.
please transfer the cash to the following bank account.
Also, You can pay using your credit card with following registration link https://eur.cvent.me/5qQEq
1. 1 standard registration for FICC 2023.
Furthermore, we offer upgrade services (the registration link is: https://eur.cvent.me/k85BK) for academic or industrial institutions that have been selected for the Top 100 achievements ranking list (AI100, Chip100, Open100, Bench100). Institutions that register for the upgraded service can enjoy additional benefits at the FICC 2023 conference. The number of registrations, duration of plenary session speech, and exhibition space will vary depending on whether you are from academia or industry. Also, we provide customized services upon request.
Academia Upgrade Service | ||||
---|---|---|---|---|
Service Level |
Bronze Service |
Silver Service |
Golden Service |
Platinum Service |
Expense | USD $1000 | USD $3000 | USD $4000 | USD $5000 |
Free Registration | 2 free registrations for FICC 2024 | 4 free registrations for FICC 2024 | 4 free registrations for FICC 2024 | 6 free registrations for FICC 2024 |
Session Time | 20-minute regular session | 20-minute regular session | 30-minute primary session | 30-minute plenary session |
Exhibition | Double standard-size exhibition space for 2 days | Double standard-size exhibition space for 4 days | Double standard-size exhibition space for 4 days | Double standard-size exhibition space for 4 days |
Evaluation Report | 5-page evaluation report. Please visit https://www.benchcouncil.org/evaluation/ for more information |
Industry Upgrade Service | ||||
---|---|---|---|---|
Service Level |
Bronze Service |
Silver Service |
Golden Service |
Platinum Service |
Expense | USD $2000 | USD $4000 | USD $8000 | USD $10000 |
Free Registration | 2 free registrations for FICC 2023 | 4 free registrations for FICC 2023 | 6 free registrations for FICC 2023 | 8 free registrations for FICC 2023 |
Session Time | 20-minute regular session | 20-minute regular session | 30-minute primary session | 30-minute plenary session |
Exhibition | Double standard-size exhibition space for 2 days | Double standard-size exhibition space for 4 days | Double standard-size exhibition space for 4 days | Four standard-size exhibition spaces for 4 days |
Full Papers: July 31, 2023, at 11:59 PM AoE
Notification: September 30, 2023, at 11:59 PM AoE
Final Papers Due: October 31, 2023, at 11:59 PM AoE
Conference Date: December 3-5, 2023
Venue: Sanya, China
Please note that citizens from up to 59 nations can visit Sanya without a Visa from the Chinese Government. Sanya is a beautiful seaside city, well known as Hawaii in China.
Submission website: https://bench2023.hotcrp.com/
The Bench conference encompasses a wide range of topics in benchmarks, datasets, metrics, indexes, measurement, evaluation, optimization, supporting methods and tools, and other best practices in computer science, medicine, finance, education, management, etc. Bench’s multidisciplinary and interdisciplinary emphasis provides an ideal environment for developers and researchers from different areas and communities to discuss practical and theoretical work. The topics of interest include, but are not limited to the following:
1. The online discussion is blind. While the reviewers discuss the papers, they don’t know others’ identities beyond reviewer #A, #B, …. Hence, a single reviewer cannot easily assert seniority and silence other voices, or influence them beyond the strength of their arguments.
2. When the reviewers pointing out closeness to prior work that informs the reviewer’s decision to lower the novelty and contribution of a paper, they should provide a full citation to that prior work.
3. When the reviewers asking authors to draw a comparison with concurrent work (e.g., work that was published or appeared online *after* the paper submission deadline) or with preliminary work (e.g., a poster or abstract that is not archival), this comparison should not inform a lower score by the reviewer.
4. Provide useful and constructive feedback to the authors. Be respectful, professional and positive in your reviews and provide suggestions for the authors to improve their work.
5. Score the paper absolutely and relative to the group of papers you are reviewing.
Absolute overall merit - There are 4 grades you can give to each paper for absolute overall merit; the top 2 ratings mean that you think the paper is acceptable to the conference and the bottom 2 ratings mean that in your opinion the paper is below the threshold for the conference. Please assign these values thinking, whether the paper is above the threshold for the conference or below.
Relative overall merit – is based on the papers that you are reviewing. You can rank your papers and then group the papers into the 4 bins. Except for fractional errors, you should be dividing your papers equally into the 4 categories.
6. Reviewers must treat all submissions as strictly confidential and destroy all papers once the technical program has been finalized.
7. Reviewers must contact the PC chair or EIC if they feel there is an ethical violation of any sort (e.g., authors seeking support for a paper, authors seeking to identify who the reviewers are).
8. Do not actively look for author identities. Reviewers should judge a paper solely on its merits.
9. If you know the authors, do not publicize the authors. If you would like to recuse yourself from the review task, contact the PC Chair.
10. Reviewers should review the current submission. If you have reviewed a previous submission, make sure your review is based on the current submission.
11. Reviewers must not share the papers with students/colleagues.
12. Reviewers must compose the reviews themselves and provide unbiased reviews.
13. Do not solicit external reviews without consulting the PC chairs or EIC. If you regularly involve your students in the review process as part of their PhD training, contact the PC chairs. You are still responsible for the reviews. You may do this on no more than one of your reviews.
14. Reviewers must keep review discussions (including which papers you reviewed) confidential.
15. Do not discuss the content of a submitted paper/reviews with anyone other than officially on the submission management system like HotCRP or EasyChair during the online discussion period or the PC meeting (from now until paper publication in any venue).
16. Do not reveal the name of paper authors in case reviewers happen to be aware of author identity. (Author names of accepted papers will be revealed after the PC meeting; author names of rejected papers will never be revealed.)
17. Do not disclose the outcome of a paper until its authors are notified of its acceptance or rejection.
18. Do not download or acquire material from the review site that you do not need access to.
19. Do not disclose the content of reviews, including the reviewers' identities, or discussions about a paper.
Acknowledgments:
This set of review ethics is derived and based on the MICRO 2020, ASPLOS
2020-2021, ISCA 2020-21 review guidelines.
Bench 2023 Submission website: https://bench2023.hotcrp.com/
Papers must be submitted in PDF. For a full paper, the page limit is 15 pages in the LNCS format, not including references. For a short paper, the page limit is 8 pages in the LNCS format, not including references. The review process follows a strict double-blind policy per the established Bench conference norms. The submissions will be judged based on the merit of the ideas rather than the length. After the conference, the proceedings will be published by Springer LNCS (Pending, Indexed by EI). Please note that the LNCS format is the final one for publishing.
At least one author must pre-register for the symposium, and at least one author must attend the symposium to present the paper. Papers for which no author is pre-registered will be removed from the proceedings.
LNCS latex template: https://www.benchcouncil.org/file/llncs2e.zip
Please make sure your submission satisfies ALL of the following requirements:
General Co-Chairs:
Rakesh Agrawal, Data Insights Laboratories, San Jose, CA, USA
Aoying Zhou, East China Normal University
Program Co-Chairs:
Weining Qian, East China Normal University
Program Vice-Chair:
Biwei Xie, Institute of Computing Technology, CAS
Kai Shu, Illinois Institute of Technology
Web Co-chairs:
Jiahui Dai, BenchCouncil
2023 BenchCouncil Achievement Award Committee:
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Tony Hey, Rutherford Appleton Laboratory STFC (Since 2020)
Prof. David J. Lilja, University of Minnesota, Minneapolis (Since 2021)
Prof. Jack J. Dongarra, University of Tennessee (Since 2022)
John L. Henning, Oracle (Since 2023)
2023 BenchCouncil Rising Star Award Committees:
Prof. D. K. Panda, the Ohio State University
Prof. Lizy Kurian John, the University of Texas at Austin
Prof. Geoffrey Fox, Indiana University
Prof. Jianfeng Zhan, University of Chinese Academy of Sciences
Prof. Torsten Hoefler, ETH Zürich (Since 2021)
Prof. Vijay Janapa Reddi, Harvard University (Since 2022)
Dr. Peter Mattson, Google, USA (Since 2022)
Dr. Wanling Gao , ICT, Chinese Academy of Sciences (pending)
Dr. Douwe Kiela, Stanford University (Since 2023)
2023 BenchCouncil Achievement Award Committees:
BenchCouncil Distinguished Doctoral Dissertation Award Committee in Other Areas:
Prof. Jack Dongarra, University of Tennessee
Dr. Xiaoyi Lu, The University of California, Merced
Dr. Jeyan Thiyagalingam, STFC-RAL
Dr. Lei Wang, ICT, Chinese Academy of Sciences
Dr. Spyros Blanas, The Ohio State University
BenchCouncil Distinguished Doctoral Dissertation Award Committee in Computer Architecture:
Prof. Resit Sendag, University of Rhode Island, USA
Dr. Peter Mattson, Google
Dr. Vijay Janapa Reddi, Harvard University
Dr. Wanling Gao, Chinese Academy of Sciences
Prof. Dr. Jack Dongarra, University of Tennessee
Prof. Dr. Geoffrey Fox, Indiana University
Prof. Dr. D. K. Panda, The Ohio State University
Prof. Dr. Felix, Wolf, TU Darmstadt
Prof. Dr. Xiaoyi Lu, University of California, Merced
Prof. Dr. Resit Sendag, University of Rhode Island, USA
Dr. Wanling Gao, ICT, Chinese Academy of Sciences & UCAS
Prof. Dr. Jianfeng Zhan, BenchCouncil