SC24: Technical Program Leaders Focus on Their Position and Scientific Imaginative and prescient

This text was ready by Kevin Jackson for the annual Supercomputer Convention; SC24 can be held in Atlanta from Sunday, Nov. 17 to Friday, Nov. 22.

Science lies on the coronary heart of the annual Supercomputing convention, and the Technical Program is without doubt one of the most necessary and difficult elements of the convention. To study extra about what this program does, in addition to the scientific imaginative and prescient that drives each determination inside the program, right here’s an interview with SC24 Technical Program Chair Guillaume Pallez (Inria) and Vice Chair Judith Hill (LLNL).

Q: Are you able to inform us about your function as SC24 Technical Program Chair?

TP: Everybody might be conscious of the breadth of the SC Technical Program.

The seen half to most attendees contains the Technical Papers, Workshops, Tutorials, Panels, Posters, and Birds of a Feather. This 12 months we even launched a brand new monitor referred to as Artwork of HPC. The Technical Program additionally contains SC-specific awards similar to Take a look at of Time, and we coordinate with the computing societies (ACM, IEEE, SIGHPC, TCHPC) for his or her particular awards and the award ceremony.

Then there are the much less seen elements, that are simply as necessary. These embody the reproducibility analysis for the Technical Papers, in addition to the coordination with the SC Scholar Cluster Competitors to pick the paper from final 12 months that can be reproduced by the SCC groups. We additionally handle the proceedings for papers and workshops in cooperation with IEEE.

Our first function is making certain that the entire course of round and between these parts is nicely coordinated. This implies not solely offering a wonderful scientific occasion to attendees, but additionally making certain that in SC, individuals can navigate from one ingredient to a different in a stress-free setting.

Our second function is to supply a world scientific imaginative and prescient and oversight over the Technical Program of SC.

For all of those parts, we’re supported by the respective ingredient chairs and vice-chairs. With out their arduous work and dedication to SC, it might not be potential to host an occasion of the scale and scope of the SC Technical Program.

Q: What do you imply by offering a “scientific imaginative and prescient?”

TP: Main the Technical Program of SC is a chance to have an effect on our subject, and on the science that we wish to see. We consider that on this function we’ve a accountability to the neighborhood to enhance our scientific course of.

For SC24, there are three parts that we wish to spotlight:

• Offering a key venue the place attendees have entry to the variety of the science that’s executed round HPC. This contains subjects the place HPC is the secondary scientific ingredient or an enabling expertise, and has traditionally not been submitted to SC.
• Ensuring that by doing this, we don’t degrade the standard of the technical program (i.e. we would like the highest paper from these subjects).
• Lastly, an necessary ingredient is one of the best practices that our neighborhood ought to subscribe to, and SC’s management function in establishing these.

Q: Are you able to give us examples of what you imply?

TP: To additional diversify the SC Technical Program, we augmented two parts to draw extra submissions that won’t have in any other case felt welcome at SC:

1. Workshops present a singular alternative to handle particular scientific subjects with early concepts that might not be mature but, however that may be brainstormed with a set of colleagues. As well as, there is a chance to debate subjects which are attention-grabbing to the SC neighborhood, however the place the primary outcomes have a tendency to not be submitted at SC (i.e. outcomes that might be revealed in excessive rating conferences from different scientific domains). For these subjects the place SC might not be their main venue for publication, we’ve tried to encourage proceedings-free symposiums, i.e. specialised occasions the place submissions are the newest key outcomes of their subject, even when they’re already revealed in different venues.
2. Posters are one other change that we’ve carried out. Posters often visually signify a brand new outcome or a piece in progress. We now enable a small portion of “Challenge Posters”. With these “undertaking” posters, we’re providing the chance to presenters to share a piece or a sequence of works which have already been revealed (outdoors of SC conferences, for instance), however that they consider would curiosity the SC neighborhood.

In each instances, the concept is to take away the “publication” ingredient, in order that we will truly give attention to the scientific contribution and foster dialogue inside the SC neighborhood.

With respect to greatest practices and enhancing the standard of the scientific contributions, the adjustments that we’ve made embody:

1. Reproducibility reviews. The reproducibility of HPC outcomes is an advanced drawback. For a couple of years now we’ve been awarding “reproducibility badges” to papers, primarily based on what a devoted committee was in a position to do. However these badges are very binary. As an illustration, what occurs if you happen to would not have the total scale to breed a outcome, however had been in a position to reproduce it on the scale you’ve gotten entry to? Ought to this paper get a “outcome replicated” badge? To enhance this course of, we’ve launched a “reproducibility report”, which is able to describe precisely what outcome was reproduced and the way. It’s going to additionally present recognition to committee members. On a facet word, we’re extraordinarily grateful to Sascha Hunold (Reproducibility Chair) who has executed an unbelievable job at implementing this and who in all probability wasn’t anticipating this large workload.
2. A much less seen procedural change is that we’ve requested each ingredient to remind submitters of the definition of authorship (as outlined by IEEE). Precisely representing authors is a part of tutorial integrity. In a time the place the general public’s belief in Science/Academia is reducing, it’s our accountability to maintain our ethics and trustworthiness to the very best requirements.

Q: Any sudden challenges with this 12 months’s Technical Program?

TP: An attention-grabbing drawback that we’ve needed to cope with this 12 months is the utilization of Giant Language Fashions (LLM) in paper (and evaluate) writing. IEEE and ACM have began to draft insurance policies, however it’s nonetheless unclear how authors use them. That is one thing new that’s evolving shortly.

For Tech Papers and Workshops, we’ve added a subject for authors to explain if and the way they’ve used LLM in writing their work (per IEEE coverage it must also be included into the accepted paper).

For SC24, we plan to review how these instruments are included into the writing course of, and it ought to be studied over time.

One other problem that we’ve confronted is the rise in variety of submissions in Tech Papers (+30%!). This has apparent implications by way of volunteers (you’ll be able to apply right here for SC25) wanted to maintain the standard of evaluate and choice that SC strives for. It additionally has implications within the variety of papers that we can settle for. Fortunately, the conference middle in Atlanta is kind of giant and we should always have the ability to match extra paper periods if wanted. It additionally has many difficult implications: has the neighborhood elevated and will we count on a a lot bigger attendance in November (with direct implications on room sizes)?

Q: What weren’t you in a position to change? What’s subsequent for the neighborhood?

TP: Now we have thought of having open critiques for a very long time. Open evaluate is the method the place the scientific dialogue between (nameless) professional reviewers and paper authors is made public to the readers. This tactic is rising in lots of scientific communities, like NeurIPS, one of many main venues for ML, which has been implementing this for some years. Nature has additionally began to publish some evaluate reviews since 2020.

Open critiques deliver additional details about a paper. They might assist to know what the bounds of a contribution are, which we consider are an necessary a part of the scientific course of. With the present unhappy state of scientific publishing (see right here for instance), the standard and availability of the evaluate are a wonderful indicator of the standard of the convention and evaluate course of.

It has additionally facet advantages – for instance,  it reveals new reviewers what is predicted of them when reviewing an SC paper, and helps new submitters to see how a piece is evaluated.

We had hoped to implement it this 12 months, however couldn’t. One thing left to do for our successors!