Call for Papers

Over the last decade, deep learning has brought about astonishing improvements in computer vision, signal processing, language modeling and beyond. The complexity of real high-dimensional data and deep network architectures used in practice stands as a challenge to traditional mathematical analysis, which is limited to simplified, tractable models that can be analyzed rigorously from the bottom up. Hence, many aspects remain mysterious and our understanding of the success and failure modes of deep learning remains very limited.

This workshop aims to promote a complementary approach to further our understanding of deep learning, through the lens of the scientific method. This approach uses carefully designed experiments in order to answer precise questions about how and why deep learning works. Such results can then be used to ground theoretical models in empirical observations or formulate conjectures, but also can inform engineering decisions and spur new research directions.

Topics

We invite researchers from machine learning and related fields to submit their latest work on the science of deep learning to the workshop. Accepted papers will be presented as posters during the poster sessions. Selected works will also be highlighted as contributed talks.

We encourage submissions that further our understanding of deep learning using the scientific method. Works that are a good fit for the workshop use empirical experiments on real-world datasets in order to:

  • validate or falsify hypotheses about the inner workings of deep networks,
  • make observations to inform or inspire theoretical models,
  • evidence new phenomena or empirical regularities (e.g., scaling laws).

We invite studies that employ the scientific method of investigation in any field of application, including but not limited to:

  • in-context learning in transformers,
  • generalization properties of generative models,
  • inductive biases of learning algorithms,
  • (mechanistic) interpretability,
  • empirical studies of loss landscapes, training dynamics, and learned weights and representations.

We explicitly welcome submissions that fall outside standard acceptance criteria, such as improving state-of-the-art performance or proving rigorous theorems, yet have a high impact potential by shedding light on deep network mechanisms.

Challenge

Authors can opt-in to be included to our Debunking Challenge, a competition aiming to interrogate commonly-held beliefs in the deep learning community. Authors may add an additional page to their submission to submit to the challenge. Prizes will be awarded to the top submission. For full details and submission criteria please visit our Challenge page.

Important Dates

  • Submission Deadline: Sep 17 ‘24 (Anywhere on Earth)
  • Acceptance Notification: Oct 9 ‘24 (Anywhere on Earth)

Submission Details

To ensure your submission is considered, please adhere to the following guidelines:

  • Formatting Instructions: Submissions are limited to 4 pages, with unlimited additional pages for references and appendices. Please use this LaTeX style files template.
  • Reviews: The review process will be double-blind. All submissions must be anonymized. Submissions that breach anonymity will be desk-rejected.
  • Dual Submission Policy: If the submission was accepted at prior conferences, journal, or workshops (including Neurips 2024), it should be extended and include new results to be considered for acceptance. Papers that are currently under review are welcome to be submitted.
  • There will be no proceedings for the venue. We will post the list of accepted papers on the workshop website. In addition, accepted submissions will be made public on OpenReview.
  • Submissions will be reviewed on OpenReview: submission page.
  • To apply for financial assistance fill out this form.

Questions

If you have any questions, please do not hesitate to contact us at scienceofdl.workshop.2024@gmail.com.