Papers

Learn more about AI2's Lasting Impact Award
Viewing 1-10 of 844 papers
  • Data-Efficient Finetuning Using Cross-Task Nearest Neighbors

    Hamish Ivison, Noah A. Smith, Hannaneh Hajishirzi, Pradeep DasigiACL Findings2023 Language models trained on massive prompted multitask datasets like T0 (Sanh et al., 2021) or FLAN (Wei et al., 2021a) can generalize to tasks unseen during training. We show that training on a carefully chosen subset of instances can outperform training on…
  • HINT: Hypernetwork Instruction Tuning for Efficient Few- and Zero-Shot Generalisation

    Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. PetersACL2023 Recent NLP models have shown the remarkable ability to effectively generalise `zero-shot' to new tasks using only natural language instructions as guidance. However, many of these approaches suffer from high computational costs due to their reliance on…
  • Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation

    Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, D. Klakow, Yanai ElazarFindings of ACL 20232023 Few-shot fine-tuning and in-context learning are two alternative strategies for task adaptation of pre-trained language models. Recently, in-context learning has gained popularity over fine-tuning due to its simplicity and improved out-of-domain…
  • Improving the reliability of ML-corrected climate models with novelty detection.

    Clayton Sanford, Anna Kwa, Oliver Watt-Meyer, Spencer K. Clark, Noah D. Brenowitz, Jeremy McGibbon, and Christopher S. BrethertonESSOAr2023 The use of machine learning (ML) for the online correction of coarse-resolution atmospheric models has proven effective in reducing biases in near-surface temperature and precipitation rate. However, this often introduces biases in the upper atmosphere and…
  • Improving Language Model Negotiation with Self-Play and In-Context Learning from AI Feedback

    Yao Fu, Hao-Chun Peng, Tushar Khot, Mirella LapataarXiv.org2023 We study whether multiple large language models (LLMs) can autonomously improve each other in a negotiation game by playing, reflecting, and criticizing. We are interested in this question because if LLMs were able to improve each other, it would imply the…
  • Pace v0.2: a Python-based performance-portable atmospheric model

    Johann Dahm, Eddie Davis, Florian Deconinck, Oliver Elbert, Rhea George, Jeremy McGibbon, Tobias Wicky,Elynn Wu,Christopher Kung, Tal Ben-Nun, Lucas Harris , Linus Groner, Oliver FuhrerGeoscientific Model Development2023 Progress in leveraging current and emerging high-performance computing infrastructures using traditional weather and climate models has been slow. This has become known more broadly as the software productivity gap. With the end of Moore's law driving forward…
  • From Centralized to Ad-Hoc Knowledge Base Construction for Hypotheses Generation.

    Shaked Launer-Wachs, Hillel Taub-Tabib, Jennie Tokarev Madem, Orr Bar-Natan, Yoav Goldberg, Y. ShamayJournal of Biomedical Informatics2023 Objective To demonstrate and develop an approach enabling individual researchers or small teams to create their own ad-hoc, lightweight knowledge bases tailored for specialized scientific interests, using text-mining over scientific literature, and…
  • Machine-Learned Climate Model Corrections From a Global Storm-Resolving Model: Performance Across the Annual Cycle

    Anna Kwa, Spencer K. Clark, Brian Henn, Noah D. Brenowitz, Jeremy McGibbon, Oliver Watt-Meyer, W. Andre Perkins, Lucas Harris, Christopher S. BrethertonJournal of Advances in Modeling Earth Systems2023 One approach to improving the accuracy of a coarse-grid global climate model is to add machine-learned (ML) state-dependent corrections to the prognosed model tendencies, such that the climate model evolves more like a reference fine-grid global storm…
  • Embedding Recycling for Language Models

    Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug DowneyFindings of EACL2023 Training and inference with large neural models is expensive. However, for many application domains, while new tasks and models arise frequently, the underlying doc-uments being modeled remain mostly un-changed. We study how to decrease computational cost in…
  • Complexity-Based Prompting for Multi-Step Reasoning

    Yao Fu, Hao-Chun Peng, Ashish Sabharwal, Peter Clark, Tushar KhotICLR2023 We study the task of prompting large-scale language models to perform multi-step reasoning. Existing work shows that when prompted with a chain of thoughts (CoT), sequences of short sentences describing intermediate reasoning steps towards a final answer…