REGAL: Extracting implicit rules in text using LLMs with logic program feedback
Solving textual reasoning problems by translating them into logic has proven effective, as it reduces hallucinations and allows the logic solver to handle complex reasoning. However, there is one major challenge that makes this technique difficult to apply to many practical reasoning problems. Arguments presented in text often have implicit rules that are assumed to be common sense and are therefore omitted. They need to be identified and explicitly added to a logic program for accurate reasoning. This process is typically called Argument Reconstruction. Discovering these implicit logic rules is a challenging problem that previous text-to-logic translation systems struggle with. In this paper, we present a novel system that reconstructs these implicit rules in 3 stages: 1) Translating the problem from text to First Order Logic (FOL), 2) Translating FOL to an equivalent s(CASP) answer set program that can compute gap predicates (predicates whose derivation requires implicit rules), and iii) Using an LLM to generate required implicit rules for these gap predicates. We show that our system identifies implicit rules and correctly solves reasoning problem samples drawn from popular benchmarks designed to be challenging for LLMs.
Mon 12 JanDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
16:00 - 17:30 | PADL M4PADL at Salle 12 Chair(s): Nada Amin Harvard University, Joaquín Arias Universidad Rey Juan Carlos | ||
16:00 30mTalk | Declarative Debugging for Modern Networks PADL | ||
16:30 30mTalk | Interpretable Configuration Optimization for Static Program Verification via Rule-Based and Counterfactual Reasoning PADL Jaeseong Lee The University of Texas at Dallas, Sopam Dasgupta The University of Texas at Dallas, Gopal Gupta The University of Texas at Dallas, Shiyi Wei University of Texas at Dallas | ||
17:00 30mTalk | REGAL: Extracting implicit rules in text using LLMs with logic program feedback PADL Abhiramon Rajasekharan The University of Texas at Dallas, Gopal Gupta The University of Texas at Dallas | ||