Delta-complete linear programming techniques for satisfiability and numerical optimisation

Abstract

In this thesis, I develop rigorous algorithms for solving linear satisfiability and optimi sation problems within a Satisfiability Modulo Theories (SMT) framework. Over the last 20 years, extensive applications for such algorithms have opened up in the formal veri fication of software and, more recently, of rectified linear-unit (ReLU) neural networks. These networks learn a function that is continuous and piecewise linear over the inputs. Many of the most important safety properties for a ReLU neural network, in addition to the linear segments of the network itself, can be encoded as conjunctions of linear inequalities. This suggests that developing better methods to solve satisfiability problems over conjunctions of linear inequalities (“linear feasibility problems”) should be a priority for the neural network verification community. The most popular, and in many cases the most efficient method for solving linear feasibility problems is the simplex method. There also exists a family of methods known as interior-point methods, which have been found in some cases to outperform the simplex method. In this thesis, two algorithms are developed for the rigorous δ-complete solution of linear feasibility problems. One is based on the simplex method, and the other is based on an interior-point method. (A solver is considered δ-complete if it is a sound and complete method for solving the δ-decision problem corresponding to an SMT decision problem. The δ-decision problem is the problem of deciding between unsatisfiability of the original SMT decision problem and satisfiability of the δ-relaxed problem, which is the problem in which all arithmetic constraints, expressed as a set of inequalities, have been “relaxed” by adding or subtracting δ on one side or the other, in such a way as to make them easier to satisfy.) Both algorithms were implemented in software, and the interior-point method was found to be slower by many orders of magnitude. As a result, the majority of this thesis focuses on the simplex-based method, implemented as dLinear4,1 which is my modified version of the delta-complete SMT solver dReal4.2 To maximise the efficiency of the implementation, the simplex algorithm was built on top of an inexact, floating-point solver. This solver is then enclosed in a loop that tries an increasing sequence of floating-point precisions. I prove that the algorithm terminates, as the floating-point solver will eventually identify the exact solution. Correctness (for the δ decision problem) is ensured by the rational checks that follow the call to the floating-point solver. The time savings of making the method δ-complete are realised as a reduction in the number of calls to the floating-point solver, when an infeasible basis is nevertheless found to yield a δ-satisfying primal solution. 1https://github.com/martinjos/dlinear4 2https://github.com/dreal/dreal4 iii My implementation dLinear4 was tested on a wide range of SMT- and LP-derived problem instances. It was found to beat leading SMT solvers such as z33 and cvc44 on most LP-derived instances, even when an exact solution was sought (δ = 0). Some additional time savings have been found on larger instances when δ > 0. Finally, a new concept of δ-completeness for full linear programs is developed. This concept demands rigorous bounds confining the optimal objective function value (where applicable) to a range no larger than δ. An algorithm is defined, proven δ-complete according to this new concept (building upon the proof for the feasibility case), and implemented in dLinear4.