Quadratic Forms, Positive Definite, Negative Definite, and Semi-Definite Matrices

In this post and in the accompanying YouTube tutorial, we explain the following topics:

1. Quadratic forms. The connection between quadratic forms and definite and semi-definite matrices.
2. Definite matrices. Tests for determining if a matrix is positive (semi) definite, negative (semi) definite, or indefinite.
3. Plotting of quadratic forms in MATLAB.
4. Leading principal minors of matrices.

Let us consider the following function

(1)  

where is a symmetric matrix, that is, . This function represents a quadratic form written in a compact matrix form. Function , where for several different forms of the matrix (that are given in Eq. (7)), is shown in Figs. 1-5 below. For example, consider this function:

(2)  

where and are real variables. This function is actually a quadratic form since

(3)  

where

(4)  

In order to better illustrate the structure of quadratic forms, let us consider a 2D case below

(5)  

and consequently, our original function takes the following form

(6)  

Let us visualize these functions, for several values of the matrix . Consider the following five matrices

(7)  

The function for these five cases is shown in Figs 1-5 below.

The MATLAB code used to generate these plots is given below.

clear,pack,clc
x1=-50:2:50;
x2=-50:2:50;

[X1,X2]=meshgrid(x1,x2);

syms x1s
syms x2s

% P1 - positive definite completely symmetric with respect to the zero axis
%P=[1 0; 0  1];
% P2 - positive definite not completely symmetric
% P=[1 -2;1  5];
% P3 - negative definite 
%P=[-1 -2;1  -5];
%P4 - indefinite 
%P=[-1 -2; 1  2];
%P5 - positive semi-definite
P=[1 1; 1 1];

expression= [x1s x2s]*P*[x1s; x2s];

% print this expression in the command window to see the expanded form of
% the Z function

% use the expanded expression to define a new function
expand(expression)

V= X1.^2+X2.^2+(X1.*X2).*2

figure(1)
surf(V)
xlabel('x1')
ylabel('x2')
zlabel('V(x1,x2)')

Here, we need to mention one important fact about quadratic forms. Notice that in the definition of the quadratic form, given by Eq. (1), it is assumed that the matrix is symmetric. However, many matrices are not symmetric. Is it possible to define quadratic forms for such matrices? The answer is yes, since

(8)  

Consequently, any non-symmetric matrix defines a quadratic form with a symmetric matrix. That is, for any non-symmetric matrix, we can find a quadratic form with a symmetric matrix.

From (6), we can see that the function is quadratic. For this function to be positive definite, the matrix has to be a positive definite matrix. The symmetric real matrix is said to be a positive definite matrix if and only if, the following condition is satisfied

(9)  

Positive-definite quadratic forms corresponding to the positive definite matrices and are illustrated in Figs. 1 and 2. Let us now discuss a test for testing positive definiteness of a matrix. Let us consider the 2D case given by Eq.(6). From this equation, we have:

(10)  

Let us analyze the last expression. Let us assume that is positive. Then, it is easy to see that the first term is positive for any and different from zero, that is

(11)  

Let us consider the second term

(12)  

The second term is positive for all if

(13)  

The last equation is equivalent to

(14)  

To summarize, the quadratic form (6) is positive definite if

(15)  

On the other hand, consider again the original matrix

(16)  

The number is the leading principal minor of order (or briefly, the first-order leading principal minor) of the matrix and the expression is the leading principal minor of order (or briefly, the second-order leading principal minor) of the matrix . The leading principal minors of certain order are determinants of the upper-left submatrices of the matrix. Generally speaking, we can say that the leading principal minor of order is the determinant of a matrix obtained by erasing the last columns and rows of the matrix . Thus, the second-order matrix is positive definite if and only if, its first-order and second-order leading principal minors are positive.

By using this test, we can determine that the matrices and defined in (7) are positive. Consequently, the corresponding quadratic forms that are shown in Fig. 1. and Fig. 2. are positive definite.

Condition for positive definiteness:
Generally speaking, the n-dimensional matrix :

(17)  

is positive definite if and only if

• the first-order leading principal minor is positive, that is, the determinant of the upper left 1-by-1 corner of P is positive:
  

  (18)  

• the second-order leading principal minor is positive, that is, the determinant of the upper left 2-by-2 corner of P is positive:
  

  (19)  

• the third-order leading principal minor is positive, that is, the determinant of the upper left 3-by-3 corner of P is positive:
  

  (20)  

• …
• the determinant of the matrix P is positive (this is the -th order leading principal minor)
  

  (21)  

This is the Sylvester condition for testing the positive definiteness of a matrix.

Let us apply this test on the following matrix

(22)  

We have

(23)  

Consequently, the matrix is positive-definite.

Let us now state definitions of negative definite, negative semidefinite, positive semidefinite, and indefinite matrices.

Negative definite matrix: A matrix is negative definite if and only if

(24)  

Negative semi-definite matrix: A matrix is negative definite if and only if

(25)  

Positive semi-definite matrix: A matrix is negative definite if and only if

(26)  

Indefinite matrix: A matrix is indefinite if and only if for some , and for some .

The matrix in (7) is negative definite and the matrix is indefinite. The quadratic form defined for the negative definite matrix is shown in Fig. 3, whereas the quadratic form defined for the indefinite matrix is shown in Fig. 4. The matrix in (7) is positive semi-definite. The quadratic form defined for this matrix is shown in Fig. 5. The quadratic form for the matrix is

(27)  

  We see that this quadratic form is positive semi-definite since for , the quadratic form is zero, and otherwise it is positive (except at ). That is, there is some point that is different from for which this quadratic form is zero. Next, we state tests for negative definiteness, semi-definiteness, and indefiniteness.

Condition for negative definiteness: A matrix in (17) is negative definite if and only if:

(28)  

That is, the matrix is negative definite if and only if its leading principal minors alternate the sign, starting from the first-order minor that should be negative. Or in other words, the matrix is negative definite if and only if its leading principal minors of odd degree are negative and leading principal minors of even degree are positive.

Condition for indefiniteness: A matrix is indefinite if and only if some leading principal minor is non-zero, but its sign does not follow the pattern for either positive definite or negative definite matrices.

The conditions for positive and negative semidefiniteness are different from the conditions for strict positive and negative definiteness since they involve all principal minors, and not only leading principal minors. Principal minors are simply obtained by erasing certain columns and rows of a matrix and computing the determinant. Here, the process of erasing the columns and rows is not only restricted to the last rows and columns such as in the case of leading principal minors. The column and row numbers should be identical. That is, the indices of deleted rows must be equal to the indices of deleted columns. For example, consider the 3×3 matrix :

(29)  

Its second order principal minors are

(30)  

Condition for positive semi-definiteness: A matrix is positive semi-definite if and only if its all principal minors are .

Condition for negative semi-definiteness: A matrix is negative semi-definite if and only if its principal minors of odd degree are , and principal minors of even degree are .