lec28.mws

L-28 MCS 320 Monday 31 October 2005

> restart;

> with(LinearAlgebra);

[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...[&x, Add, Adjoint, BackwardSubstitute, BandMatrix, Basis, BezoutMatrix, BidiagonalForm, BilinearForm, CharacteristicMatrix, CharacteristicPolynomial, Column, ColumnDimension, ColumnOperation, ColumnSp...

LinearAlgebra is a very extensive package.  Compared to MATLAB, Maple has the advantage of allowing symbolic operations on vectors and matrices.

1. Linear Equations and Matrices

> A := RandomMatrix(3,3);

A := Matrix([[27, 99, 92], [8, 29, -31], [69, 44, 67]])

> b := convert(RandomMatrix(3,1),Vector);

b := Vector[column]([[-22], [45], [-81]])

We will define the linear system A*x = b.  First we pick variables:

> X := [seq(x[i],i=1..3)];

X := [x[1], x[2], x[3]]

> eqs := GenerateEquations(A,X,b);

eqs := [27*x[1]+99*x[2]+92*x[3] = -22, 8*x[1]+29*x[2]-31*x[3] = 45, 69*x[1]+44*x[2]+67*x[3] = -81]

This is the symbolic view of a linear system.  To work with a linear system, we typically work with the coefficient matrix A and the right hand side vector b.

> AA,bb := GenerateMatrix(eqs,X);

AA, bb := Matrix([[27, 99, 92], [8, 29, -31], [69, 44, 67]]), Vector[column]([[-22], [45], [-81]])

> AA; bb;

Matrix([[27, 99, 92], [8, 29, -31], [69, 44, 67]])

Vector[column]([[-22], [45], [-81]])

2. Vector and Matrix Arithmethic

We noticed already the distinction between the type Vector and the type Matrix:

> whattype(bb); whattype(AA);

Vector[column]

Matrix

> mb := convert(bb,Matrix); whattype(mb);

mb := Matrix([[-22], [45], [-81]])

Matrix

But still, we cannot add AA to mb:

> AA+mb;

Error, (in rtable/Sum) invalid arguments

> MatrixAdd(AA,mb);

Error, (in LinearAlgebra:-MatrixAdd) matrix dimensions don't match: 3 x 3 vs 3 x 1

There are two errors, first the overloading of the "+" did not work.  The MatrixAdd complains about the wrong dimensions.  

> B := RandomMatrix(3);

B := Matrix([[25, -2, -16], [94, 50, -9], [12, 10, -50]])

> C := MatrixAdd(A,B);

C := Matrix([[52, 97, 76], [102, 79, -40], [81, 54, 17]])

> MatrixMatrixMultiply(A,mb);

Matrix([[-3591], [3640], [-4965]])

> MatrixVectorMultiply(A,bb);

Vector[column]([[-3591], [3640], [-4965]])

Notice the dimensions.  If we multiply a 3-by-3 matrix with a 3-by-1 matrix, we obtain a 3-by-1 result, this is a matrix of 3 rows and 1 column.  Notice that a column vector is not automatically converted into a matrix.

3. Basic Matrix Functions

To determine whether a matrix is singular or not, to find there exists a unique solution to the linear system with coefficient matrix A, we compute the determinant:

> Determinant(A);

-327244

To solve the system A*x = b, we can apply LU decomposition:

> P,L,U := LUDecomposition(A);

P, L, U := Matrix([[1, 0, 0], [0, 1, 0], [0, 0, 1]]), Matrix([[1, 0, 0], [8/27, 1, 0], [23/9, 627, 1]]), Matrix([[27, 99, 92], [0, (-1)/3, (-1573)/27], [0, 0, 327244/9]])

> MatrixMatrixMultiply(L,U) - A;

Matrix([[0, 0, 0], [0, 0, 0], [0, 0, 0]])

With this LU decomposition, the system A*x = b becomes (L*U)*x = b, or L*(U*x) = b.  We can first solve the lower triangular system L*y = b, where y is auxiliary, and then solve U*x = y.

> y := ForwardSubstitute(L,b); # solve L*y = b

y := Vector[column]([[-22], [1391/27], [(-290942)/9]])

> sol := BackwardSubstitute(U,y); # solve U*x = y

sol := Vector[column]([[(-137809)/163622], [136409/163622], [(-145471)/163622]])

> MatrixVectorMultiply(A,sol); b; # verify if sol is a solution

Vector[column]([[-22], [45], [-81]])

Vector[column]([[-22], [45], [-81]])

> fA := convert(A,float);

fA := Matrix([[27., 99., 92.], [8., 29., -31.], [69., 44., 67.]])

> P,L,U := LUDecomposition(fA);

P, L, U := Matrix([[0, 1, 0], [0, 0, 1], [1, 0, 0]]), Matrix([[1.0, 0., 0.], [.391304347826086973, 1.0, 0.], [.115942028985507248, .292220450115186936, 1.0]]), Matrix([[69., 44., 67.], [0., 81.7826086...P, L, U := Matrix([[0, 1, 0], [0, 0, 1], [1, 0, 0]]), Matrix([[1.0, 0., 0.], [.391304347826086973, 1.0, 0.], [.115942028985507248, .292220450115186936, 1.0]]), Matrix([[69., 44., 67.], [0., 81.7826086...

With floating-point numbers on input, the result looks quite different.  Now, for numerical stability, Maple has permuted the first and third row.

> y := ForwardSubstitute(L,MatrixVectorMultiply(P,b));

y := Vector[column]([[45.00000000], [-98.60869565], [1.598086120]])

> nsol := BackwardSubstitute(U,y);

nsol := Vector[column]([[1.43367640043231992], [-1.18357555833348104], [-0.275574188530882148e-1]])

> MatrixVectorMultiply(A,nsol); b;

Vector[column]([[-80.9999999978261088], [-22.0000000037666582], [44.9999999999999858]])

Vector[column]([[-22], [45], [-81]])

Up to permutations, we get the solution...

4. Vector Operations

We can apply the Gram-Schmidt orthogonalization procedure:

> GramSchmidt(A);

Error, (in LinearAlgebra:-GramSchmidt) invalid input: LinearAlgebra:-GramSchmidt expects its 1st argument, V, to be of type {list(Vector), set(Vector)} but received Matrix(3, 3, [[...],[...],[...]], datatype = anything)

> v := seq(Column(A,i),i=1..ColumnDimension(A));

v := Vector[column]([[27], [8], [69]]), Vector[column]([[99], [29], [44]]), Vector[column]([[92], [-31], [67]])

> basis := GramSchmidt({v});

basis := {Vector[column]([[92], [-31], [67]]), Vector[column]([[35015108/1209299], [1485360516/30232475], [(-39596524)/2325575]]), Vector[column]([[(-127675)/6957], [323941/13914], [500513/13914]])}

> Matrix(3,3,(i,j) -> DotProduct(basis[i],basis[j]));

Matrix([[13914, 0, 0], [0, 107088635536/30232475, 0], [0, 0, 30232475/13914]])

We just verified that the three vectors returned by GramSchmidt are perpendicular to each other.

5. Smith Normal Form and the SVD