Regular Inverse & Pseudo Inverse Matrix Calculation using Singular Value Decomposition

written by Antonius Ringlayer

https://ringlayer.github.io http://www.ringlayer.com

  • In order to calculate inverse kinematic solutions for robotic arm, we can use several methods. One of my favorite method is using the pseudo inverse of a jacobian matrix.

So what is pseudo inverse of a matrix and how it differs from the regular inverse of a matrix ?

Regular Inverse of a Square Matrix

Inversion a Matrix means a process to create a reciprocal version of a Matrix.

In math, the reciprocal of a number means inverse of a number, for example, we have a number called b, inverse of b number can be written as b-1 . b-1 comes from: 1/b (reciprocal of b). Meanwhile for a matrix, e.g matrix A, inverse of A matrix can be written as A-1

In order to be inverted, a matrix must meet 2 conditions :

  1. the matrix has the same number of rows and columns (square matrix)
  2. determinant of the matrix is not zero.

The Pseudo Inverse of a Matrix

The Pseudo inverse matrix is symbolized as A dagger.

However, sometimes there are some matrices that do not meet those 2 requirements, thus cannot be inverted. Where the condition is overdetermined, we can use a method called pseudo inversing to create a pseudo inverse matrix version of our original matrix.

We are going to use SVD (Singular Value Decomposition) Trick to decompose a non square matrix which doesn’t have eigenvalue and eigenvector.

Inversing an Invertible Square Matrix

Inversing a square matrix is easy. For example, We have A matrix:

1

In order to get inverse of A, we are going to use adjoint method :

2

Inverse of A can be acquired from these steps:

  • Step 1. calculating determinant of A
  • Step 2. find cofactor of A
  • Step 3. find adjoint of A
  • Step 4. find the inverse

Based on https://ringlayer.wordpress.com/2018/07/03/matrix-determinant-laplace-and-sarrus-method , we found determinant of our matrix = -6.

So we can go straight ahead to step 2 : “find cofactor of A”.

First of all, we need to get minor of A.  Minor of A can be obtained by getting 2×2 possible matrix determinants of A :

Visually, we can do the trick a lot easier:

3.jpg

Next:

5.jpg

hence we got :

|a| = 2 ,  |b| = 1 , |c| = -3, |d| = 2, |e| = -1, |f| = -3, |g| = -1, |h| = -2 , |i|= -3

Thus we got minor matrix:

mino3.jpg

Cofactor matrix can be obtained easily:

proc.jpg

Hence we got cofactor matrix:

cofactor.jpg

Step 3, find adjoint of A :

Adjoint can be obtained by transforming cofactor matrix columns into rows, Adjoint :

final_adjoint

Finally based on our formula:

6.jpg

A-1 =  1/-6 *  adjoint matrix, Hence we got our inverse matrix :

inverse

Pseudo Inverse Matrix using SVD

Sometimes, we found a matrix that doesn’t meet our previous requirements (doesn’t have exact inverse), such matrix doesn’t have eigenvector and eigenvalue. In order to find pseudo inverse matrix, we are going to use SVD (Singular Value Decomposition) method.

For Example,  Pseudo inverse of matrix A is symbolized as A+

When the matrix is a square matrix :

A+ = A-1

when the matrix is overdetermined :

A+ = -1 Ut

V, Σ and U are matrices from SVD of A.

Where SVD of A :

UΣVt

can be notated :

UsVt

or

USVt

or

UDVt

 

Example of Pseudo Inverse Calculation using SVD Method

Graphical-depiction-of-SVD-of-a-matrix-X-annotated-with-notations-adopted-in-this.png

For example, We have a non squared matrix called “A”  as follow :

svd1.jpg

 

U, s, VT of this matrix can be acquired manually or automatically using this simple python numpy script:

#!/usr/bin/env python3
import numpy as np
# example taken from Video Tutorials – All in One
# https://www.youtube.com/watch?v=P5mlg91as1c
a = np.array([[1, 2, 1],[2, 1, -1]])
np.set_printoptions(suppress=True)
np.set_printoptions(precision=3)
U, s, VT = np.linalg.svd(a, full_matrices=True)
print (“U:\n {}”.format(U))
print (“s:\n {}”.format(s))
print (“VT:\n {}”.format(VT))

svd8

or we can calculate SVD manually using this following steps:

Step 1. Finding singular vector and singular value of U & V

in order to find singular vector and singular value of U & V, we have to find eigenvector and eigenvalue of kU & kV, where :

kU = A . At

kV = At . A

svd2.jpg

= svd3

Finding eigenvalue and eigenvector of kU:

| A – λ I | = 0

λ = scalar (eigenvalue corresponding to u)

I = identity matrix

svd6.jpg

svd7

So :

(6 – λ)2 – 25 = 0

we got 2 eigenvalue :

λ1 = 11

and

λ2 = 1

next, we are going to find eigenvector of kU.

eigenvector of kU = x

(A – λ.I) * x = 0

svd9

for λ = 11 :

svd10

(-5 * x1) + (5 * x2) = 0

hence:

x1 = x2

x1 = -1

x2 = -1

for λ = 1 :

svd11.jpg

(5 * x1 ) + (5 * x2) = 0

hence:

x1 – x2 = 0

x1 = -1

x2 = 1

Finally we have collected enough informations for U:

λ1 = 11 -> x1 = -1 and x2 = -1

λ2 = 1 -> x1 = -1 and x2 = 1

Du =  svd12

u =  svd13.jpg

in order to get U we need to normalize u (orthonormal) using this following steps:

We have our u matrix:

svd16.jpg

to normalize u :

svd15

 

hence:

svd17

we got:

svd18

2 power 1/2 = square root of 2.

svd22.jpg

we got:

svd23.jpg

using the same method to normalize, another column, we got:

svd24.jpg

We got our normalize matrix:

svd25

so U =

svd26

in order to find V, we can do the same steps just like U, we got :

dv1.jpg

 

V

By transposing matrix V, we got:

V1.jpg

 

Step 2. Find D.

We can acquire D matrix by multiplying Du with Dv:

D = Du * Dv

= svd12dvv.jpg

 

V3.jpg

 

Finally we have collected enough informations to calculate pseudo inverse matrix using SVD method, we got this informations:

U = svd26

V

V3.jpg

Based on  our pseudo inverse matrix formula:

A+ = VΣ-1 Ut

or

A+ = VD-1 Ut

We can got our pseudo inverse matrix easily:

dvv1.jpg* Ut

 

We got our Pseudo inverse matrix (A dagger):

dvv3.jpg

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s