Regular Inverse & Pseudo Inverse Matrix Calculation using Singular Value Decomposition

written by Antonius Ringlayer

https://ringlayer.github.io http://www.ringlayer.com

  • In order to calculate inverse kinematic solutions for robotic arm, we can use several methods. One of my favorite method is using the pseudo inverse of a jacobian matrix.

So what is pseudo inverse of a matrix and how it differs from the regular inverse of a matrix ?

Regular Inverse of a Square Matrix

Inversion a Matrix means a process to create a reciprocal version of a Matrix.

In math, the reciprocal of a number means inverse of a number, for example, we have a number called b, inverse of b number can be written as b-1 . b-1 comes from: 1/b (reciprocal of b). Meanwhile for a matrix, e.g matrix A, inverse of A matrix can be written as A-1

In order to be inverted, a matrix must meet 2 conditions :

  1. the matrix has the same number of rows and columns (square matrix)
  2. determinant of the matrix is not zero.

The Pseudo Inverse of a Matrix

The Pseudo inverse matrix is symbolized as A dagger.

However, sometimes there are some matrices that do not meet those 2 requirements, thus cannot be inverted. Where the condition is overdetermined, we can use a method called pseudo inversing to create a pseudo inverse matrix version of our original matrix.

We are going to use SVD (Singular Value Decomposition) Trick to decompose a non square matrix which doesn’t have eigenvalue and eigenvector.

Inversing an Invertible Square Matrix

Inversing a square matrix is easy. For example, We have A matrix:

1

In order to get inverse of A, we are going to use adjoint method :

2

Inverse of A can be acquired from these steps:

  • Step 1. calculating determinant of A
  • Step 2. find cofactor of A
  • Step 3. find adjoint of A
  • Step 4. find the inverse

Based on https://ringlayer.wordpress.com/2018/07/03/matrix-determinant-laplace-and-sarrus-method , we found determinant of our matrix = -6.

So we can go straight ahead to step 2 : “find cofactor of A”.

First of all, we need to get minor of A.  Minor of A can be obtained by getting 2×2 possible matrix determinants of A :

Visually, we can do the trick a lot easier:

3.jpg

Next:

5.jpg

hence we got :

|a| = 2 ,  |b| = 1 , |c| = -3, |d| = 2, |e| = -1, |f| = -3, |g| = -1, |h| = -2 , |i|= -3

Thus we got minor matrix:

mino3.jpg

Cofactor matrix can be obtained easily:

proc.jpg

Hence we got cofactor matrix:

cofactor.jpg

Step 3, find adjoint of A :

Adjoint can be obtained by transforming cofactor matrix columns into rows, Adjoint :

final_adjoint

Finally based on our formula:

6.jpg

A-1 =  1/-6 *  adjoint matrix, Hence we got our inverse matrix :

inverse

Pseudo Inverse Matrix using SVD

Sometimes, we found a matrix that doesn’t meet our previous requirements (doesn’t have exact inverse), such matrix doesn’t have eigenvector and eigenvalue. In order to find pseudo inverse matrix, we are going to use SVD (Singular Value Decomposition) method.

For Example,  Pseudo inverse of matrix A is symbolized as A+

When the matrix is a square matrix :

A+ = A-1

when the matrix is overdetermined :

A+ = -1 Ut

V, Σ and U are matrices from SVD of A.

Where SVD of A :

UΣVt

can be notated :

UsVt

or

USVt

or

UDVt

 

Example of Pseudo Inverse Calculation using SVD Method

Graphical-depiction-of-SVD-of-a-matrix-X-annotated-with-notations-adopted-in-this.png

For example, We have a non squared matrix called “A”  as follow :

svd1.jpg

 

U, s, VT of this matrix can be acquired manually or automatically using this simple python numpy script:

#!/usr/bin/env python3
import numpy as np
# example taken from Video Tutorials – All in One
# https://www.youtube.com/watch?v=P5mlg91as1c
a = np.array([[1, 2, 1],[2, 1, -1]])
np.set_printoptions(suppress=True)
np.set_printoptions(precision=3)
U, s, VT = np.linalg.svd(a, full_matrices=True)
print (“U:\n {}”.format(U))
print (“s:\n {}”.format(s))
print (“VT:\n {}”.format(VT))

svd8

or we can calculate SVD manually using this following steps:

Step 1. Finding singular vector and singular value of U & V

in order to find singular vector and singular value of U & V, we have to find eigenvector and eigenvalue of kU & kV, where :

kU = A . At

kV = At . A

svd2.jpg

= svd3

Finding eigenvalue and eigenvector of kU:

| A – λ I | = 0

λ = scalar (eigenvalue corresponding to u)

I = identity matrix

svd6.jpg

svd7

So :

(6 – λ)2 – 25 = 0

we got 2 eigenvalue :

λ1 = 11

and

λ2 = 1

next, we are going to find eigenvector of kU.

eigenvector of kU = x

(A – λ.I) * x = 0

svd9

for λ = 11 :

svd10

(-5 * x1) + (5 * x2) = 0

hence:

x1 = x2

x1 = -1

x2 = -1

for λ = 1 :

svd11.jpg

(5 * x1 ) + (5 * x2) = 0

hence:

x1 – x2 = 0

x1 = -1

x2 = 1

Finally we have collected enough informations for U:

λ1 = 11 -> x1 = -1 and x2 = -1

λ2 = 1 -> x1 = -1 and x2 = 1

Du =  svd12

u =  svd13.jpg

in order to get U we need to normalize u (orthonormal) using this following steps:

We have our u matrix:

svd16.jpg

to normalize u :

svd15

 

hence:

svd17

we got:

svd18

2 power 1/2 = square root of 2.

svd22.jpg

we got:

svd23.jpg

using the same method to normalize, another column, we got:

svd24.jpg

We got our normalize matrix:

svd25

so U =

svd26

in order to find V, we can do the same steps just like U, we got :

dv1.jpg

 

V

By transposing matrix V, we got:

V1.jpg

 

Step 2. Find D.

We can acquire D matrix by multiplying Du with Dv:

D = Du * Dv

= svd12dvv.jpg

 

V3.jpg

 

Finally we have collected enough informations to calculate pseudo inverse matrix using SVD method, we got this informations:

U = svd26

V

V3.jpg

Based on  our pseudo inverse matrix formula:

A+ = VΣ-1 Ut

or

A+ = VD-1 Ut

We can got our pseudo inverse matrix easily:

dvv1.jpg* Ut

 

We got our Pseudo inverse matrix (A dagger):

dvv3.jpg

 

Advertisements

Calculus 3 – First Order Partial Derivatives

Recently, I encountered a matrix called jacobian matrix. Jacobian matrix is a matrix that consists of first order partial derivatives of vector value function. Pseudo Inverse of jacobian matrix can be used to solve inverse kinematic problem in robotic field. So what is partial derivatives ? Partial derivative  symbolized as (partial dee). Partial Derivative is  a part of calculus. Based on literature :

“a derivative of a function of two or more variables with respect to one variable, the other(s) being treated as constant.”

example function:

f(x,y) = y³x + 4x + 5y

∂f/∂x  means partial derivative of f(x,y) in respect to x. where we treat y as constant.

∂f/∂x = y³ + 4

steps:

  • partial derivative (∂f/∂x) of y³x = y³x / x  = y³
  • partial derivative  (∂f/∂x) of 4x = 4x  / x = 4
  • partial derivative (∂f/∂x) of constant 5y = 0
  • so ∂f/∂x = y³ + 4

∂f/∂y  means partial derivative of f(x,y) in respect to y. where we treat x as constant.

f(x,y) = y³x + 4x + 5y

∂f/∂y = 2yx + 5

steps :

  • partial derivative (∂f/∂y) of y³x = 2yx
  • partial derivative  (∂f/∂y) of contsant 4x = 0
  • partial derivative (∂f/∂y) of  5y = 5y / y = 5
  • so ∂f/∂y = 2yx + 5

Another example, partial derivative of this explicit function :

y = 3x2 – 5z2 + 2x2z – 4xz2 – 9

∂y/∂x = ?

steps :

  • ∂y/∂x of 3x2  = 2.3x2-1 = 6x
  • ∂y/∂x of 5z2 = 0
  • ∂y/∂x of 2x2z  = 2.2x2-1z = 4xz
  • ∂y/∂x of 4xz2 = 4z2
  • ∂y/∂x of 9 = 0
  • So ∂y/∂x of y = 6x + 4xz- 4z2

what about ∂y/∂z ?

steps:

  • ∂y/∂z of 3x2  = 0
  • ∂y/∂z of 5z2 = 2.5z2-1 = 10z
  • ∂y/∂z of 2x2z  =  2x2z * 1/z = 2x2
  • ∂y/∂z of 4xz2 = 2. 4xz2-1 =  8xz
  • ∂y/∂z of 9 = 0
  • So ∂y/∂z of y = -10z + 2x2 – 8xz

First Order Partial Derivative of a function that consists of 3 variables ?

Example : f(x,y,z) = z4 − x6y2 + 5

f’x = ?

steps :

  • ∂f/∂x of z4 = 0
  • ∂f/∂x of x6y2 = 6x6-1y2   = 6x5y2
  • ∂f/∂x of 5 = 0
  • thus ∂f/∂x of  f(x,y,z) = -6x5y2

f’y = ?

  • ∂f/∂y of z4 = 0
  • ∂f/∂y of x6y2 = x6 2y2-1   = x6 2y
  • ∂f/∂y of 5 = 0
  • thus ∂f/∂y of  f(x,y,z)  = -x6 2y

f’z = ?

  • ∂f/∂z of z4 = 4z4-1 = 4z3
  • ∂f/∂z of x6y2 = 0
  • ∂f/∂z of 5 = 0
  • thus ∂f/∂z of  f(x,y,z)  =  4z3

 

www.ringlayer.com

https://ringlayer.github.io