Systems Theory

Field

A field is a set \( \mathbb{F} \) with two binary operators, addition \((+)\) and multiplication \((\cdot)\), defined on the set \(S\). The binary operators satisfy the following axioms:

Additive associativity \[(\alpha + \beta) + \gamma = \alpha + (\beta + \gamma)\] \[(\alpha \cdot \beta) \cdot \gamma = \alpha \cdot (\beta \cdot \gamma)\]
Commutativity \[\alpha + \beta = \beta + \alpha\] \[\alpha \cdot \beta = \beta \cdot \alpha\]
Additive identity: \(\exists\) a distinct element \(0\in S\) s.t. \[\alpha + 0 = \alpha = 0 + \alpha\]
Multiplicative identity: \(\exists\) a distinct element \(1\in S\) s.t. \[\alpha \cdot 1 = \alpha = \alpha\]
Additive inverse \[\forall \alpha \in S, \exists (-\alpha) \text{ s.t. } \alpha + (-\alpha) = 0\]
Distributivity \[\alpha \cdot (\beta + \gamma) = (\alpha \cdot \beta) + (\alpha \cdot \gamma)\]
Multiplicative inverse\[\forall \alpha \neq 0 \in S, \exists \alpha^{-1} \text{ s.t. } \alpha \cdot \alpha^{-1} = 1\]

Examples

Real numbers \(\mathbb{R}\)
Complex numbers \(\mathbb{C}\)
Rational numbers \(\mathbb{Q}\)
Integers \(\mathbb{Z}\) modulo a prime a number \(p\): \(\mathbb{Z}/p\).
For modular arithmetic fields, \( p \) must be prime because if we choose modulus \( n = pq \) where \( p,q \) are prime, the inverse of \( p \) cannot exist:

\[ \begin{aligned} \text{Assume } & \exists m \text{ such that } p \cdot m = 1 \mod n \\ \Rightarrow & p \cdot m \cdot q = q \mod n \\ \text{But } & p \cdot q = 0 \mod n \\ \Rightarrow & 0 = q \mod n \quad \text{(Contradiction)} \end{aligned} \]

Vector space

Vector Space

Vector space over a field \(\mathbb{F}\) is a non-empty set \(V\) with a vector addition \((+)\) and a scalar multiplication \((\cdot)\):

Vector addition (+): \( V \times V \to V \)
Scalar multiplication (·): \( \mathbb{F} \times V \to V \)

Vector addition and scalar multiplication satisfy the following axioms:

Associativity: \[ (x + y) + z = x + (y + z) \]
Commutativity: \[ x + y = y + x \]
Identity: \( \exists 0 \in V \) such that \[ x + 0 = 0 + x = x \]
Inverse: \( \forall x \in V, \exists (-x) \in V \) such that \[ x + (-x) = 0 \]
Distributivity of scalar multiplication over vector addition: \[ \alpha \cdot (x + y) = \alpha \cdot x + \alpha \cdot y \]
Distributivity of scalar multiplication over field addition: \[ (\alpha + \beta) \cdot x = \alpha \cdot x + \beta \cdot x \]
Compatibility of scalar multiplication: \[ (\alpha \times \beta) \cdot v = \alpha \cdot (\beta \cdot v) \]
Identity of scalar multiplication: \[ 1 \cdot v = v \] where \( 1 \) is the multiplicative identity in \( F \)

Examples

\( \mathbb{R}^n \) over field \( \mathbb{R} \)
Space of polynomials of degree \( \leq n \): \( P_n \) over \( \mathbb{R} \)
Space of continuous functions \( C^0([1,2]) \) over field \(\mathbb{R}\) with operations: \[ (f_1 + f_2)(t) \triangleq f_1(t) + f_2(t) \] \[ (\alpha f_1)(t) \triangleq \alpha \cdot f_1(t) \]

Linearly independent set of vectors

A set of vectors \( M = \{x_1, \ldots, x_p\} \) are linearly independent if:

\[ \sum_{i=1}^p \alpha_i x_i = 0 \Rightarrow \alpha_i = 0 \quad \forall i \]

Span of a set of vectors

For \( M = \{x_1, \ldots, x_p\} \), the span is:

\[ \text{span}(M) = \left\{ z \in V : z = \sum_{i=1}^p \alpha_i x_i, \alpha_i \in \mathbb{F} \right\} \]

Basis of a vector space

A set \( M = \{x_1, \ldots, x_p\} \) is a basis for the vector space \( V \) if:

\( M \) is linearly independent
Every \( z \in V \) can be written as \( z = \sum_{i=1}^p \alpha_i x_i \)

Dimension of a vector space

The dimension of a vector space is the number of elements in its basis.

Isomorphism

Two vector spaces \( V_{\mathbb{F}} \) and \( W_{\mathbb{F}} \) are isomorphic if there exists a bijective mapping \(V_\mathbb{F} \mapsto W_\mathbb{F}\) that preserves vector addition and scalar multiplication: \[ \alpha_1 x_1 + \alpha_2 x_2 \mapsto \alpha_1 y_1 + \alpha_2 y_2 \]

Example

Set of functions and their fourier transforms form an isomorphism.

Subspace

A subset \( S \subseteq V \) is called a subspace if for every \( x,y \in S \) and \( \alpha, \beta \in F \):

\[ \alpha x + \beta y \in S \]

A subspace of a vector space is also a vector space.

Dual space

Linear functional

A linear functional \(\phi\) on a vector space \(V\) is a mapping \(\phi:V\rightarrow \mathbb{F}\) that satisfies: \[ \phi(\alpha v_1+\beta v_2) = \alpha \phi(v_1) + \beta \phi(v_2) \] Here, \(\alpha,\beta\in\mathbb{F}\), and \(v_1,v_2\in V\)

Dual space

Dual space \(V^\prime\) of a vector space \(V\) is the set of all linear functionals on \(V\).

Example

For the vector space \(\mathbb{R}^3\) a linear functional is given by: \[ y(x) = (1\ 2\ 3) \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = x_1 + 2x_2 + 3x_3 \]
For the vector space of continuous functions (\( C^0[1,2] \)), the integral operator is a linear functional. \[ \psi(f) = \int_1^2 f(s) ds \]
Evaluation of a function at a point is a linear functional: \[ \psi(f) = c f(1.5) \quad (c \in F) \]
The derivative of a function evaluated at a point is a linear functional: \[ \psi(f) = \left.\frac{d}{ds}f(s)\right|_{s=1.5} \]

Dual vector space

Set of all linear functional forms a vector space.

Basis for dual space

Let \( \{x_1, \ldots, x_n\} \) be a basis for \( V \). A dual basis \( \{y_1, \ldots, y_n\} \) is given by:

\[ y_i(x_j) = \delta_{ij} = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases} \]

For any \( y \in V^\prime \):

\[ y(x) = \sum_{i=1}^n y(x_i) y_i(x) = \sum_{i=1}^n \alpha_i y_i(x) \]

Direct sum

For subspaces \(M, N \subset V\), direct sum \(V = M \oplus N\) means every \(x \in V\) can be written uniquely as:

\[ x = z + t \quad \text{where } z \in M, t \in N \]

Annihilators

For a subspace \( M \subseteq V \), its annihilator \( M^\circ \) is given by:

\[ M^\circ = \{ y \in V^\prime \mid y(x) = 0 \ \forall x \in M \} \]

Properties of Annihilators

\( M^\circ \) is a subspace of \( V^\prime \)
If \( y_1, y_2 \in M^\circ \), then \( \alpha_1 y_1 + \alpha_2 y_2 \in M^\circ \): \[ (\alpha_1 y_1 + \alpha_2 y_2)(x) = \alpha_1 y_1(x) + \alpha_2 y_2(x) = 0 \quad \forall x \in M \]

Example

For \( M = \text{span}\left\{ \begin{pmatrix}1\\2\\3\end{pmatrix} \right\} \subset \mathbb{R}^3\):

\[ M^\circ = \left\{ y \in (\mathbb{R}^3)^* \mid y_1 + 2y_2 + 3y_3 = 0 \right\} \]

A basis for \( M^\circ \) could be:

\[ \left\{ \begin{pmatrix}2\\-1\\0\end{pmatrix}, \begin{pmatrix}3\\0\\-1\end{pmatrix} \right\} \]

Thus \( \dim(M^\circ) = 2 \) (a 2D subspace of \( V^\prime \)).

Fundamental dimension theorem

For any subspace \( M \subseteq V \):

\[ \dim(M) + \dim(M^\circ) = \dim(V) \]

Scanned notes

These scanned lecture notes are from the course SC 625 (Systems Theory) at Indian Institute of Technology, Bombay. Use them at your own discretion. If you would like to help digitize these lecture notes, contact the editor.

systems theory notes_1.jpg

systems theory notes_2.jpg

systems theory notes_3.jpg

systems theory notes_4.jpg

systems theory notes_5.jpg

systems theory notes_6.jpg

systems theory notes_7.jpg

systems theory notes_8.jpg

systems theory notes_9.jpg

systems theory notes_10.jpg

systems theory notes_11.jpg

systems theory notes_12.jpg

systems theory notes_13.jpg

systems theory notes_14.jpg

systems theory notes_15.jpg

systems theory notes_16.jpg

systems theory notes_17.jpg

systems theory notes_18.jpg

systems theory notes_19.jpg

systems theory notes_20.jpg

systems theory notes_21.jpg

systems theory notes_22.jpg

systems theory notes_23.jpg

systems theory notes_24.jpg

systems theory notes_25.jpg

systems theory notes_26.jpg

systems theory notes_27.jpg

systems theory notes_28.jpg

systems theory notes_29.jpg

systems theory notes_30.jpg

systems theory notes_31.jpg

systems theory notes_32.jpg

systems theory notes_33.jpg

systems theory notes_34.jpg

systems theory notes_35.jpg

systems theory notes_36.jpg

systems theory notes_37.jpg

systems theory notes_38.jpg

systems theory notes_39.jpg

systems theory notes_40.jpg

systems theory notes_41.jpg

systems theory notes_42.jpg

systems theory notes_43.jpg

systems theory notes_44.jpg

systems theory notes_45.jpg

systems theory notes_46.jpg

systems theory notes_47.jpg

systems theory notes_48.jpg

systems theory notes_49.jpg

systems theory notes_50.jpg

systems theory notes_51.jpg

systems theory notes_52.jpg

systems theory notes_53.jpg

systems theory notes_54.jpg

systems theory notes_55.jpg

systems theory notes_56.jpg

systems theory notes_57.jpg

systems theory notes_58.jpg

systems theory notes_59.jpg

systems theory notes_60.jpg

systems theory notes_61.jpg

systems theory notes_62.jpg

systems theory notes_63.jpg

systems theory notes_64.jpg

systems theory notes_65.jpg

systems theory notes_66.jpg

systems theory notes_67.jpg

systems theory notes_68.jpg

systems theory notes_69.jpg

systems theory notes_70.jpg

systems theory notes_71.jpg

systems theory notes_72.jpg

systems theory notes_73.jpg

systems theory notes_74.jpg

systems theory notes_75.jpg

systems theory notes_76.jpg

systems theory notes_77.jpg

systems theory notes_78.jpg

systems theory notes_79.jpg

systems theory notes_80.jpg

systems theory notes_81.jpg

systems theory notes_82.jpg

systems theory notes_83.jpg

systems theory notes_84.jpg

systems theory notes_85.jpg

systems theory notes_86.jpg

systems theory notes_87.jpg

systems theory notes_88.jpg

systems theory notes_89.jpg

systems theory notes_90.jpg

systems theory notes_91.jpg

systems theory notes_92.jpg

systems theory notes_93.jpg

systems theory notes_94.jpg

systems theory notes_95.jpg

systems theory notes_96.jpg

systems theory notes_97.jpg

systems theory notes_98.jpg

systems theory notes_99.jpg

systems theory notes_100.jpg

systems theory notes_101.jpg

systems theory notes_102.jpg

systems theory notes_103.jpg

Author

Anurag Gupta is an M.S. graduate in Electrical and Computer Engineering from Cornell University. He also holds an M.Tech degree in Systems and Control Engineering and a B.Tech degree in Electrical Engineering from the Indian Institute of Technology, Bombay.

Comment

Past Comments

No comments yet. Be the first!