MATHEMATICAL METHODS FOR INFORMATION ENGINEERING

Metric Spaces. Completeness. Separability. Normed vector spaces. Banach spaces. Bounded linear operators. Continuous linear functionals. Dual space. Hilbert Spaces. Subspaces. Orthogonality. Inverse Problems. Bounded Operators on Hilbert Space. Elements of Abstract Integration. Measure spaces. Measurable functions. Simple functions. Abstract integral. Beppo Levi's theorem. Dominated convergence. Lebesgue spaces. BIBO stability in continuous time. Differentiability and abstract integral. Square integrable functions. Different types of convergence and their relation. Harmonic analysis: Reisz-Fischer theorem.Convergence of Fourier series. Fourier transform and DTFT. Reproducing Kernel Hilbert Spaces. Smoothness and decay of Fourier coefficients. Kernel methods in function estimation and identification. Representer theorem. Elements of Calculus of Variations. Euler-Lagrange equations. Hamilton principle in classical mechanics. Convex sets, convex functions. Jensen's inequality. Fundamental result of convex optimization. Application to the Neyman-Pearson Lemma. Constrained optimization: Lagrange Lemma. Karush-Kuhn-Tucker conditions. Optimal Control Problems. Hamilton-Jacobi-Bellman equations. Existence and uniqueness for ordinary differential equations. Maximum entropy problems: Boltzmann, Dempster, Burg. Frechet differential. Geometry of maximum entropy problems. Subgradients and subdifferentials. Jensen's inequality in general. Relative entropy (Kullback-Leibler index). Relative entropy rate. Elements of Markov Chains theory. Perron-Frobenius theorem.