LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of a discrete-time stochastic system, convergence of the occupational measures set and averaging of a hybrid control system

The aim of this thesis is twofold. Firstly, we aim at the development of a linear programming based approach to evaluating the Cesàro and Abel limits of optimal values in problems of control of a discrete-time stochastic system in the case when these limits may depend on initial conditions. Secondly, we aim at establishing that, under certain controllability type conditions, the set of random occupational measures generated by the state-control trajectories of the system converges (in a certain sense) to the convex and weak* compact set of stationary measures and that this convergence allows one to use an averaging technique for finding near-optimal controls of hybrid systems that evolve in continuous time and that are subject to abrupt random changes of parameters.

The thesis consists of four chapters. Chapter 1 is the introduction. In Chapter 2, we address the first aim. That is, we use a linear programming (LP) based approach for analysing optimal values in problems of control of a discrete-time stochastic system considered on long or infinite time horizons in the general non-ergodic case. In Chapters 3 and 4 we address the second aim of the thesis. More specifically, in Chapter 3 we establish that, under certain controllability conditions, the set of occupational measures generated by the state-control trajectories of this system converges to the set of stationary measures. In Chapter 4, we use the results of Chapter 3 to develop an averaging technique allowing one to find a near-optimal control of a hybrid system that evolves in continuous time and that is subject to abrupt random changes in parameters synchronised with and controlled by a discrete-time stochastic control system.