joint work with Monica Motta, Franco Rampazzo
We consider a nonlinear control system, affine with respect to an unbounded control u taking values in a closed cone of Rm , and with drift depending on a second, ordinary control a, ranging inn a bounded set. We provide first and higher order necessary optimality conditions for a Bolza problem associated to this system. We illustrate how the chance of using impulse perturbations, makes it possible to derive a Maximum Principle which includes both the usual maximum condition and higher order conditions involving Lie brackets of drift and non-drift vector fields
First order necessary conditions for optimal control problems with mixed constraints are well-known when a regularity criterion is satisfied. On the contrary, problems involving nonregular mixed constraints have received little attention, except for some isolated works, despite being relevant in many areas such as robotics and problems involving sweeping systems. We present necessary conditions for problems involving nonregular mixed constraints derived via infinite dimensional optimization. Moreover, we present a form of regularity which leads to the usual first order necessary conditions.
The talk discusses one possible model problem for certain tasks in reinforcement learning. The model provides a framework to deal with situations in which the system dynamics are not known and encodes the available information about the state dynamics as a measure on the space of functions. Such a measure updates in time, taking into account all the previous measurements and extracting new information from them. Technical results about the value function of such a problem are stated, and several possible applications to reinforcement learning and the exploration-exploitation tradeoff are discussed.