{"id":12716,"date":"2021-11-17T19:00:27","date_gmt":"2021-11-18T00:00:27","guid":{"rendered":"https:\/\/carleton.ca\/scs\/?page_id=12716"},"modified":"2026-06-02T14:59:26","modified_gmt":"2026-06-02T18:59:26","slug":"tr-226-switching-models-for-non-stationary-random-environments","status":"publish","type":"page","link":"https:\/\/carleton.ca\/scs\/research\/scs-technical-reports\/technical-reports-1993\/tr-226-switching-models-for-non-stationary-random-environments\/","title":{"rendered":"TR-226: Switching Models for Non-Stationary Random Environments"},"content":{"rendered":"\n<section class=\"w-screen px-6 cu-section cu-section--white ml-offset-center md:px-8 lg:px-14\">\n    <div class=\"space-y-6 cu-max-w-child-5xl  md:space-y-10 cu-prose-first-last\">\n\n            <div class=\"cu-textmedia flex flex-col lg:flex-row mx-auto gap-6 md:gap-10 my-6 md:my-12 first:mt-0 max-w-5xl\">\n        <div class=\"justify-start cu-textmedia-content cu-prose-first-last\" style=\"flex: 0 0 100%;\">\n            <header class=\"font-light prose-xl cu-pageheader md:prose-2xl cu-component-updated cu-prose-first-last\">\n                                    <h1 class=\"cu-prose-first-last font-semibold !mt-2 mb-4 md:mb-6 relative after:absolute after:h-px after:bottom-0 after:bg-cu-red after:left-px text-3xl md:text-4xl lg:text-5xl lg:leading-[3.5rem] pb-5 after:w-10 text-cu-black-700 not-prose\">\n                        TR-226: Switching Models for Non-Stationary Random Environments\n                    <\/h1>\n                \n                                \n                            <\/header>\n\n                    <\/div>\n\n            <\/div>\n\n    <\/div>\n<\/section>\n\n<p>Carleton University<br>\n<a href=\"https:\/\/carleton.ca\/scs\/research\/scs-technical-reports\/technical-reports-1993\/\">Technical Report<\/a> <strong>TR-226<\/strong><br>\nJuly 1993<\/p>\n\n\n\n<h2 id=\"switching-models-for-non-stationary-random-environments\" class=\"wp-block-heading tr_t1\">Switching Models for Non-Stationary Random Environments<\/h2>\n\n\n\n<div class=\"tr_t3\">\n<div class=\"tr_t3\">\n<div class=\"tr_t3\">B. John Oommen &amp; Hassan Masum<\/div>\n<\/div>\n<\/div>\n\n\n\n<div>\n<h3>Abstract<\/h3>\n<p>Learning automata are stochastic finite state machines that attempt to learn the characteristic of an unknown random environment with which they interact. The fundamental problem is that of learning, through feedback, the action which has the highest probability of being rewarded by the environment. The problem of designing automata for stationary environments has been extensively studied. When the environment is non-stationary, the question of modelling the nonstationarity is, in itself, a very interesting problem. In this paper, we generalize the model used in [14,15] to present three models of non-stationarity. In the first two cases, the non-stationarity is modelled by a homogeneous Markov chain governing the way in which the characteristics change. The final model considers the more general case when the transition matrix of this chain itself changes with time in a geometric manner. In each case we have analyzed the stochastic properties of the resultant switching environment. The question of analyzing the various learning machines when interacting with these environments introduces an entire new avenue of open research problems. We are currently investigating how the three models introduced here (and in particular, the time varying model) are applicable to modelling telephone traffic.<\/p>\n<\/div>\n\n\n\n<p><a href=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/TR226.pdf\">TR-226.pdf<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Carleton University Technical Report TR-226 July 1993 Switching Models for Non-Stationary Random Environments B. John Oommen &amp; Hassan Masum Abstract Learning automata are stochastic finite state machines that attempt to learn the characteristic of an unknown random environment with which they interact. The fundamental problem is that of learning, through feedback, the action which has [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":11912,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_cu_dining_location_slug":"","footnotes":"","_links_to":"","_links_to_target":""},"cu_page_type":[],"class_list":["post-12716","page","type-page","status-publish","hentry"],"acf":{"cu_post_thumbnail":false},"_links":{"self":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/pages\/12716","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/comments?post=12716"}],"version-history":[{"count":1,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/pages\/12716\/revisions"}],"predecessor-version":[{"id":12717,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/pages\/12716\/revisions\/12717"}],"up":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/pages\/11912"}],"wp:attachment":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/media?parent=12716"}],"wp:term":[{"taxonomy":"cu_page_type","embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/cu_page_type?post=12716"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}