Markov Chains

Andy Long
Spring, 2018

This is an introductory example from Olinick’s book An Introduction to Mathematical Models in the Social and Life Sciences: Flakes No More!

TransitionMatrix={{.7,.3},{.4,.6}};
p0={.25,.75};
MatrixForm[TransitionMatrix]
p0.TransitionMatrix
MatrixForm[MatrixPower[TransitionMatrix,4]]
p0.MatrixPower[TransitionMatrix,4]
MatrixForm[MatrixPower[TransitionMatrix,100]]
p0.MatrixPower[TransitionMatrix,100]

Another more interesting example: “A recent study focused on the relationship between the birth weights of English women and the birth weights of their daughters. The weights were split into three categories: low (below 6 pounds), average (between 6 and 8 pounds), and high (above 8 pounds).”

TransitionMatrix={{.5,.45,.05},{.25,.5,.25},{.3,.3,.4}};
p0={.25,.60,.15};
MatrixForm[TransitionMatrix]
p0.TransitionMatrix
MatrixForm[MatrixPower[TransitionMatrix,2]]
p0.MatrixPower[TransitionMatrix,2]
MatrixForm[MatrixPower[TransitionMatrix,4]]
p0.MatrixPower[TransitionMatrix,4]
MatrixForm[MatrixPower[TransitionMatrix,100]]
p0.MatrixPower[TransitionMatrix,100]

Suppose you start with a certain SIR equilibrium; can we determine the SIR model that gave rise to it? 
No -- it’s not unique.

If the equilibrium vector (probability vector, which sums to 1) is ((Subscript[w, S]),(Subscript[w, I]),(Subscript[w, R])), then so long as
(Subscript[w, S])(Subscript[p, SI])=(Subscript[w, I])(Subscript[p, IR])=(Subscript[w, R])(Subscript[p, RS])
this Markov chain will stay in equilibrium. 

There’s only one free parameter in the transition probability matrix: let’s choose to make it (Subscript[p, SI]): without that, we have no infectivity! 

Then 
(Subscript[p, IR])=((Subscript[w, S])/(Subscript[w, I]))(Subscript[p, SI])
and
(Subscript[p, RS])=((Subscript[w, S])/(Subscript[w, R]))(Subscript[p, SI])
Here’s an example:

{ws,wi,wr}={.3,.1,.6};
(* Let this be our stable distribution: *)
ws/wi
ws/wr
equilibrium={ws,wi,wr};
Manipulate[
Module[
{},
prs=ws/wr psi;
pir=ws/wi*psi;
probabilities={psi,pir,prs};
Print[probabilities];
Print[MatrixForm[MatrixPower[TransitionMatrix,30]]];
TransitionMatrix={
{1-psi,psi,0     },
{0     ,1-pir, pir},
{prs,     0, 1-prs}};
equilibrium.TransitionMatrix
]
,{{psi,.1},0,Min[1,wi/ws,wr/ws],.01}
,Text["The equilibrium states don't change:"]
]

Generally, however, one has an idea of the transition probabilities, and then one iterates the matrix until it reaches its own equilibrium, as a matrix of rank 1, and with the equilibrium distribution as that repeated row:

TransitionMatrix={
{1-psi,psi,0     },
{0     ,1-pir, pir},
{prs,     0, 1-prs}};
psi=.1;
pir=.3;
prs=.4;
(* Raising it to the power 300 should be enough...:) *) 
MatrixForm[MatrixPower[TransitionMatrix,300]]

This is an example from Olinick, p. 274, on the birth weights of English women and the birth weights of their daughters: low, below six pounds; average, between 6 and 8 pounds; and high, greater than 8 pounds.

TransitionMatrix={
{.5,.45,.05 },
{.25,.5,.25},
{.3,.3,.4}};
(* Raising it to the power 300 should be enough...:) *) 
MatrixForm[MatrixPower[TransitionMatrix,300]]