Question For a stationary Discrete-Time Markov Chain (DTMC) we have, in standard form, P, P2, and P3, Po and theassociated matrix (I – Q). The state space for this system is the set {1, 2, 3, 4, 5}. 0.2 0.1 0.4 0.2 0.1 0.1 0.12 0.220.32 0.24 0.2 0.2 0.2 0 0.4 0.1 0.1 0.18 0.08 0.54 P = 0.1 0.2 0.3 0.2 0.2 P2 = 0.09 0.11 0.17 0.28 0.3… Show more… Show more1-f the initial state at time 0 is 3, and three thousand transitions later the state is 1, what is the probability that the process enters a transient state at epoch 3002?2-the process starts in State 1, what is the expected number of visits to State 3 in the first 3 epochs?3-assuming that at epoch 3, the system is in State 3, compute the probability that the system ever visits State 4 in two ways?4-What is the mean number of epochs spent in State 2, per visit to State 2?5-Will solving the set of simultaneous linear equations ? = ? P, along with the normalizing equation ? 1 = 1 provide a meaningful solution? If so, specify the algorithm and offer an interpretation for the resulting vector ?, and if not, then explain why not. Here 1 is a 5 1 column vector.6-the process starts at time zero in State 1, what is the mean time until the process enters its terminating state?7-Since this P is already written in standard form, specify the R submatrix (actual values are necessary here), and interpret the meaning of R.8-n the long-run, what is the variance of the time until the first visit to State 4 from initial state State 1, if we know that State 4 will definitely be visited? Write out the steps of your algorithm and then compute the exact number using MATLAB.MathStatistics and Probability ISE 3414
