Here is a Wikipedia article describing continuous-time Markov chains.
Here is the Maple worksheet for the basic model.
And here is the worksheet for the model with extra capacity.
The transient solution involves the modified Bessel function of the 1st kind, and it is calculated in this Maple worksheet.
The Maple worksheet for this bivariate CTMC is here. I wrote two pages of notes on this problem which is here.
This can be achieved using Laplace transforms, see my handwritten notes with an example.