Working with ipyparallel
In this example we want to use basico from ipython, in a ipyparallel
, as the normal approach with using multiprocessing
does not seem to work across operating systems within the jupyter notebooks (read: i could not make it work on Windows).
[1]:
from basico import *
import ipyparallel as ipp
we could create a cluster manually, in my case i just created one in the cluster menu in the jupyter setup, so to access it we just need to run:
[2]:
cluster = ipp.Cluster.from_file()
rc = cluster.connect_client_sync()
# wait to get the engines and print their id
rc.wait_for_engines(6); rc.ids
Using existing profile dir: 'C:\\Users\\fbergmann\\.ipython\\profile_default'
Using existing profile dir: 'C:\\Users\\fbergmann\\.ipython\\profile_default'
[2]:
[0, 1, 2, 3, 4, 5]
Now we create a direct view with all of the workers:
[3]:
dview = rc[:]
The model we use is the BioModel 68, since we will need the model many times, i download it once and save it to a local file:
[4]:
m = load_biomodel(68)
save_model('bm68.cps', model=m)
remove_datamodel(m)
And define the worker method, the worker just loads a biomodel (in case it was not loaded into the worker before, otherwise it accesses the current model). It then chooses a random initial concentration for 2 species, and computes the steady state of the model. Finally the initial concentrations used and the fluxes computed are return. In case a seed
is specified, it will be used to initialize the rng:
[5]:
def worker_method(seed=None):
import basico
import random
if seed is not None:
random.seed(seed)
if basico.get_num_loaded_models() == 0:
m = basico.load_model('bm68.cps')
else:
m = basico.get_current_model()
# we sample the model as described in Mendes (2009)
cysteine = 0.3 * 10 ** random.uniform(0, 3)
adomed = random.uniform(0, 100)
# set the sampled initial concentration.
basico.set_species('Cysteine', initial_concentration=cysteine, model=m)
basico.set_species('S-adenosylmethionine', initial_concentration=adomed, model=m)
# compute the steady state
_ = basico.run_steadystate(model=m)
# retrieve the current flux values
fluxes = basico.get_reactions(model=m).flux
# and return as tuple
return (cysteine, adomed, fluxes[1], fluxes[2])
We can invoke this method synchronusly in the notebook to obtain the result:
[6]:
worker_method()
[6]:
(0.3570060030772281,
95.66492382480568,
0.012320973396985456,
0.9876790266030145)
now we want to do that many times over (passing along the index to set the seed every time to get about the same result on my machine):
[7]:
%%time
sync_results = []
for i in range(2000):
sync_results.append(worker_method(i))
Wall time: 8.86 s
here a utility method plotting the result:
[8]:
def plot_result(results):
cys = [x[0] for x in results]
ado = [x[1] for x in results]
y1 = [x[2] for x in results]
y2 = [x[3] for x in results]
plt.plot(ado, y1, 'x')
plt.plot(ado, y2, 'o')
plt.show()
[9]:
plot_result(sync_results)

So that for me took some 10 seconds (even though the model was loaded already), so the next thing to do is to run it in parallel. Here i start 10000 runs, passing along the seed for each one, to make it reproducible:
[10]:
%%time
ar_map = dview.map_async(worker_method, [i for i in range(10000)])
ar_map.wait_for_output()
Wall time: 21.5 s
[10]:
True
And now we can plot that as well:
[11]:
plot_result(ar_map.get())
