Statistical Analysis of the simulation results

spamdfba saves the simulation results frequently during training. the stats module is designed to statistically analyze the simulation results. For instance, we can compare the concentrations of metabolite x under two different policies. Here, we compare the starch concentration at the end of the episodes when the agents are trained on high vs. low mass transfer rate. To start, we need to import the stats module.

from spamdfba import stats

We now define the adress to .pkl files that contain the observations.

obs1="./statistical_analysis/Toy-Exoenzyme-Five-Agents-Mass-Tansfer-high/observations_4500.pkl"
obs2="./statistical_analysis/Toy-Exoenzyme-Five-agents-mass-transfer-low/observations_4500.pkl"

For the next step we need to think about what do we want to compare: We want to compare the concentration of starch at the end of episodes for high and low mass transfer rate. Please refer to the starch/amylase casestudy notebook for more information about this environment. In this environment, the index of starch as a compound is 6. We also would like to compare the concentration of starch at the end of the episodes for the two environment. we can use -1 index to show the latest timepoint, or use the exact time index which is 999 for the end time point. Here we use -1 for simplicity.

res=stats.compare_observations(obs1,
                           obs2,
                           compounds=[6],
                           on_index=-1)

print(res[0])

    
    mean1:9.499237060546875
    mean2:9.211033821105957
    std1:0.00024246313842013478
    std2:0.00032797938911244273
    -------------------
    ANOVA: statistic=1497866.257148871, pvalue=2.008538191244597e-17
    

We see that the results are significantly different and lower mass transfer rate leads to lower starch concentration at the end of the episodes. This means that less agitation increases the starch breakdown because of the cheating problem mentioned in the manuscript. Finally, we can plot the results.

fig=res[0].box_plot(plot=False)

fig.show(renderer="svg")