Package: simglucose
simglucoseA Type-1 Diabetes simulator implemented in Python for Reinforcement Learning purposeThis simulator is a python implementation of the FDA-approvedUVa/Padova Simulator (2008 version)for research purpose only. The simulator includes 30 virtual patients, 10 adolescents, 10 adults, 10 children.HOW TO CITE: Jinyu Xie. Simglucose v0.2.1 (2018) [Online]. Available:https://github.com/jxx123/simglucose. Accessed on: Month-Date-Year.Notice: simglucose no longer supports python 3.7 and 3.8, please update to >=3.9 verison. Thanks!Announcement (08/20/2023): simglucose now supports gymnasium! Checkexamples/run_gymnasium.pyfor usage.AnimationCVGA PlotBG Trace PlotRisk Index StatsMain FeaturesSimulation environment followsOpenAI gymandrllabAPIs. It returns observation, reward, done, info at each step, which means the simulator is "reinforcement-learning-ready".Supports customized reward function. The reward function is a function of blood glucose measurements in the last hour. By default, the reward at each step isrisk[t-1] - risk[t].risk[t]is the risk index at timetdefined in thispaper.Supports parallel computing. The simulator simulates multiple patients in parallel usingpathos multiprocessing package(you are free to turn parallel off by settingparallel=False).The simulator provides a random scenario generator (from simglucose.simulation.scenario_gen import RandomScenario) and a customized scenario generator (from simglucose.simulation.scenario import CustomScenario). Commandline user-interface will guide you through the scenario settings.The simulator provides the most basic basal-bolus controller for now. It provides very simple syntax to implement your own controller, like Model Predictive Control, PID control, reinforcement learning control, etc.You can specify random seed in case you want to repeat your experiments.The simulator will generate several plots for performance analysis after simulation. The plots include blood glucose trace plot, Control Variability Grid Analysis (CVGA) plot, statistics plot of blood glucose in different zones, risk indices statistics plot.NOTE:animateandparallelcannot be set toTrueat the same time in macOS. Most backends of matplotlib in macOS is not thread-safe. Windows has not been tested. Let me know the results if anybody has tested it out.InstallationIt is highly recommended usingpipto installsimglucose, follow thislinkto install pip.Auto installation:pipinstallsimglucoseManual installation:gitclonehttps://github.com/jxx123/simglucose.gitcdsimglucoseIf you havepipinstalled, thenpipinstall-e.If you do not havepip, thenpythonsetup.pyinstallIfrllab (optional)is installed, the package will utilize some functionalities in rllab.Note: there might be some minor differences between auto install version and manual install version. Usegit cloneand manual installation to get the latest version.Quick StartUse simglucose as a simulator and test controllersRun the simulator user interfacefromsimglucose.simulation.user_interfaceimportsimulatesimulate()You are free to implement your own controller, and test it in the simulator. For example,fromsimglucose.simulation.user_interfaceimportsimulatefromsimglucose.controller.baseimportController,ActionclassMyController(Controller):def__init__(self,init_state):self.init_state=init_stateself.state=init_statedefpolicy(self,observation,reward,done,**info):'''Every controller must have this implementation!----Inputs:observation - a namedtuple defined in simglucose.simulation.env. Fornow, it only has one entry: blood glucose level measuredby CGM sensor.reward      - current reward returned by environmentdone        - True, game over. False, game continuesinfo        - additional information as key word arguments,simglucose.simulation.env.T1DSimEnv returns patient_nameand sample_time----Output:action - a namedtuple defined at the beginning of this file. Thecontroller action contains two entries: basal, bolus'''self.state=observationaction=Action(basal=0,bolus=0)returnactiondefreset(self):'''Reset the controller state to inital state, must be implemented'''self.state=self.init_statectrller=MyController(0)simulate(controller=ctrller)These two examples can also be found in examples\ folder.In fact, you can specify a lot more simulation parameters throughsimulation:simulate(sim_time=my_sim_time,scenario=my_scenario,controller=my_controller,start_time=my_start_time,save_path=my_save_path,animate=False,parallel=True)OpenAI Gym usageUsing default rewardimportgym# Register gym environment. By specifying kwargs,# you are able to choose which patient or patients to simulate.# patient_name must be 'adolescent#001' to 'adolescent#010',# or 'adult#001' to 'adult#010', or 'child#001' to 'child#010'# It can also be a list of patient names# You can also specify a custom scenario or a list of custom scenarios# If you chose a list of patient names or a list of custom scenarios,# every time the environment is reset, a random patient and scenario will be# chosen from the listfromgym.envs.registrationimportregisterfromsimglucose.simulation.scenarioimportCustomScenariofromdatetimeimportdatetimestart_time=datetime(2018,1,1,0,0,0)meal_scenario=CustomScenario(start_time=start_time,scenario=[(1,20)])register(id='simglucose-adolescent2-v0',entry_point='simglucose.envs:T1DSimEnv',kwargs={'patient_name':'adolescent#002','custom_scenario':meal_scenario})env=gym.make('simglucose-adolescent2-v0')observation=env.reset()fortinrange(100):env.render(mode='human')print(observation)# Action in the gym environment is a scalar# representing the basal insulin, which differs from# the regular controller action outside the gym# environment (a tuple (basal, bolus)).# In the perfect situation, the agent should be able# to control the glucose only through basal instead# of asking patient to take bolusaction=env.action_space.sample()observation,reward,done,info=env.step(action)ifdone:print("Episode finished after{}timesteps".format(t+1))breakCustomized reward functionimportgymfromgym.envs.registrationimportregisterdefcustom_reward(BG_last_hour):ifBG_last_hour[-1]>180:return-1elifBG_last_hour[-1]<70:return-2else:return1register(id='simglucose-adolescent2-v0',entry_point='simglucose.envs:T1DSimEnv',kwargs={'patient_name':'adolescent#002','reward_fun':custom_reward})env=gym.make('simglucose-adolescent2-v0')reward=1done=Falseobservation=env.reset()fortinrange(200):env.render(mode='human')action=env.action_space.sample()observation,reward,done,info=env.step(action)print(observation)print("Reward ={}".format(reward))ifdone:print("Episode finished after{}timesteps".format(t+1))breakrllab usagefromrllab.algos.ddpgimportDDPGfromrllab.envs.normalized_envimportnormalizefromrllab.exploration_strategies.ou_strategyimportOUStrategyfromrllab.policies.deterministic_mlp_policyimportDeterministicMLPPolicyfromrllab.q_functions.continuous_mlp_q_functionimportContinuousMLPQFunctionfromrllab.envs.gym_envimportGymEnvfromgym.envs.registrationimportregisterregister(id='simglucose-adolescent2-v0',entry_point='simglucose.envs:T1DSimEnv',kwargs={'patient_name':'adolescent#002'})env=GymEnv('simglucose-adolescent2-v0')env=normalize(env)policy=DeterministicMLPPolicy(env_spec=env.spec,# The neural network policy should have two hidden layers, each with 32 hidden units.hidden_sizes=(32,32))es=OUStrategy(env_spec=env.spec)qf=ContinuousMLPQFunction(env_spec=env.spec)algo=DDPG(env=env,policy=policy,es=es,qf=qf,batch_size=32,max_path_length=100,epoch_length=1000,min_pool_size=10000,n_epochs=1000,discount=0.99,scale_reward=0.01,qf_learning_rate=1e-3,policy_learning_rate=1e-4)algo.train()Advanced UsageYou can create the simulation objects, and run batch simulation. For example,fromsimglucose.simulation.envimportT1DSimEnvfromsimglucose.controller.basal_bolus_ctrllerimportBBControllerfromsimglucose.sensor.cgmimportCGMSensorfromsimglucose.actuator.pumpimportInsulinPumpfromsimglucose.patient.t1dpatientimportT1DPatientfromsimglucose.simulation.scenario_genimportRandomScenariofromsimglucose.simulation.scenarioimportCustomScenariofromsimglucose.simulation.sim_engineimportSimObj,sim,batch_simfromdatetimeimporttimedeltafromdatetimeimportdatetime# specify start_time as the beginning of todaynow=datetime.now()start_time=datetime.combine(now.date(),datetime.min.time())# --------- Create Random Scenario --------------# Specify results saving pathpath='./results'# Create a simulation environmentpatient=T1DPatient.withName('adolescent#001')sensor=CGMSensor.withName('Dexcom',seed=1)pump=InsulinPump.withName('Insulet')scenario=RandomScenario(start_time=start_time,seed=1)env=T1DSimEnv(patient,sensor,pump,scenario)# Create a controllercontroller=BBController()# Put them together to create a simulation objects1=SimObj(env,controller,timedelta(days=1),animate=False,path=path)results1=sim(s1)print(results1)# --------- Create Custom Scenario --------------# Create a simulation environmentpatient=T1DPatient.withName('adolescent#001')sensor=CGMSensor.withName('Dexcom',seed=1)pump=InsulinPump.withName('Insulet')# custom scenario is a list of tuples (time, meal_size)scen=[(7,45),(12,70),(16,15),(18,80),(23,10)]scenario=CustomScenario(start_time=start_time,scenario=scen)env=T1DSimEnv(patient,sensor,pump,scenario)# Create a controllercontroller=BBController()# Put them together to create a simulation objects2=SimObj(env,controller,timedelta(days=1),animate=False,path=path)results2=sim(s2)print(results2)# --------- batch simulation --------------# Re-initialize simulation objectss1.reset()s2.reset()# create a list of SimObj, and call batch_sims=[s1,s2]results=batch_sim(s,parallel=True)print(results)Run analysis offline (example/offline_analysis.py):fromsimglucose.analysis.reportimportreportimportpandasaspdfrompathlibimportPath# get the path to the example folderexmaple_pth=Path(__file__).parent# find all csv with pattern *#*.csv, e.g. adolescent#001.csvresult_filenames=list(exmaple_pth.glob('results/2017-12-31_17-46-32/*#*.csv'))patient_names=[f.stemforfinresult_filenames]df=pd.concat([pd.read_csv(str(f),index_col=0)forfinresult_filenames],keys=patient_names)report(df)Release Notes08/20/2023Fixed numpy compatibility issues for risk index computation (thanks to @yihuicai)Support Gymnasium.NOTE: the observation in gymnasium version is no longer a namedtuple with a CGM field. It is a numpy array instead (to be consistent with its space definition).NOTE: Python 3.7 and 3.8 are no longer supported. Please update to >3.9 version.03/10/2021Fixed some random seed issues.5/27/2020Add PIDController at simglucose/controller/pid_ctrller. There is an example at examples/run_pid_controller.py showing how to use it.9/10/2018Controllerpolicymethod gets access to all the current patient state throughinfo['patient_state'].2/26/2018Support customized reward function.1/10/2018Added workaround to select patient when make gym environment: register gym environment by passing kwargs of patient_name.1/7/2018Added OpenAI gym support, usegym.make('simglucose-v0')to make the environment.Noticed issue: the patient name selection is not available in gym.make for now. The patient name has to be hard-coded in the constructor ofsimglucose.envs.T1DSimEnv.Reporting issuesShoot me any bugs, enhancements or even discussion bycreating issues.How to contributeThe following instruction is originally from thecontribution instructions of sklearn.The preferred workflow for contributing to simglucose is to fork themain repositoryon
GitHub, clone, and develop on a branch. Steps:Fork theproject repositoryby clicking on the 'Fork' button near the top right of the page. This creates
a copy of the code under your GitHub user account. For more details on
how to fork a repository seethis guide.Clone your fork of the simglucose repo from your GitHub account to your local disk:$gitclonegit@github.com:YourLogin/simglucose.git
$cdsimglucoseCreate afeaturebranch to hold your development changes:$gitcheckout-bmy-featureAlways use afeaturebranch. It's good practice tonever work on themasterbranch!Develop the feature on your feature branch. Add changed files usinggit addand thengit commitfiles:$gitaddmodified_files
$gitcommitto record your changes in Git, then push the changes to your GitHub account with:$gitpush-uoriginmy-featureFollowthese instructionsto create a pull request from your fork. This will email the committers.(If any of the above seems like magic to you, please look up theGit documentationon the web, or ask a friend or another contributor for help.)