Sugar Rush! Machine Learning for Insulin Dosages
Implementation of a Machine Learning Algorithm to Improve Insulin Dosage Accuracy for Insulin Dependent Diabetics
Neil Thistlethwaite and Sarkis Ter Martirosyan
Thomas Jefferson High School for Science and Technology
This paper was originally included in the 2018 print publication of Teknos Science Journal.
Individuals afflicted with Type 1 Diabetes cannot produce insulin naturally and must regulate their blood sugar by injecting insulin before meals. Insulin dosage is determined based on the number of carbohydrates in a meal. However, depending on the individual, some foods impact blood sugar disproportionately, requiring more insulin. The aim of this project was to create an interactive Android application, DiaHelp, which would assist Type 1 Diabetics in calculating a precise insulin dosage based on a multi-featured, linear-regression, machine learning algorithm. The application uses an interactive interface to collect user specific meal information on the type and amount of food consumed. The system calculates the amount of carbohydrates in a meal, and uses that information along with a set of multipliers derived by the machine learning algorithm to calculate insulin dosage. Over time, the algorithm uses that information to further adjust the multipliers with increasing precision, giving the user better control over blood sugar. The algorithm, rigorously tested with synthesized data, performed well across 30 data sets with 50 “meals” each. This application will allow Type 1 Diabetics to attain better blood sugar control, ultimately improving quality of life and decreasing risk of complications.
Introduction
Both type one diabetics and many type two diabetics are insulin dependent, meaning that they require external insulin doses to maintain their blood sugar, also known as blood glucose (BG). Normally, sufficient insulin is created by the body to regulate blood sugar to healthy levels, but diabetes complicates this. In type 1 diabetics, the body can no longer produce enough insulin, while in type 2 diabetics, the body becomes insulin-resistant, and requires more than the body can produce. Typically, dosages are taken before meals to account for the accompanying blood sugar increase, which raises the question of how much insulin to shoot for each meal, and how to adjust this based on certain features of each meal. Ideally, diabetics should administer a quantity of insulin so as to keep their blood sugar within a well defined healthy range, to avoid both long and short term complications.
Diabetes Control and Complications Trial Research Group (1993) detail how accurate blood glucose management is extremely important to a diabetic’s health: hyperglycemia, high blood sugar, can have severe long-term complications such as retinopathy and nephropathy, and hypoglycemia, low blood sugar, can have fatal short-term complications, resulting in a coma followed by death if left untreated. According to Gillespie et al., the most commonly used method of calculating insulin dose is carbohydrate counting, simply adding up the grams of carbohydrates in a meal, and then dividing by a number, or “carb ratio”, to obtain a quantity of insulin units to administer for the meal. However, different foods affect blood sugar disproportionately, especially with regards to starch and simple carbohydrate content, leading to the development of the glycemic index in Jenkins et al. (1993). However, Jenkins et al. (1998) summarized the weaknesses of the glycemic index method, notably the individual variation in glycemic responses, varying by “type of diabetes … age, sex, body weight, and race.”
Currently, there are no feasible methods that remedy this problem on a per-individual basis, accounting for individual idiosyncrasies in blood sugar response as a function of meal content. In our research, we attempt to address this issue by creating a machine learning model that learns different parameters for each individual, effectively generating a custom food index. Under our method, every individual has a different “carb ratio” for every food, providing a powerful augmentation to the traditional carbohydrate counting method as well as the glycemic index method, which allows for improved blood sugar management.
Methods and Materials
The machine learning backend of our application was initially developed in Python 3.5, with no outside libraries, given the comparatively simple architecture of our model, which also gave us the ability to rapidly prototype and test it. We chose to use a supervised learning algorithm, with a linear regression based model, representing “carb multipliers” for each food as the variables, optimizing on the cost function defined by
where hθ(x(i)) represents the model’s current prediction for a meal x(i), given parameter vector θ, y(i) represents the actual ground truth for that meal, as calculated by measured blood sugar before and after, m represents the number of meals in the dataset, λ represents the regularization coefficient, and θ represents the parameter vector, the coefficients used on each food to determine comparatively how much its carbohydrates are “weighted” versus the other foods. Overall, this is a relatively typical model for multivariate linear regression, where coefficients for each food represent the unknown parameters and the quantity of each food in a meal represents the variables defining that meal. One notable difference, however, is that the parameters in θ are regularized around one, instead of zero, since in our model having all parameters equal to one represents the traditional carb counting method, for which a regularization cost penalty should not be added in. The regularization simply serves to prevent the model from overfitting to the data by deviating too far from the original.
This cost function represents the sum of squared errors on each “meal” in the dataset, where the error on each individual meal is calculated by subtracting the amount of insulin predicted by the model for the meal’s contents from the real correct amount of insulin associated with that meal. The prediction was obtained by summing the products of the current learned carbohydrate “multiplier” and quantity of carbohydrates of that food in the meal for each food type, then dividing by the overall carbohydrate ratio. In this way, through minimizing the total error, the model would effectively learn better carbohydrate ratios for each food type. If a particular food tends to affect blood sugar much more than others, the model will essentially learn this fact from the meals that have that food, inferring that meals with food X tend to have higher insulin dosages. Ultimately, with more training data, the model ideally becomes capable of predicting very fine dosages, even through noise over individual meal measurements.
One inherent difficulty in getting a machine learning model to learn parameters for a diabetic’s personal insulin dosage is the lack of data; typically, models have access to vast amounts of data, enabling them to generalize trends instead of fitting to specific examples. Any model that has more features than the number of training examples has capacity to overfit. Our solution for this problem was to choose a model with a relatively low parameter space dimensionality, meaning it has fewer weights to learn, and to institute regularization, further constraining the possible learned parameters. It is critical that our model be able to output reasonable predictions after a reasonable amount of data points, as it is infeasible if the app requires a user to input thousands of meals before it predicts anything, and that these predictions be safe and accurate.
Another consideration was use of our model in an online learning setting. Rather than receiving a large amount of data and yielding a single prediction, the use case of our model is in continuous learning, receiving new data points from the user on a regular basis, fine tuning and adjusting its model as necessary in order to yield consistently accurate predictions. This was also part of the motivation for picking a multivariate linear regression model, as it was easily adaptable to such a usage, integrated into an app.
The model was trained through standard batch gradient descent, given that the defined cost function is convex, and is guaranteed to have only one minimum. In the app, adding another meal, or data point, initiates training from where the last training left off, so as to continuously improve the prediction with data.
For testing, we used artificially generated labeled data, due to the inherent difficulty in gathering a real dataset with accurate ground truth labels, both in data collection and ethical reasons before proving that the algorithm is safe for human use. Notably our method of generating artificial data is predicated on the assumptions we made about food response, that is, that each distinct type of food has a consistent blood sugar response over time (i.e., that a given food does not fundamentally change how it affects blood sugar between two meals), and that each food’s impact on blood sugar is linearly related to its carbohydrate content by some arbitrary coefficient c, with c staying constant for a given food.
Specifically, artificial data was generated by first synthesizing a list of “foods” and their hidden coefficients (which the model has no access to), and then creating a set of data points with random amounts of each food type (each meal had random quantities of between 3 and 6 food types), and then using the inverse process to find out how many “effective carbs” each meal had. Then, an arbitrary starting blood sugar was chosen, an amount of insulin based on actual carbs was calculated, and finally the “ending” blood sugar three hours after the meal was calculated based on the former values. Finally, measurements were fuzzed with gaussian variation, modeling real-world random variation in measurements. Thus, the information available to our model to learn for each meal was the starting blood sugar, ending blood sugar, meal content, and amount of insulin administered, the same information that a patient using the app would provide at each meal.
We assessed the efficacy of our model in two ways: both qualitatively by comparing how close the learned parameters on the artificial dataset were to the hidden parameters, which the model never directly saw, and quantitatively by measuring average errors in mealtime insulin predictions and comparing this to a baseline of the traditional method of carb counting, without taking into account food type differences.
We chose to develop an Android application to house the algorithm, as mobile applications can significantly improve glycemic control [7]. Furthermore, about 85% of today’s mobile phones are running some version of the Android mobile operating system [4]. Fundamentally, the application must allow the user to enter foods either based on either the amount of servings or the mass consumed. In addition, there must be an option to enter foods not already defined in the app. From there, the app should convert the meal data into grams of carbohydrates and collect the starting blood glucose, actual insulin dosage injected, and—two hours later—the ending blood glucose. From historical data and this new information, the application will perform the above algorithm to update its glycemic index model.
Results
After testing on artificial data, we found our model to yield significantly better results than traditional “blind” carb-counting. We tested each method on the test set of artificial data (which the machine learning model had never seen before, only the training set generated from the same hidden parameters), and recorded the difference between each method’s predicted dosage and the ground truth actual dosage. The mean error from the machine learning model we developed was found to be statistically significantly less than the error from traditional carb counting, p < 0.001. This lead us to reject the null hypothesis that our method was no better than blind carb-counting. These results were found by running a paired 2-sample T-test on the data (t = 6.0784, df = 29, std. err. of diff. = 0.413). In fact, after a full training run, the algorithm was off from the actual prediction by less than 0.45 units 90% of the time, while the traditional method was regularly off by over 2 units. These results suggest that, at least for artificial data, our model is extremely capable of learning these hidden parameters in foods and adjusting dosage predictions to match.
We also found that as the model gained access to more training examples, it increasingly improved its predictions (Figure 4). This indicates that our model was not overfitting, despite the low training set error, as, with enough examples, it was able to generalize to new examples it hadn’t seen before. This also justified our thresholds for the minimum number of examples the app would need before it would recommend a prediction, as well as the minimum number before it indicated it had a higher level of confidence in its prediction.
In designing the application, minimality and Material Design [3] was kept in mind. The interface is intentionally simple as to allow even the least technologically experienced user to intuitively use the application. The application was designed in Android Studio, and tested on the 2016 Google Pixel and Nexus 5.
When the user installs the DiaHelp application (henceforth referred to as “application” or “app”), some food items are preprogrammed in the form of an XML array. User-defined carbohydrate data is stored in a custom pipe (“|”) delimited format. The programmatic workflow, which most accurately reflects the app in terms of software, is shown below (see Figure 1). The code for the application is linked in the References section. The visual diagram (see Figure 2) accurately depicts the user’s experience as the user enters all the pertinent information for a single meal. When the user opens the application, they are presented by the home screen (“MainActivity”), from where they can enter food items based on servings (“ByServingActivity”), weight (“ByMassActivity”), or in a form of the user’s own choosing (“ByCustomActivity”). As the user enters food items, the food items appear on the home screen with the name and the grams of carbohydrates in a table. Once the user has finished entering their meal, they proceed to a screen which asks for the user’s blood sugar (“StartBGActivity”). Immediately after, the user receives their insulin dosage if enough entries have been made or a message indicating how many more entries must be made until the model is considered adequately trained. At that screen (“ResultActivity”), the user is also prompted to enter the actual amount of insulin the user plans to inject. Two hours later, the user receives a notification which, when clicked, takes the user to a screen to enter their after-meal blood glucose (“EndBGActivity”). When the user confirms the entry, the app returns to the home screen (“MainActivity”) and batch gradient descent is run.
As the user enters more and more meals into the application, the accuracy of the model increases. Based on empirical estimates from software testing, the app first begins to recommend insulin dosages after fifty observations, with an accompanying disclaimer to take the recommendation with a grain of salt (see Figure 3). Eventually, after one-hundred meals are entered, the app has full confidence in its model, and displays the recommendation in green with not disclaimer.
Charts and Figures
Figure 1: Programmatic Flow-Chart of Single Meal Entry
Figure 2: Visual Flow of a Single Meal Entry
Figure 3: Dosage Feedback Modes
Figure 4: Learning Curve showing accuracy as model gains access to more training examples.
Figure 5. Sample hidden and learned ratios, demonstrating how the model closely imitates the hidden parameters, despite having no direct access to them.
Conclusion
The machine learning algorithm was successfully able to predict insulin dosages more accurately by learning ratios for each food type, achieving a very low error within ~50 meals data, and performing significantly better than “blind” carb-counting, the most commonly used current method. Although our results are limited due to the use of synthetic data alone, we were able to show that the algorithm could adapt to “hidden parameters” of different food types even in the presence of random variation. The android application proved to be a useful utility for both managing input of the meal data as well as a general keeping track of meals and blood sugar readings.
Our application provides an intuitive, easy-to-use, and effective portal for a user to interface with the algorithm. The workflow for entering a meal is clear, and the color palette is minimal yet aesthetically pleasing. Overall, the application collects meal, blood glucose, and insulin dosage data from the user, stores it in the user’s device’s internal memory, and provides dynamically calculated insulin dosage recommendations. In the future, the application can be enhanced by allowing the user to see his or her meal history, display data as useful visualizations, and interface with continuous glucose monitors and insulin pumps.
References
[1] Diabetes Control and Complications Trial Research Group. (1993). The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl j Med, 1993(329), 977-986.
[2] Gillespie, S. J., Kulkrani, K. , & Daly, A. E. (1998). Using carbohydrate counting in diabetes clinical practice. Journal of the American Dietetic association, 98(8), 897-905.
[3] Google. (n.d.). Material Design. Retrieved September 14, 2017, from https://material.io/guidelines/
[4] IDC Quarterly Mobile Phone Tracker. (n.d.). Smartphone OS Market Share. Retrieved September 03, 2017, from http://www.idc.com/promo/smartphone-market-share/os
[5] Jenkins, D. J., Wolever, T. M., Taylor, R. H., Barker, H., Fielden, H., Baldwin, J. M., ... & Goff, D. V. (1981). Glycemic index of foods: a physiological basis for carbohydrate exchange. The American journal of clinical nutrition, 34(3), 362-366.
[6] Jenkins, D. J., Wolever, T. M., & Jenkins, A. L. (1988). Starchy foods and glycemic index. Diabetes care, 11(2), 149-159.
[7] Kirwan, M., Vandelanotte, C., Fenning, A., & Duncan, M. J. (2013). Diabetes self-management smartphone application for adults with type 1 diabetes: randomized controlled trial. Journal of medical Internet research, 15(11).