In Phase 1 of our project, we developed the first ever open source, extensible, universal machine learning framework for healthcare. As we received data from different hospitals, we realized that we were encountering similar problems across multiple datasets. While the problems were similar, the approaches we used to solve them were initially ad hoc, non-repeatable, and not shared. To overcome this, we built an extensible open source library called Cardea. This library includes a universal data schema called FHIR, the ability to create an adaptive data map, and built-in AutoML to find best covariates and machine learning models. In addition to delivering on all of the goals presented in the original statement of work, the overall framework now affords us — its designers — the ability to develop machine learning models for a new data set or a new problem, and to assess them across different axes including accuracy and reproducibility, all in less than a week.

Phase 2 focused squarely on end users. It aimed to categorize users into different types, identify key issues in framework adoption, and develop algorithmic and technological solutions for mitigating these barriers. Our goal in this phase was to enable evaluation of machine learning models across multiple axes and to provide users with an in-depth understanding of models by: 1) enabling users to create robust models easily, 2) making models transparent and interpretable, and 3) assessing fairness implications of the models.