Case Study: Forio Epicenter

Modelers and Educators create amazing applications that have the power to train, engage, & convey important information, but unfortunately there are a lot of these models that stay trapped because getting them online is just too hard. Forio Epicenter helps to solve some of the biggest problems facing these applications: creating an interface, managing user access, & sharing the interface with colleagues.

When an author creates a model, they most often use an IDE that contains tools for error logging and control, but when the model gets moved to the web, logging can become more complex. Tools like Postman, though powerful, are not as understandable to people not familiar with programming. So how can a educator, modeler, or data scientist ensure that the data being produced is accurate? In addition, once that data is accurate and available, how can they analyze the performance and identify potential problems?

Error Logging

We realized that error logging was a problem for our users, so working in a team comprised of a few developers, another designer, and the product manager, we started by examining the current offering. How were authors determining that their simulations had errors? Could there be a use for onboard logging?

I interviewed a few internal stakeholders as well as a handful of external stakeholders and users. I dug into their usage and uncovered that the demand for both logging and analytics was greater than initially estimated. We determined 2 major areas of improvement: model output / error logging & user analytics.

Our users’ models often worked flawlessly offline, but often when they started to connect them to our API’s, they would encounter errors. However, they didn’t know what the errors were or how they could fix them. I started by creating user stories that could outline the usage options. As I learned more, the stories grew. The logging needs quickly grew to encompass not just potential model errors, but also Node.js errors and access problems that their users were having.

We identified 3 major categories of logs:

• Model system logs: that contained the critical errors as well as details they coded into their models to report out.
• User access logs: these were the listing of all the ways that people accessed the simulation.
• System errors: these were most important to us, they told us and the authors about errors with the API’s or how they were being used.

I created a series of interactive tables in HTML that allowed us to see the errors and determine the ways a user would want the information. We decided to divide the data 2 ways. In a searchable, filterable table, and as a stacked column chart. The column chart additionally acted a one of the key filter mechanisms. Users could change the ranges in the chart, and the table would update.

Analytics:

As for the analytics dashboard, I had ideas of what data could be useful, but I wanted to have as much of an understanding of what was possible as I could. To best tackle this problem, my team and I created a page containing all the raw data the platform could provide to uncover what information was actually useful. There is no use in plotting data that rarely (or never) changes as a line chart. This raw data dump of sorts allowed us to experiment with the wealth of data possibilities.

We discovered that there were several possibilities for data comparisons and I jumped to wireframe a few of the possible options so we could test the data and see some results. We gathered a small group of our stakeholders from previous interviews and had them explore the dashboard options explaining to us the usefulness (or uselessness) of some of the new features.

The team decided that we wanted to perform testing on a higher-fidelity prototype, so I created a few mockups of the possible dashboard designs. We kept them entirely static, to see how the users understood the data from a glance and then later take the deeper dive.

After very successful user tests, we moved the features to be developed. However, at the current moment, both features are still in development and are tentatively planned on being completed early next year.