Perhaps the largest phase because every project needs to start with User Research (aka the Empathize and Define stages).
I will be BRIEFLY go through some of the deliverables coming out of the research and definition phase!
Competitor analysis
I explored many existing tools online and captured what they did well and not so well. I also went through some of the previously created Kaizen dashboards before I was hired. See below for a collage of tools I analyzed.
User persona
It's also important to define the potential users and what their main needs are. See below for our main target user, who is essentially going to be someone who reports to a business owner and brings their investigation to executives. This was created from a compilation of insights from many existing clients who have documented interests in an Anomaly Detection capabilities.
Ideation
In person whiteboarding sessions are a great tool for bouncing ideas together. It allows for collaboration on preliminary ideas, key features, sketches, and the formation of a lofi mockup.
Key features
There were several other features that came along for the ride, but these were the ones required for MVP:
- View list of anomalies with an anomaly score (how anomalous it is), the segment (the combinations of dimension values), the expected and actual values of what happened, and the magnitude of the anomaly.
- Ability to view all the underlying records in an anomaly and mark them as viewed and / or remove them from analysis
- Rerun the analysis with certain records excluded
Lofi Mockup
Behold! Version 3 of a lofi mockup that all subsequent iterations were based on.
Testing And iterating
There were MANY iterations of this. I mainly tested internally by conducting several rounds of feedback sessions until things became more polished and could put this in front leadership and clients.
One of the main challenges of this phase was creating a UI framework that seemed familiar and made intuitive sense. After a few brainstorming sketches that didn't quite crack the code, inspiration struck while checking my Gmail account one night! I emulated a similar structure on being able to mark lines as read or unread, with the ability to remove.
Hifi prototype
After testing, I created a high fidelity prototype that showcased the key features and several future features that would be implemented at a later state. I did one more round of usability testing with 3 users and incorporated their feedback.
I used the same design process as mentioned above to produce a high fidelity prototype. Skipping all details to keep this short!
Key Features
- Drill into an anomaly and see an easy to digest graph
- The graph would show expected and actual amounts with tooltips
- Ability to add comparison lines to the graph to showcase other relevant segments
In this phase, I needed to find a way to showcase exactly WHY a particular segment was anomalous. In the pictures above, the lofi wireframe on the left hand side demonstrated an early idea as to how to visualize how the platform works and why something could be considered anomalous.
In a nutshell, I had to find a way to demonstrate that platform looks at both historical trends as well as the most relevant comparable segments that are listed out in order from the most relevant and least relevant (and capped at 10) . Through many internal and external feedback sessions, I came up with the design above on the right, where a user can turn on and off different comparison lines to gain further understanding.
For example, if we are tracking technician spend and we see an anomaly for the tech Andrew Hillman at Location 02037, we can see in this graph that Andrew's spending compared to his past spend at this location is way off the charts! Also, for a more in-depth look, a user can turn on the comparison lines. Above, we see that all the other technicians at Location 020307 had their spending go down that week, so something is definitely up with Andrew's spending behavior!
After testing and approval of the prototype, it was time to switch gears to PHASE 3!
Key Features
- Way to see the $ IMPACT the anomalies are having on a business
- Way to see how many anomalies there are
- Way to see Top anomalies by different dimensions / metrics
In this final main phase of design, I was tasked with finding a way to summarize a business's anomalies in one easy-to-read screen.
From many internal sessions and several external interviews, the main insight that users need summarized are really 3 things: "I want to know the how much of my activity is anomalous, how much it's costing me, and what the top areas of concern are."
After several iterations and multiple rounds of user testing on of all kinds of different visualizations and layouts, I came up with something simple and easy to understand. At the time of writing this, this design was going through several rounds of discussion with developers to ensure that each piece of functionality can be built as intended without compromising fluidity and load times due to the massive data sets this tool looks through.
Conclusion and Next steps
As mentioned above in the phase 3 section, there are still undergoing discussions with the development team to ensure that everything in the summary page can be built properly. Once those finish, the development team will begin building. There are still several features and enhancements not mentioned in this case study that have been designed and / or implemented after each part of the initial phases were completed.
