📊 Analytics – Avoiding Chaos and How Less Data Is More
Let us start this by saying we do not proclaiming to be analytics gurus. We are simply developers who have had the pleasure, or perhaps misfortune, of working for several companies that don’t have a proper grasp of data and analytics.
At its core the idea is simple. Connect to some services that consumes events. Create and send some events from a client platform. Observe, learn, grow.
But than… things go awry.
The most common pitfall – ️Track Everything.
We think it’s very common to approach analytics with the idea that by simply “tracking everything” you will have built a fountain of data. And in the future when questions need answered, you simply return to the fountain and the clairvoyance is yours.
We caution you; this is a dangerous philosophy.
The analytics dashboard is a tool that is often shared between many teams and departments. By tracking everything, you introduce so many events that it becomes unclear which events are actually important. In particular events with similar names, but often slightly different numbers, result in internal debates, speculation, and a lot of frustration.
This is one of the biggest reason to avoid trying to create an event for absolutely everything. Each event requires maintenance. A small amount of maintenance perhaps. But eventually as features change and code is refactored. Events will be forgotten. Or worse, misused. This rarely, if ever, surfaces in any kind of testing. Instead it tends to surface when product managers are trying to answer important questions only to discover that months of data is wrong.
Tracking Everything Except What’s Important
Adopting the “Track everything” policy is one part good intention, and one part bad planning. When developing features the vague guidance “Track Everything” leaves a developer guessing what exactly ‘everything’ means. The developer will do his/her best to implement all the things that seemed obvious at the time. Only to discover weeks down the road that the most important metric was somehow missed. This is because everyone wants different information out of a feature. The action orientated lens of developers doesn’t really align with the holistic perspective product managers tend to be looking for.
- Key Metrics – In contrast to “Track Everything” and the issues I’ve listed above. Define the events that are truly important. Try to limit them to meaningful actions that are important. ‘Subscriptions’, ‘Downloads’, ‘Purchases’ are obvious Key Metrics. You should only need a few events for any given feature.
- Testing – Include measuring analytics in the quality assurance process. By openly acknowledging each event requires testing time it becomes clear that maintaining good clean data also means avoiding meaningless events.
- Plan – Before merging in any feature, it is important to cycle back to the product manager and ask them, “What event(s) would you like tracked from this feature?”. I find this only works once a feature is finished. Planning the analytics before a feature is complete can result in a misalignment with how the feature works and what the product manager actually wanted to measure.
- Differentiate Metrics from A/B Test – If possible try to use separate dashboards/system for measuring the general performance of the app vs test designed to experiment on small groups of users. Combining the two actually makes it much harder to measure either.
- Measure the action rather than the UI – It’s temping to add events like “Video Download Button Clicked”. But this event really doesn’t capture the result. For example: “Video Download Button Clicked” doesn’t, on its own, tell us if the download actually started. Maybe the users connection failed. Or perhaps users click the button many times while impatiently waiting for their download to start. The button is just the UI leading up to the action. Whereas the action is likely the metric we actually care about. In this case measuring “Video Download Started” would be more appropriate. And if you find yourself thinking that you actually care about the UI more than the resulting action. Then you probably want to use an A/B test instead.
- A/B Test – When you do want to measure the performance of UI. You really need to do so from the perspective of an A/B test. Just adding or removing a button and trying to measure event counts from before and after leads to unreliable answers. The classic problem of causation vs correlation. Instead you must run at least two variants targeted to small but identical subsets of users. By separating analytics intended for measuring these test it also means once the test is complete the event can be removed. Thus preventing the expense of future maintenance.
General rules for naming analytics.
- Past tense
- Prefixed with category. Suffixed with action.
The rule here is underscore separated all lowercase words.
snake_case offers two important benefits.
1 – Prevents case sensitivity issues. Often event names need to be identical between platforms (I.E. Android/iOS/Web). By adhering to all lowercase it is much easier to avoid this mistake of using
PdfDownloaded on one platform and
PDFDownloaded on another.
2 – You can very easily write a client side validation check to ensure all events are lowercase before logging them.
All events should be written in past tense.
By describing events in paste tense it is more clear the action happened rather than was about to happen. Again
pdf_downloaded is much informative than
pdf_download which leaves if the download actually finished rather ambiguous.
Prefixed With Category, Suffixed With Action.
Prefix related events with a common word. Suffix events with the action taken.
Using common words for event types does require some planning. But you can see how effective it is at displaying which events are correlated to each-other.
In addition using a common word and putting the action and the end of the event ensures like events are sorted together. I.E.
pdf_downloaded conveniently sit next to each-other when browsing events alphabetically.
There is a ton of misinformation floating around on the internet. Rarely does this form a cohesive proper starting point for managing actual meaningful data that can be properly utilized later on. We hope this puts you in the right direction for building a solid foundation when it comes to implementing analytics properly from the beginning.
Thank you for reading. Stay tuned for more examples and some code snippets written in Swift in our upcoming articles!