SYSTEM METRICS TO SUPPORT DECISION-MAKING
Four reasons why you must use metrics to determine your feature decisions – ProductPlan.com
1. Relying on your instincts is not sustainable. It’s our responsibility to perform due diligence for every decision point we face. But despite the fact that 81% of people believe data should drive decision making, it’s actually only the case less than one-quarter of the time.
2. Overcome inherent biases and use metrics in your product decisions. We don’t always realize we’re relying on our instincts because, well, they’re instinctual! Just like we don’t think about our breathing, we’re constantly assessing multiple variables. Then, we make decisions based on the information at our disposal without even realizing it.
3. Metrics help to avoid making snap product decisions. Beyond your own decision-making, leverage data to the feedback that you receive from stakeholders and customers. Just because one vocal individual (or even a handful) feels a certain way doesn’t mean an idea should be quashed or prioritized.
4. Usage metrics will help quantify what happens when people use your product. Usage metrics are one of the best ways to quantify what’s actually happening when people use your product.
What is our aspiration?
Enterprise Systems will develop and implement a divisionwide standard for collecting system and feature metrics that includes reporting in support of decision-making.
Why is this important?
Our systems are used by tens of thousands of IU constituents daily. Whether someone is checking email, finding a task in One.IU, or building a plan for the upcoming semester, our systems are in use 24 hours a day, 365 days a year. But we have little information readily available to understand which specific systems are used, how they are being used, and by whom.
With limited resources and growing needs from our stakeholders, it is important that we invest our time wisely. Having readily available metrics capturing who, how, and how much our systems are used will help inform us and our stakeholders on how best to invest our time.
What are our concrete goals?
- Integrate Google Analytics within all enterprise systems, where appropriate, to enable standardized metrics.
- Encourage outcome-based metric definition as a part of new bodies of work, with a focus on actionable analytics.
- Architect and implement a single platform for gathering operational and business telemetry data from our applications based on the three pillars of observability: logs, metrics, and traces.
What specific targets are we setting for ourselves for 2023?
- Integrate Google Analytics within all enterprise systems, where appropriate, to enable standardized metrics.
- Add support to IU Login and related systems for providing an anonymized analytics identifier attribute.
- Direct a Metrics task force to document and disseminate the standards for integrating applications into the Google Analytics 360 environment.
- Develop metrics dashboards that tie into the Primary Constituent Data Source (PCDS) and display distinct users per application and constituent group.
- Encourage outcome-based metric definition as a part of new bodies of work, with a focus on actionable analytics.
- Establish a working group with representatives for each project to collaborate on approaches to collecting and measuring outcomes.
- Identify a pilot project from each director’s area for which outcome-based metrics can be identified in advance.
- Organize an InfoShare that documents the successes, failures, and lessons learned from this effort.
- Architect and implement a single platform for gathering operational and business telemetry data from our applications based on the three pillars of observability: logs, metrics, and traces.
- Create an architecture for handling application metrics that aligns with the metrics portion of our Goals for Observability Infrastructure, using either Prometheus or Dynatrace.
- Implement a reusable baseline for visualizing application metrics for applications using phase 2 of the Enterprise Platform.
- Work with the Technology Advisory Group (TAG) to define a standard logging format for applications and implement a logging library that supports this format for applications using phase 2 of the Enterprise Platform.