Configuring DORA metrics
Configuring DORA metrics for data collection helps you measure and monitor your team’s software development performance. To collect data for DORA metrics:
- Configure data collection for Deployment Frequency, Change Lead Time, Change Failure Rate and Time to Restore Service.
- Integrate code repositories, such as GitHub or GitLab, with data collection and visualization tools.
- Use tools such as Four Keys to set up a data ingestion pipeline and consolidate the collected data into a dashboard.
- Monitor DORA metrics and trends over time.
- Use the collected data to evaluate your team’s operational efficiency and performance.
Adding deployment information
Deployment data can be obtained in one of two ways:
- Adding the connection where release info is available (tools such as, ArgoCD) and add the service configuration from the tools configuration page.
- Posting the Record from the Bring Your Own Deployment.
When adding a connection, the incident Host URL fields should be populated with your ServiceNow instance host URL.
To post the record from the BYO deployment, refer to the configure page, here: Bring your own deployBring your own deploy.
Ensure the following:
- The fields Environment and Release match the provided change request.
- The itsm_host and endpoint hostname match the ServiceNow instance url.
If Posting the data by Bring Your Own Deployment set isProduction to true.
Deployment frequency
This section describes how to configure data collection for Deployment frequency:
Connect to your release info source
You can connect existing tools, where release information is available, and add the service config from the tool's configuration page. For instance: If you are using ArgoCD, the service config should be created with the details available at the documentation page for configuring that service, in this case, ArgoCD. Instructions for configuring all supported services are available in the DevOps Intelligence documentation (Kyndryl Docs) by clicking the Supported tools and integration tile
Remember to populate the Incident Host URL fields with your ServiceNow instance host URL.
Use the BYO deployment API
Alternatively, you can post the record from the BYODeployment using supported APIs. Ensure that the fields environment and release match those provided in the change request. Also, make sure that itsm_host and 'endpoint hostname' match your ServiceNow instance URL. If you post the data via BYODeployment, set isProduction to true.
Change lead time (CLT)
This section presents configuring Change lead time from BitBucket and GitHub, GitLab, and Azure DevOps.
Atlassian BitBucket:
- BitBucket Cloud classifies issues as 'proposal' for epic tasks and 'task' for user stories.
- Link tasks to their proposals using a specific format in the description: Proposal:<space> followed by the full URL link to the proposal. Make sure to create a PR for each task when merging code. The PR body should contain the full link to the task:
- full URL link to proposal'Once done, All tasks will be connected to that proposal. When working on any task, create a PR while merging code.PR body should have a full link to the task link. The issues will start getting PR data and what all PRs are there for it. The same will be flown to Proposal. If Cherry Pick PR, the PR body should have a full link to the task and also a link to the default branch PR full link. So that we can track cherry-pick PR for which PR in master.
- Enable protection rules for release branches to prevent creation/deletion.
- Provide a release format prefix, e.g., 'release/' (BitBucket's default format).
For more information on BitBucket configuration, please see: Atlassian Bitbucket, Bitbucket Cloud and Bitbucket Server.
GitHub and GitLab:
- On GitHub and GitLab, use the 'Epic' label for epic tasks and 'User Story' for user stories.
- Link user stories to their epic tasks using a specific format in the description:
- Once done, All stories will be connected to that Epic
- When working on any story, make sure you create a PR for the same while merging code.
- PR body should have a full link to the user story task.
- The issues will start getting PR data and what all PRs are there for it and flown to Epic.
- If Cherry Pick PR, the PR body should have a full link to the user story task and also a link to the default branch PR full link. So that we can track cherry-pick PR for which PR in master.
- For Cherry Pick PRs, include the full link to the user story task and the default branch PR full link in the PR body for tracking.
- Enable protection rules for release branches.
- Provide a release format prefix, e.g., 'release-20' (all releases should start with this).
For more information on GitHub and GitLab configuration, please refer to the following links: GitHub and GitHub enterprise configuration and GitLab.
Azure DevOps:
- In Azure DevOps, create feature and user story tasks for handling epics.
- Connect user stories to their epic tasks using a specific format in the description: Feature:<space> followed by the full URL link to the feature. Create a PR for each story when merging code and include the full link to the user story in the PR body. The Description of the user story should start with:
- Once done, All stories will be connected to that Feature.
- When working on any story, create a PR while merging code. PR body should have a full link to the user story task. The issues will start getting PR data and what all PRs are there for it and the same will be flown to Epic.
- If Cherry Pick PR, the PR body should have a full link to the user story task and also a link to the master branch PR full link, so that we can track cherry-pick PR for which PR in master
- Enable protection rules for release branches.
- Provide a release format prefix, e.g., 'release-20' (all releases should start with this).
For more information on Azure DevOps configuration, please see: Configure Azure boards and repositoryAzure Boards and Azure Repo configuration
Once the release cut is done and the release branch is created, it automatically links all releases with User Stories. The Epic Change lead time calculation is determined from the first User Story's first PR merge date to the Last User Story's last merge and release deployment time.
To use Jira as an issue-tracking tool and another source control tool, you must provide the SCM Host URL when configuring the Jira Connection. Note that if you specify a separate source tool in the SCM Host URL, a separate IAM connection for that specific source tool should also be established.
Change failure rate (CFR)
For consistent data collection, ensure you format the 'short description' field in the change request as follows:
- '***deployment to ENVIRONMENTNAME *** of release RELEASENAME ***'
For "deployment to" and "of release" are required keywords. The placeholders marked with three asterisks (***) should be replaced with appropriate values.
The following image depicts the example of a short description field in a change request. Data will be visualized when incidents are in a closed state.
The Incident should also be mapped to the initiating ChangeRequest by giving the change request ID in the Related Records section Change Request and Cause By Change field.
Linking incidents with change requests
MTTR calculations are performed for Incidents that are either in a closed or resolved state. All the incidents that are moved to a resolved or closed state within 180 days are synched. While creating Incidents, the Configuration Item field should be populated, which will be mapped to TechnicalServiceName in DevOps Intelligence.
If the Configuration Item field is not populated, then by default, No CI will be marked for such incidents and reflected in the Technical Service column of the MTTR details page.
To correlate incidents with their respective Change Requests, provide the Change Request ID in the Related Records section under the Change Request and Cause By Change fields when creating an incident. The following image presents sample information of ServiceNow.
Mean time to recovery
Mean time to recovery calculations are performed for incidents in a closed or resolved state. When creating incidents, ensure that the Configuration Item field is populated. This field maps to the Technical Service Name in DI. In cases where the Configuration Item field is not populated, No CI will be labeled by default for such incidents, which will then be reflected in the Technical Service column of the MTTR details page.