Cloud Services

ModernOps

Topology feature
Published On May 16, 2024 - 1:59 PM

Topology feature

Learn about the Topology feature that provides a graphical representation of dependencies between the applications and infrastructure in a system and optional impact analysis.
This representation provides a clear view into how the system works, providing insight into dependencies between resources, allowing analysis of potential outage impacts and blast radius. The Topology map is created using the following information:
  • Resources connected:
    How resources are connected through deployment and association data derived from the public cloud.
  • Affinity:
    How resources are connected based on communication. Affinity data is available for an extra fee. Affinity is based on how many times the resources communicated in the last 30 days. This information is determined using public cloud network flow logs. Before affinity can be viewed, flow logs must be generated and imported.
Affinity helps determine how an outage will affect other nodes because the impact will be greater on a system that communicates with the nonfunctional node more, also known as blast radius. Affinity data is updated on an hourly basis.
The maximum number of nodes (parent and child) that can be displayed at one time is 10000. If this number is exceeded, apply filters to reduce it.

Prerequisites

Before the Topology feature can be used, the following prerequisites must be met:
  • Python 3.8 or later.
  • Dependencies are installed by running the
    # python3 -m pip install -r requirements.txt
    command.
  • Provider accounts are set up using the connection functionality.
  • To display affinity data, flow logs must be created. Enabling affinity requires additional resources from the provider such as S3 and CloudTrail, and therefore will generate additional costs.

Roles required

Accessing the Topology view requires the
Viewer
role.

Reaching the Topology page

The Topology view is a common service that currently can be reached from these places:
From the main console:
Click the
Open menu
icon and select
Topology
View Topology
. In this case, the
Topology
window is blank when it opens.
From the CCM dashboard:
On the page for the cluster you want to view, click the
Open menu
icon and select
Topology
View topology
to open the Topology map for that cluster. You can use the
Edit filters
feature to navigate to other views in Topology as normal.
From the Compliance SecOps dashboard:
Click the
Topology
icon to open the Topology map for that resource. You can use the
Edit filters
feature to navigate to other views in Topology as normal.

Navigating in the Topology window

The
Topology
window shows resources in a series of nested boxes based on the hierarchy (Provider, Account, and supported resources details) that shows how those resources are interconnected. Click the boxes to show a detailed view of the nodes in that cluster and how they are connected.
Communications between resources in the diagram is indicated by lines that display the number of times the resources have communicated.
Communication analysis requires affinity data for the nodes in question to be available. A + icon indicates that some extra nodes are displayed to show the hierarchy that are not part of the results, there are shared or other related resources that have an affinity to the resources in that category, or a combination of the two.
The page defaults to the
Application
view in the top pane. You can change the view by clicking
View by
in the upper right and selecting from these choices:
  • Application:
    Shows resources by application. Click an application to view its resources. The resources are grouped by application and shared resources unless there are no resources exclusive to the application, in which case they are grouped by shared resources and other related resources.
  • Infrastructure:
    Displays the hierarchy of the resources in the system. Click the provider to show accounts, accounts to show regions, and regions to show networks within that region. Click the network to display its resources.
  • Cluster:
    Shows resources by cluster (Kubernetes. EKS, AKS, IKS, ECS, and so on). Click a cluster to view its resources. The resources are grouped by namespace.
The
Group by
drop-down menu currently only has one option per
View by
option and therefore does not need to be set.
The top pane lists all of the resources of the type selected along with the number of resources associated with them. The system automatically shortens the resource names as needed. Hover over the resource to see the complete name. To view the diagram in the entire window, click the
Minimize
icon in the upper right (–) to hide the top pane.
All of the resources and resource groups can be freely moved around by dragging and dropping. In addition, you can zoom in and out using the mouse wheel/pad. The location of the cursor is the center of the zoom.
The resources are grouped by their association. The resource that is selected, in this example an application, is shown with a box that contains all services directly linked to that resource. Any association within the main one, such as the VPC in this example, is shown as a box within the main box. In addition to the services directly linked to the resource, the diagram also shows any resources that are shared with other resources in the same category in a
Shared resources
box.
If the application has no resources that are exclusive to it, you will see a
Shared resources
box and an
Other related resources
box. The
Other related resources
box lists resources that are related to the ones in the
Shared resources
box.
Association is determined by the tagging of the resources. resources that have, for example, a single Application tag will appear only in that application's diagram. resources that have multiple Application tags will be displayed as shared resources in the diagrams for all of those applications.
Tagging for applications only uses the following key names. If your application tags use a key name other than these, you can add that key name with an API.
  • app
  • appName
  • application
The services are shown in the following colors based on their status:
  • Green:
    Active
  • Yellow:
    Inactive
  • Red:
    Critical
  • Gray:
    No status available, such as for subnets
If affinity data is available, the number of communications between each service in the last 30 days are displayed on the line connecting them.

Managing filters

To add more criteria, click the
Edit filters
icon in the upper left. In the
Filters
window, select from the following fields to generate filter criteria and click
Apply
. As you select filters, the other available filters will be filtered to show only things that match what you have already selected. For example, selecting Azure will mean that only Azure accounts will be available to be selected.
Selections within a category, such as Provider, will return results that meet any of your selections (OR operand). But only selections that meet all the categories selected will be returned (AND operand). Filters function on both the
Navigation
and the
Resources
pane.
  • Choose the Provider:
    Select the provider to get results for:
    • AWS:
      Amazon Web Services
    • Azure:
      Microsoft Azure
    • ibmcloud:
      IBM Cloud
  • Choose the Account:
    Select one or more accounts for the selected provider.
  • Choose the Region:
    Select one or more regions to filter by.
  • Choose the Network:
    Select a network from the selected region.
  • Choose the Cluster:
    Select one or more clusters.
To remove a filter, click the
x
on the filter.

Viewing impact analysis data

Impact analysis requires affinity data for the nodes.
Impact analysis provides you with information about how the loss of a specified node will affect all other connected nodes. The score is expressed in terms of a percentage change.
This information helps with the following analytics:
  • Disaster recovery planning with improved RPO/RTO
  • Improved SLA management with improved availability and uptime
  • Improved cost management with targeted optimization for high availability (HA) configurations
  • Significant cost reduction with improved uptime and availability with optimized H/W
Retrieving the impact analysis scores currently can be done for a single node using an API or the UI. The following procedures show the information for the user interface.
To view the impact analysis, complete the following steps: Click the
Open menu
icon in the upper left and select
Topology
View Topology
.
  1. In the
    Topology
    window, use the filters as needed to locate the node that you want to get impact analysis data for.
  2. Right click the node that you want to analyze and select
    View Impact Analysis
    .
An
Impact Analysis
window is displayed for the node that includes all of the nodes that are affected. Click the legend or an arc on the chart to display only the nodes in that impact category. You can use the
Search
field to locate specific nodes in the list.
The
Impact Analysis
window provides a graph showing vulnerability percentages.
It also provides a Resource List that provides the following information for each node:
  • Provider
  • Account Number
  • Resource ID
  • Impact Percentage

Services supported by provider

The tables in this section show what kinds of services are displayed in Topology for each provider and whether affinity is supported for that service.
Affinity must also be set up for each provider before it can be used. For more information, see Topology configuration.
Amazon Web Services (AWS)
Service Name
Topology
Affinity
Affinity Additional Processing
Notes
Elastic Compute Cloud (EC2)
Supported
Supported
-
VPC Flow Logs is used to get the affinity.
Elastic Kubernetes Service (EKS)
Supported
Not Supported
-
-
Elastic Kubernetes Service - Namespace
Supported
Not Supported
-
-
Elastic Kubernetes Service - Pod
Supported
Supported
KB Affinity
VPC Flow Logs is used to get the following affinity types:
  • Pod to pod
  • Node to pod
Elastic Container Registry (ECR)
Supported
Supported
Affinity is obtained through CloudTrail management events for actions such as:
  • Describing images
  • List images
Virtual Private Cloud (VPC)
Supported
Not Supported
-
VPC affinity is applicable for the resources inside the VPC.
Elastic Load Balancing
Supported
Supported
At Discovery: Obtain IP Address using CLI - Describe Network Interface
VPC Flow Logs is used to get the affinity between ELB and target instance.
ElastiCache
Supported
Supported
At Discovery: Obtain IP Address using Network lookup - Describe Network Interface
VPC Flow Logs is used to get the affinity. It is achieved through Private IP address.
Relational Database Service (RDS)
Supported
Supported
At Discovery: Obtain IP Address using CLI - Describe Network Interface
VPC Flow Logs is used to get the RDS affinity. Supported engines are:
  • MySql
  • PostgresQL
  • MariaDB
Dynamo DB
Supported
Supported
Affinity query on: Table name
CloudTrail is used to get the DynamoDB table level affinity.
Simple Storage Service (S3)
Supported
Supported
-
CloudTrail is used to get the S3 bucket level affinity.
Elastic FileSystem (EFS)
Supported
Supported
At Discovery: Obtain IP Address using CLI - Describe Mount
VPC Flow Logs is used to get the affinity.
Key Management Service (KMS)
Supported
Supported
-
CloudTrail management events are used to get the KMS affinity.
Simple Queue Service (SQS)
Supported
Not Supported
Provider Limitation
Only management events, like create/delete are available so no affinity is possible.
Elastic Block Store (EBS)
Supported
Not Supported
Provider Limitation
For affinity, AWS does not log read/write operations performed on EBS volume from an EC2 instance due to security and privacy reasons.
Elastic Container Service (ECS)
Supported
Not Supported
-
-
ECS - Service
Supported
Not Supported
-
-
ECS - Task
Supported
Supported
-
-
Simple Notification Service (SNS)
Supported
Supported
-
-
Lambda
Supported
Not Supported
Provider Limitation
-
Reserved VMS Instances
Supported
Not Supported
-
-
Batch Compute Environment
Supported
Not Supported
-
-
Batch Job Queue
Supported
Not Supported
-
-
Redshift
Supported
Not Supported
-
-
Dax
Supported
Not Supported
-
-
Reserved Instances DB
Supported
Not Supported
-
-
Glacier
Supported
Not Supported
-
-
Backup
Supported
Not Supported
-
-
Direct Connect
Supported
Not Supported
-
-
API Gateway
Supported
Not Supported
-
-
Route53Resolver
Supported
Not Supported
-
-
Route53 Hosted Zone
Supported
Not Supported
-
-
EC2 Security Groups
Supported
Not Supported
-
-
Elastic IP
Supported
Not Supported
-
-
Workmail
Supported
Not Supported
-
-
Polly
Supported
Not Supported
-
-
Comprehend
Supported
Not Supported
-
-
Transcribe
Supported
Not Supported
-
-
Rekognition
Supported
Not Supported
-
-
Lex
Supported
Not Supported
-
-
Sagemaker
Supported
Not Supported
-
-
Macie
Supported
Not Supported
-
-
Stack
Supported
Not Supported
-
-
AutoScaling
Supported
Not Supported
-
-
Cloud Watch
Supported
Not Supported
-
-
Cloud Trail
Supported
Not Supported
-
-
Application Auto Scaling
Supported
Not Supported
-
-
Auto Scaling Plans
Supported
Not Supported
-
-
Resource Group
Supported
Not Supported
-
-
Config
Supported
Not Supported
-
-
ACM Private CA
Supported
Not Supported
-
-
Cloudwatch Logs
Supported
Not Supported
-
-
Events
Supported
Not Supported
-
-
Performance Insights
Supported
Not Supported
-
-
SSM
Supported
Not Supported
-
-
Pricing
Supported
Not Supported
-
-
Certificate Manager
Supported
Not Supported
-
-
License Manager
Supported
Not Supported
-
-
Cost Explorer
Supported
Not Supported
-
-
Service Catalog
Supported
Not Supported
-
-
Cost and Usage Report
Supported
Not Supported
-
-
Budgets
Supported
Not Supported
-
-
Trusted Advisor
Supported
Not Supported
-
-
Support
Supported
Not Supported
-
-
FMS
Supported
Not Supported
-
-
Reservation Purchase Recommendation
Supported
Not Supported
-
-
Rightsizing Recommendations
Supported
Not Supported
-
-
States
Supported
Not Supported
-
-
CloudHSM
Supported
Not Supported
-
-
Secrets
Supported
Not Supported
-
-
Directory Service (DS)
Supported
Not Supported
-
-
Security Token Service (STS)
Supported
Not Supported
-
-
WAF
Supported
Not Supported
-
-
Inspector
Supported
Not Supported
-
-
Shield
Supported
Not Supported
-
-
WAF - regional
Supported
Not Supported
-
-
SecurityHub
Supported
Not Supported
-
-
Organizations
Supported
Not Supported
-
-
IAM Accesskey
Supported
Not Supported
-
-
IAM Profile
Supported
Not Supported
-
-
IAM Groups
Supported
Not Supported
-
-
IAM Policies
Supported
Not Supported
-
-
IAM Roles
Supported
Not Supported
-
-
Identity And Access Management
Supported
Not Supported
-
-
GuardDuty
Supported
Not Supported
-
-
Kinesis
Supported
Not Supported
-
-
Kafka
Supported
Not Supported
-
-
Athena
Supported
Not Supported
-
-
Firehose
Supported
Not Supported
-
-
ElasticMapReduce
Supported
Not Supported
-
-
Glue
Supported
Not Supported
-
-
Kinesis Analytics
Supported
Not Supported
-
-
Workspaces
Supported
Not Supported
-
-
Code Deploy
Supported
Not Supported
-
-
Code Pipelines
Supported
Not Supported
-
-
CodeCommit
Supported
Not Supported
-
-
CodeBuild
Supported
Not Supported
-
-
X-Ray
Supported
Not Supported
-
-
CodeCommitRepository - CodeRepo
Supported
Not Supported
-
-
Elastic Search
Supported
Not Supported
-
-
IBM Cloud
Service Name
Topology
Affinity
Affinity Additional Processing
Notes
Virtual Server for VPC
Supported
Supported
Affinity is supported through flow logs.
Load Balancers for VPCs
Supported
Supported
Affinity is supported through flow logs.
Virtual Private Cloud (VPC)
Supported
Not Supported
VPC affinity is applicable for the resources inside the VPC.
Object Storage - Buckets
Supported
Supported
Affinity is supported through Cloud Activity Tracker.
Kubernetes Clusters (VPC)
Supported
Not Supported
-
Only resources can bee seen without affinity between pods.
Kubernetes Namespace
Supported
Not Supported
-
Only resources can bee seen without affinity between pods.
Kubernetes Pod
Supported
Not Supported
Only resources can bee seen without affinity between pods.
VirtualServer (Classic)
Supported
Not Supported
Affinity cannot be supported as no logs are available.
BareMetalServer (Classic)
Supported
Not Supported
Affinity cannot be supported as no logs are available.
BareMetalServer for VPC
No CD Support
Not Supported
Common Discovery does not support BareMetalServer for VPC.
Messages for RabbitMQ
Supported
Not Supported
Insufficient logs for affinity.
Databases for MongoDB
Supported
Not Supported
Insufficient logs for affinity.
Databases for Redis
Supported
Not Supported
Insufficient logs for affinity.
Microsoft Azure
Service Name 
Topology
Affinity
Affinity Additional Processing
Notes
Azure Kubernetes Cluster
Supported
Not Supported
-
Only resources can be seen without affinity between pods.
Azure Kubernetes Namespace
Supported
Not Supported
-
Only resources can be seen without affinity between pods.
Azure Kubernetes Pod
Supported
Not Supported
Only resources can be seen without affinity between pods.
Demo Cloud
Service Name 
Topology
Affinity
Affinity Additional Processing
Notes
Virtual Machine
Supported 
Not Supported 
-
Only resources can bee seen without affinity between pods. 
PostgreSQL
Supported
Not Supported
-
Only resources can bee seen without affinity between pods.
Kubernetes
Supported
Not Supported
Only resources can bee seen without affinity between pods.
Do you have two minutes for a quick survey?
Take Survey