Cloud Services

ModernOps

Topology configuration
Published On May 16, 2024 - 1:59 PM

Topology configuration

Learn how to set up your system to be able to view networks using the Topology feature.
The Affinity feature, which is available for an additional fee, must also be set up by generating and integrating provider logs into the Kyndryl Modern Operations Applications system.

Configuring customer tenant applications

Selecting the NextGen application enables the Common Discovery. The Kyndryl Modern Operations Applications NextGen application must be enabled with the following services and feature flags:
  • Services:
    • Topology CD Client
    • Topology Affinity aws
    • Topology API Service
  • Feature flag:
    • Topology UI
The Common Discovery application must be enabled with the services and feature flags listed as follows:
  • Services:
    • AWS
    • Azure
    • IBM Cloud
  • Feature flag:
    • Common Inventory

Creating a connection

Before information can be integrated into the Topology feature, a connection to the provider must be created. For more information, see IAM cloud provider connection and follow the directions for the providers you want to use.

Enabling affinity for Topology

Affinity data can be integrated into the Topology feature to provide impact analysis calculations. The data must be extracted from flow logs provided by the provider. The procedure for generating and ingesting these logs varies by provider.

Enabling affinity for Amazon Web Services

To enable the affinity display in Topology, VPC logs and AWS CloudTrail need to be created for the specified provider.

Setting up the system for AWS

Complete these steps to meet the prerequisites for running the script to set up VPC logs and CloudTrail in AWS.
Make sure that the IAM role has the required access for the regions that will be used.
For AWS EKS only, if you performed these steps before the 221026 release, you must complete these steps again before the EKS data will be displayed.
  1. Clone the following repository that contains the script and instructions: https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup.
  2. Install dependencies by running the following command:
    python -m pip install boto3 / python3 -m pip install boto3
  3. Add these credentials to
    ~/.aws/credentials
    :
    [default] aws_access_key_id = YOUR_KEY aws_secret_access_key = YOUR_SECRET
  4. Set up a default region in
    ~/.aws/config
    :
    [default] region=us-east-1
  5. Create an S3 bucket on AWS with the name
    "mcmp-topology-flow-logs-YOUR_ACCOUNT_ID"
    in the region created in the previous step.

Creating flow logs for AWS

To view affinity, the VPC flow logs and AWS CloudTrail need to be created in AWS. These are created and named using a script. To use the script, complete the following steps.
Make sure that the AWS IAM role has the proper access and permissions to enable the regions to be used. 
Run all of the following scripts in the
aws-flowlog-automation-script
folder that you cloned from the https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup repository.
  1. In the
    regions.json
    file in the
    Inputs
    folder, select the regions for which to generate flow logs. Update this file at any time by running the
    "python updateRegions.py"
    command.
  2. Run
    "python getVPCIds.py"
    to create the
    AWS-Account-<Account_Number>-VpcList.json
    file in the
    Inputs
    folder that contains all the vpc-Ids for all regions listed in
    regions.json
    .
  3. Edit the
    AWS-Account-<Account_Number>-VpcList.json
    file and remove any vpcIds for which no flow logs are needed.
  4. Run
    "python createFlowLogs.py"
    to create the logging.

Enable CloudWatch Logs for SNS

If you are using Simple Notification Service (SNS), you need to enable the CloudWatch logs to enable affinity for it by completing these steps:
  1. Sign in to your Amazon SNS console.
  2. In the
    Navigation
    pane, select
    Topics
    .
  3. Select a topic and select
    Edit
    .
  4. On the
    Edit {Topic}
    page, expand the
    Delivery status logging
    section. 
  5. Select the protocol that you want to log delivery status with. Currently only
    Amazon SQS
    and
    AWS Lambda
    are supported.
  6. Set the
    Success sample
    rate to
    100%
  7. In the
    IAM roles
    section, select one of the following options:
    • Select
      Use existing service
      role and then select the IAM roles for successful and failed deliveries.
    • Select
      Create new service role
      and then create IAM roles for successful and failed deliveries.
  8. Provide Amazon SNS with write access to use the CloudWatch logs by selecting
    Allow
    .
  9. Click
    Save changes
    .

Deleting flow logs

If you need to delete the flow logs, complete these steps:
  1. Create a file with the same format as
    AWS-Account-<Account_Number>-VpcList.json
    containing all the VPC Ids for which to delete flow logs.
  2. Run
    "python deleteFlowLog.py"
    (or
    "python deleteFlowLog.py -y"
    to skip user conformation for each region), enter the absolute path to input file such as
    /home/User/MCMP-Topology/mcmp-topology-affinity-aws/Inputs/AWS-Account-<Account_Number>-VpcList.json
    , and press
    y
    .

Enabling affinity for IBM Cloud

To enable affinity for IBM Cloud, you must set up your system, create flow logs, and enable the Activity Tracker on IBM Cloud using the steps in this section.
Setting up your system for IBM Cloud
To set up your system to display affinity for IBM Cloud, complete the following steps:
  1. Clone the following repository that contains the script and instructions: https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup.
  2. Install dependencies by running the following command:
    python3 -m pip install -r requirements.txt
  3. Create directories as (
    .mcmp/.ibmcloud
    ) in your home directory.
  4. Create a text file named
    credentials
    if using windows, or
    credentials.txt
    if using another operating system in your
    .ibmcloud
    directory that includes the following credentials:
    ibm_account = {IBM Cloud account ID} ibm_apikey = {IBM Cloud API key} resource_id = {Resource ID of resource group where you want to create buckets}
Make sure to run scripts in the
ibmcloud-flowlog-automation-script
folder.
Creating flow logs for IBM Cloud
To view affinity, the VPC flow logs need to be created in IBM Cloud. These are created and named using a script. To use the script, complete the following steps.
Make sure that the IBM Cloud IAM role has the required access to manage resources. .
  1. Open the
    regions.json
    file in the
    Inputs
    folder and remove regions until only those that you want to create flow logs for remain.
    If you make a mistake, you can restore all regions by running the
    python updateRegions.py
    command.
  2. Run the
    python3 createBuckets.py
    command to create a cloud object storage instance with the name
    mcmp-topology-flow-logs-{Account_id}
    and buckets in all the regions specified in
    regions.json
    with the name
    mcmp-topology-flow-logs-{region}-{Account_id}
    . The buckets created have expiration rule of 1 day, so they are deleted 24 hours after being archived.
    If you are re-running this file, delete the already existing
    mcmp-topology-flow-logs-{Account_id}
    file before running the command.
  3. Run the
    python3 getVPCIds.py
    command to create an
    IBM-Account-{Account_Number}-VpcList.json
    file in the
    Inputs
    folder that contains the
    vpc-Ids
    for all regions mentioned in
    regions.json
    .
  4. Run the
    python3 createFlowLogs.py
    command to create VPC Flowlog collectors.
Enabling Activity Tracker for IBM Cloud
After you have created the flow logs, you need to link to them in Kyndryl Modern Operations Applications. To do so, complete these steps:
  1. Log in to Kyndryl Modern Operations Applications.
  2. Click the
    Open menu
    icon Open menu icon in the upper right and select
    Enterprise Marketplace
    Catalog
    .
  3. On the
    Catalog
    page, select
    Logging and Monitoring
    .
  4. Click
    IBM Cloud Activity tracker
    .
  5. For each region that you want to track, select the following parameters and then click
    Create
    :
    • Select the
      Region
      where you want to create the tracker.
    • For
      Plan
      , select
      7 day Event Search
      .
    • Select the
      I have read and agree to the following license agreements
      check box.
  6. Enable Activity Tracker for each bucket in IBM Cloud Console using these steps:
    1. Select
      Storage
      .
    2. Select the service instance with your bucket.   
    3. Select the bucket you want to enable.   
    4. In the navigation pane, click
      Configuration
      .   
    5. Click the
      Activity Tracker
      tab.   
    6. Click
      Create
      .   
    7. If you already have an instance of IBM Cloud Activity Tracker, select it here. If not, select the appropriate configuration and click
      Create
      .   
    8. Select
      Track data events
      and select
      read and write
      in the field below.   
    9. Click
      Save
      .
  7. Ensure that you have a cloud object storage instance with the name
    mcmp-topology-flow-logs-
    . If you do not, create an instance with the name
    mcmp-topology-flow-logs-{AccountNumber}
    .
  8. Create a bucket in the cloud object storage instance that you just identified to archive all the activity tracker events in by completing these steps:
    1. Click
      Create bucket
      in your cloud object storage instance.
    2. For
      Name
      , enter
      mcmp-topology-activity-tracker-{AccountNumber[0:15}
      (first 15 digits of your account number).
    3. Create the bucket in the
      us-east
      region and select storage class as
      smart tier
      .
    4. Go to the
      Advanced Configurations
      section and click
      Expiration
      .
    5. Click
      Add
      .
    6. In the
      Add expiration rule
      window, click
      Simple
      .
    7. Enable
      Expiration rule
      , enter
      1
      in the
      Current version expiration
      field, and click
      Save
      .
    8. Click
      Create bucket
      .
    9. Select the bucket that you just created and copy the following information:
      • The private endpoint
      • The apikey
      • The resource_instance_id
  9. Configure archiving of your IBM Cloud Activity Tracker instance into a COS bucket by completing these steps for each of them:
    1. On the
      Dashboard
      , select
      Services and software
      .
    2. Click
      Activity tracker
      .
    3. Click
      Open dashboard
      .
    4. Click the
      Settings
      icon and select
      Archiving
      Manage
      .
    5. Click
      Enable archiving
      .
    6. Select
      IBM Cloud Object Storage
      .
    7. Enter the following parameters and then click the
      Service Credentials
      tab:
      • Bucket:
        Enter the name of the bucket.
      • Endpoint:
        The private endpoint of the bucket.
  10. Enter the following information from your service credential (create a new one if needed) and then click
    Save
    :
    • API Key:
      The apikey value from your service credential.
    • Instance ID:
      The resource_instance_id from your service credential.
  11. Click
    Save
    .
Do you have two minutes for a quick survey?
Take Survey