Kyndryl Bridge

Experience Kyndryl Bridge

Topology: Set up
Published On Jun 13, 2024 - 3:51 PM

Topology: Set up

This page describes how to set up your system to be able to view networks using the Topology Common service.

Creating a connection

Before information can be integrated into the Topology service, a connection to the provider must be created. For more information, see Connections and follow the directions for the providers you want to use.
Currently, the Topology Core service is only available for Amazon Web Services (AWS) and IBM Cloud.
Affinity data can be integrated into the Topology service to provide impact analysis calculations. The data must be extracted from flow logs provided by the provider. The procedure for generating and ingesting these logs varies by provider.

Enabling affinity for Topology for Amazon Web Services (AWS)

To enable the affinity display in Topology, VPC logs and AWS CloudTrail need to be created for the specified provider.
Step: 1 Setting up the system for AWS
Complete these steps to meet the prerequisites for running the script to set up VPC logs and CloudTrail in AWS.
Make sure that the IAM role has the required access for the regions that will be used.
  1. Clone the following repository that contains the script and instructions: https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup.
  2. Install dependencies by running the following command: 
    python -m pip install boto3 / python3 -m pip install boto3
  3. Add these credentials to ~/.aws/credentials:
    [default]   aws_access_key_id = YOUR_KEY    aws_secret_access_key = YOUR _SECRET
  4. Set up a default region in ~/.aws/config:
    [default]   region=us-east-1
  5. Create an S3 bucket on AWS with the name "mcmp-topology-flow-logs-YOUR_ACCOUNT_ID" in the region created in the previous step.
Step 2: Creating flow logs for AWS
Make sure that the AWS IAM role has the proper access and permissions to enable the regions to be used. For more information, see Understanding IAM.
To view affinity, the VPC flow logs and AWS CloudTrail need to be created in AWS. These are created and named using a script. To use the script, complete the following steps.
  1. Run all the following scripts in the
    aws-flowlog-automation-script
    folder that you cloned from the https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup repository.
  2. In the regions.json file in the inputs folder, select the regions for which to generate flow logs. Update this file at any time by running the
    python updateRegions.py
    command.
  3. Run
    python getVPCIds.py
    to create the 
    AWS-Account-<Account_Number>-VpcList.json
    file in the Inputs folder that contains all the vpc-Ids for all regions listed in regions.json.
  4. Edit the AWS-Account
    -<Account_Number>-VpcList.json
    file and remove any vpcIds for which no flow logs are needed.
  5. Run
    python createFlowLogs.py
    to create the logging.
Step 3: Enable CloudWatch Logs for SNS
If you are using Simple Notification Service (SNS), you need to enable the CloudWatch logs to enable affinity for it by completing these steps:
  1. Sign into your Amazon SNS console.
  2. In the
    Navigation
    pane, select
    Topics
    .
  3. Select a topic and select
    Edit
    .
  4. On the Edit {Topic} page, expand the Delivery status logging section.
  5. Select the protocol that you want to log delivery status with. Currently only Amazon SQS and AWS Lambda are supported.
  6. Set the Success sample rate to
    100%
    .
  7. In the IAM roles section, select one of the following options:
    • Select
      Use existing service role
      and then select the
      IAM roles
      for successful and failed deliveries.
    • Select
      Create new service role
      and then create
      IAM roles
      for successful and failed deliveries.
  8. Provide Amazon SNS with write access to use the CloudWatch logs by selecting
    Allow
    .
  9. Click
    Save changes
    .
Step 4: Deleting flow logs
If you need to delete the flow logs, complete these steps:
  1. Create a file with the same format as
    AWS-Account-<Account_Number>-VpcList.json
     containing all the VPC Ids for which to delete flow logs.
  2. Run
    python deleteFlowLog.py
    or
    python deleteFlowLog.py -y
    (to skip user conformation for each region), enter the absolute path to input file such as
    /home/User/MCMP-Topology/mcmp-topology-affinity-aws/Inputs/AWS-Account-<Account_Number>-VpcList.json
    , and press
    y
    .

Enabling affinity for IBM Cloud

To enable affinity for IBM Cloud, you must set up your system, create flow logs, and enable the Activity Tracker on IBM Cloud using the following steps.
Step 1: Setting up your system for IBM Cloud
To set up your system to display affinity for IBM Cloud, complete the following steps:
  1. Clone the following repository that contains the script and instructions: https://github.kyndryl.net/MCMP-Topology/mcmp-topology-affinity-setup.
  2. Install dependencies by running the following command:
    python3 -m pip install -r requirements.txt
  3. Create directories as (.mcmp/.ibmcloud) in your home directory.
  4. Create a text file named credentials if using windows, or credentials.txt if using another operating system in your .ibmcloud directory that includes the following credentials:
    ibm_account =  {IBM Cloud account ID}  ibm_apikey = {IBM Cloud API key} resource_id =  {Resource ID of resource group where you want to create buckets}
Make sure to run scripts in the ibmcloud-flowlog-automation-script folder.
Step 2: Creating flow logs for IBM Cloud
Make sure that the IBM Cloud IAM role has the required access to manage resources. For more information, see IBM Cloud IAM roles.
To view affinity, the VPC flow logs need to be created in IBM Cloud. These are created and named using a script. To use the script, complete the following steps.
  1. Open the
    regions.json
    file in the Inputs folder and remove regions until only those that you want to create flow logs for remain. To restore all regions (due to an error), run the
    python updateRegions.py
    command.
  2. Run the
    python3 createBuckets.py
    command to create a cloud object storage instance with the name mcmp-topology-flow-logs-{Account_id} and buckets in all the regions specified in regions.json with the name mcmp-topology-flow-logs-{region}-{Account_id}. The buckets created have an expiration rule of 1 day, so they are deleted 24 hours after being archived.  If you are re-running this file, delete the already existing mcmp-topology-flow-logs-{Account_id} file before running the command.
  3. Run the python3 getVPCIds.py command to create an IBM-Account-{Account_Number}-VpcList.json file in the Inputs folder that contains the vpc-Ids for all regions mentioned in regions.json.
  4. Run the python3 createFlowLogs.py command to create VPC Flowlog collectors.
Step 3: Enabling Activity Tracker for IBM Cloud
After you have created the flow logs, you need to link to them. To do so, complete these steps:
  1. Go to
    My Console
    landing page.
  2. Select
    Logging and Monitoring
    .
  3. Click
    IBM Cloud Activity tracker
    .
  4. For each region that you want to track, select the following parameters and then click
    Create
    :
    1. Select the region where you want to create the tracker.
    2. For plan, select
      7 day Event Search
      .
    3. Check the box: 
      I have read and agree to the following license agreements
      .
  5. Enable activity tracker for each bucket in IBM Cloud Console using these steps:
    1. Select
      Storage
      .
    2. Select the service instance with your bucket.
    3. Select the bucket you want to enable.
    4. In the navigation pane, click
      Configuration
      .
    5. Click the
      Activity Tracker
      tab.
    6. Click
      Create
      . If you already have an instance of IBM Cloud Activity Tracker, select it here. If not, select the appropriate configuration and click
      Create
      .
    7. Select
      Track data
      events and select read and write in the field below.
    8. Click
      Save
      .
  6. Ensure that you have a cloud object storage instance with the name mcmp-topology-flow-logs-. If you do not, create an instance with the name mcmp-topology-flow-logs-{AccountNumber}.
  7. Create a bucket in the cloud object storage instance that you just identified to archive all the activity tracker events in by completing these steps:
    1. Click
      Create bucket
      in your cloud object storage instance.
    2. For Name, enter topology-activity-tracker-{AccountNumber[0:15} (first 15 digits of your account number).
    3. Create the bucket in the us-east region and select
      storage
      class as smart tier.
    4. Go to the
      Advanced Configurations
      section and click
      Expiration
      .
    5. Click
      Add
      .
    6. In the
      Add expiration
      rule window, click
      Simple
      .
    7. Enable
      Expiration rule
      , enter 1 in the Current version expiration field, and click
      Save
      .
    8. Click
      Create bucket
      .
    9. Select the bucket that you just created and copy the following information:
      1. The private endpoint
      2. The apikey
      3. The resource_instance_id
  8. Configure archiving of your IBM Cloud Activity Tracker instance into a COS bucket by completing these steps for each of them:
    1. On the
      Dashboard
      , select
      Services and software
      .
    2. Click
      Activity tracker
      .
    3. Click
      Open dashboard
      .
    4. Click the Settings icon and select
      Archiving→ Manage
      .
    5. Click
      Enable archiving
      .
    6. Select
      IBM Cloud Object Storage
      .
    7. Enter the following parameters and then click the Service Credentials tab:
      1. Bucket: Enter the name of the bucket.
      2. Endpoint: The private endpoint of the bucket.
  9. Enter the following information from your service credential (create a new one if needed) and then click
    Save
    :
    1. API Key: The apikey value from your service credential.
    2. Instance ID: The resource_instance_id from your service credential.
  10. Click
    Save
    .
Do you have two minutes for a quick survey?
Take Survey