Cloud Services

Cost & Asset Management

Managing GPD files
Published On May 16, 2024 - 10:39 AM

Managing GPD files

Learn to create GPD files to view specific Provider data.
When the provider's asset, cost, and utilization data is properly formatted and compressed into a Generatic Provider Data (GPD) file, that file can be uploaded and the data can be displayed in Cost & Asset Management (CAM), so that you can analyze the data within CAM. Create the folders and files, adhering to the folder structure, sample files, and required columns for the CSV files described below. Do not use Microsoft™ Excel. The file size limit is 250MB.
This document includes the following procedures to complete the creating and uploading GPD files:

Compressed folder structure

When a GPD file is uploaded, the code that imports the data looks for the folder structure in order to understand what data it is importing, this is why compress folder structure is important.
  • The compressed file should contain one provider_metadata.yml file in the root folder with one /multiple folder per period.
  • Folder names should be in YYYYMM format.
  • Each
    Period
    folder must have one /multiple folder per billing account.
  • Each
    Billing Account
    folder must have one CSV (Cost file) and can have additional folders for asset accounts.
    • Empty
      Billing Account
      folders are invalid. The Cost file must begin with the letter C.
  • If an
    Asset Account
    folder is present, it must contain an asset CSV and optionally an Util CSV.
    • Empty
      Asset Account
      folders are invalid. Asset files must begin with the letter A, and utilization files must begin with the letter U.
The sample file below has the date of 201802, but the current month is mandatory for the GPD file.
Sample folder structure:
201802 87349685 C_87349685_20180330202901.csv 87349686 C_87349686_20180330202901.csv 201803 87349685 11cbe031-a9d9-4e02-a25e-85901deb071d A_11cbe031-a9d9-4e02-a25e-85901deb071d_20180330202901.csv U_11cbe031-a9d9-4e02-a25e-85901deb071d_20180330202901.csv C_87349685_20180330202901.csv 87349686 0a2824fb-2386-44c2-b328-337299b2f7f1 A_0a2824fb-2386-44c2-b328-337299b2f7f1_20180330202901.csv U_0a2824fb-2386-44c2-b328-337299b2f7f1_20180330202901.csv C_87349686_20180330202901.csv
Asset Data:
This data populates the
Asset
tab of the CAM application.
The .csv files that start with the letter A provide a description of the fields for asset data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • providerAssetId*
  • providerName*
  • extractDate*
  • assetName*
  • assetStartDate
  • assetEndDate
  • assetCreatedBy
  • assetEndedBy
  • capacityText
  • assetType*
  • providerApiText
  • linkedAssetId
  • assetAccountId*
  • billingAccountId
  • url
  • providerRegionCode
  • operationalState*
  • orderId
  • tags
  • correlationAssetId
  • assetSource
  • attributes
Cost Data:
This data populates the
Cost
tab of the CAM application.
The .csv files that start with the letter C provide a description of the fields for cost data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • period
  • providerRecordId
  • extractDate*
  • serviceStartDate
  • serviceEndDate
  • invoiceStartDate
  • invoiceEndDate
  • providerInvoiceId
  • providerFinalBillFlag
  • providerName*
  • assetAccountId
  • billingAccountId*
  • providerRegion
  • assetName
  • providerAssetId
  • assetParentId
  • description
  • quantity
  • unitOfMeasure
  • providerCurrency
  • pcProviderUnitCost
  • pcProviderTotalCost
  • pcProviderTaxCost
  • pcCustomerUnitCost
  • pcCustomerTotalCost
  • customerCurrency
  • customerConversionRate
  • customerCurrencyDate*
  • orderId
  • providerServiceDescription
  • tags
  • assetType
  • correlationAssetId
  • attributes
  • secondaryCorrelationId
Utilization Data:
This data populates the utilization data for the assets and drives the policies that monitor the assets.
The .csv files that start with the letter U describe the data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • assetAccountId*
  • providerAssetId*
  • startTime
  • endTime
  • minValue
  • maxValue
  • avgValue*
  • unitOfMeasure*
  • correlationAssetId
  • ranges*
  • supportingMetrics
Provider Metadata:
This data is required for onboarding a new private cloud as a provider. This data assigns asset types to service categories at a high level and provides location data for the private cloud to populate the maps in CAM. The file must be named provider_metadata.yaml.

Creating a GPD file

To create a GDP file, complete the following steps:
  1. Build the entire data with no empty folders in your GPD structure and test, before you upload to a production system.
  2. Certain files are expected to be in certain folders. If those folders are empty, your GPD file cannot be validated.
  3. Specified the following information:
    • Add the current month in your compressed GPD file.
      The first time you ingest a provider, thirteen months of data is going to be ingested, after that, each ingestion includes only the current and any previous months.
    • Fill all the GPD fields, the mandatory fields are identifies with an asterix (*).
    • Verify that the date stamp format is accurate. Supported date format example: "2017-08-10 13:57:43.000".
    • Map your assets into the same categories as the ones chosen for the public clouds.
    • Save your file in the CSV format including CSV headers in the correct order and verify that the file containstext with comma-separated values, using a text editor. XLSX files are not supported in the compressed file.
    • Do not add multiple locations close to each other in the
      provider_metadata.yml
      , otherwise the usability of zooming in and out is compromised.
    • correlationAssetID
        must be a unique ID, unique providerAssetId listings is required per asset account basis. If you override the formula to have this field auto-generated, you must create your own unique correlation scheme.
    • UnitOfMeasure
      value is a percentage.
  4. Set operationalState field to
    On
    ,
    Off
    , or
    Decomissioned
    .
  5. Avoid using parent/child classes in private clouds. This includes the use of the
    AssetParentID
    field.
  6. Upload the file.

Uploading Generic Provider Data (GPD) files manually

Users authorized to set up providers and upload files can manually upload Generic Provider Data (GPD) files so that
Cost & Asset Management (CAM)
can analyze any provider's files. The files must be in compressed format and must contain properly formatted Asset, Cost, Utilization, and Provider data. The formatting ensures that CAM accurately displays the data in the Asset, Cost, and Utilization maps and tabs.
To learn more about navigating to the different services from each tenant, refer to Landing page navigation or Kyndryl Bridge Landing page navigation.

Uploading GPD files

To upload GPD files, complete the following steps:
  1. Navigate to
    CAM > Settings
    .
  2. Select
    Manual Ingestion
    from the list.
  3. Select the
    Custom Provider
    tab.
  4. Select the provider from the list for which you want to upload a GPD file.
  5. Click the
    Main
    menu on the right-side of the provider row and select
    Upload GPD Files
    .
  6. Drag and drop your GPD file into the Upload GPD Files screen. The file size limit is 250 MB.
  7. From the
    Custom Provider
    tab, select the provider and then click
    Start Enrichment
    .
CAM recognizes data in a 60-day window. Older data remains in the system, but new data is ingested for only the last 60 days. CAM requires a minimum of 60 days of data for proper utilization readings.

Uploading GPD files through S3 bucket

Users authorized to set up providers and upload GPD files, can do it via S3 bucket. The files must be in compressed format and must contain properly formatted Asset, Cost, Utilization, and Provider data. The provider is created through an API, for more information on this API, go to: Supported APIs The formatting ensures that CAM accurately displays the data in the Asset, Cost, and Utilization maps and tabs.
To upload GPD files using S3 bucket
, complete the following steps:
  1. Go to
    CAM
    , on settings you see the provider already created.
  2. Upload your GPD file using the S3 bucket.
  3. From the
    Custom Provider
    tab, select the provider and click
    Start Enrichment
    .
CAM recognizes data in a 60-day window. Older data remains in the system, but new data is ingested for only the last 60 days. CAM requires a minimum of 60 days of data for proper utilization readings.
Do you have two minutes for a quick survey?
Take Survey