Cloud Services

Cost & Asset Management

Managing GPD files
Published On Oct 18, 2024 - 1:26 PM

Managing GPD files

Learn how to create GPD files for Cost & Asset Management ingestion, enabling access to Provider data for analysis.
When the provider's asset, cost, and utilization data is properly formatted and compressed into a Generatic Provider Data (GPD) file, that file can be uploaded and the data can be displayed in Cost & Asset Management (CAM), so that you can analyze the data within CAM. Create the folders and files, adhering to the folder structure, sample files, and required columns for the CSV files described below. Do not use Microsoft™ Excel. The file size limit is 250MB.
This document includes the following procedures to complete the creating and uploading GPD files:

Compressed folder structure

When a GPD file is uploaded, the code that imports the data looks for the folder structure in order to understand what data it is importing, this is why compress folder structure is important.
  • The compressed file should contain one provider_metadata.yml file in the root folder with one /multiple folder per period.
  • Folder names should be in YYYYMM format.
  • Each
    Period
     folder must have one /multiple folder per billing account.
  • Each
    Billing Account
     folder must have one CSV (Cost file) and can have additional folders for asset accounts.
    • Empty
      Billing Account
       folders are invalid. The Cost file must begin with the letter C.
  • If an
    Asset Account
     folder is present, it must contain an asset CSV and optionally an Util CSV.
    • Empty
      Asset Account
       folders are invalid. Asset files must begin with the letter A, and utilization files must begin with the letter U.
The sample file below has the date of 201802, but the current month is mandatory for the GPD file.
Sample folder structure:
201802
    87349685
        C_87349685_20180330202901.csv
    87349686
        C_87349686_20180330202901.csv
201803
    87349685
        11cbe031-a9d9-4e02-a25e-85901deb071d
            A_11cbe031-a9d9-4e02-a25e-85901deb071d_20180330202901.csv
            U_11cbe031-a9d9-4e02-a25e-85901deb071d_20180330202901.csv
        C_87349685_20180330202901.csv
    87349686
        0a2824fb-2386-44c2-b328-337299b2f7f1
            A_0a2824fb-2386-44c2-b328-337299b2f7f1_20180330202901.csv
            U_0a2824fb-2386-44c2-b328-337299b2f7f1_20180330202901.csv
        C_87349686_20180330202901.csv
Asset Data:
 This data populates the
Asset
 tab of the CAM application.
The .csv files that start with the letter A provide a description of the fields for asset data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • providerAssetId*
  • providerName*
  • extractDate*
  • assetName*
  • assetStartDate
  • assetEndDate
  • assetCreatedBy
  • assetEndedBy
  • capacityText
  • assetType*
  • providerApiText
  • linkedAssetId
  • assetAccountId*
  • billingAccountId
  • url
  • providerRegionCode
  • operationalState*
  • orderId
  • tags
  • correlationAssetId
  • assetSource
  • attributes
Cost Data:
 This data populates the
Cost
 tab of the CAM application.
The .csv files that start with the letter C provide a description of the fields for cost data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • period
  • providerRecordId
  • extractDate*
  • serviceStartDate
  • serviceEndDate
  • invoiceStartDate
  • invoiceEndDate
  • providerInvoiceId
  • providerFinalBillFlag
  • providerName*
  • assetAccountId
  • billingAccountId*
  • providerRegion
  • assetName
  • providerAssetId
  • assetParentId
  • description
  • quantity
  • unitOfMeasure
  • providerCurrency
  • pcProviderUnitCost
  • pcProviderTotalCost
  • pcProviderTaxCost
  • pcCustomerUnitCost
  • pcCustomerTotalCost
  • customerCurrency
  • customerConversionRate
  • customerCurrencyDate*
  • orderId
  • providerServiceDescription
  • tags
  • assetType
  • correlationAssetId
  • attributes
  • secondaryCorrelationId
Utilization Data:
 This data populates the utilization data for the assets and drives the policies that monitor the assets.
The .csv files that start with the letter U describe the data. The following names are the field names for these files with asterixes (*) if they are mandatory:
  • assetAccountId*
  • providerAssetId*
  • startTime
  • endTime
  • minValue
  • maxValue
  • avgValue*
  • unitOfMeasure*
  • correlationAssetId
  • ranges*
  • supportingMetrics
Provider Metadata:
 This data is required for onboarding a new private cloud as a provider. This data assigns asset types to service categories at a high level and provides location data for the private cloud to populate the maps in CAM. The file must be named provider_metadata.yaml.

Creating a GPD file

To create a GDP file, complete the following steps:
  1. Build the entire data with no empty folders in your GPD structure and test, before you upload to a production system.
  2. Certain files are expected to be in certain folders. If those folders are empty, your GPD file cannot be validated.
  3. Specified the following information:
    • Add the current month in your compressed GPD file.
      The first time you ingest a provider, thirteen months of data is going to be ingested, after that, each ingestion includes only the current and any previous months.
    • Fill all the GPD fields, the mandatory fields are identifies with an asterix (*).
    • Verify that the date stamp format is accurate. Supported date format example: "2017-08-10 13:57:43.000".
    • Map your assets into the same categories as the ones chosen for the public clouds.
    • Save your file in the CSV format including CSV headers in the correct order and verify that the file containstext with comma-separated values, using a text editor. XLSX files are not supported in the compressed file.
    • Do not add multiple locations close to each other in the
      provider_metadata.yml
      , otherwise the usability of zooming in and out is compromised.
    • correlationAssetID
        must be a unique ID, unique providerAssetId listings is required per asset account basis. If you override the formula to have this field auto-generated, you must create your own unique correlation scheme.
    • UnitOfMeasure
       value is a percentage.
  4. Set operationalState field to
    On
    ,
    Off
    , or
    Decomissioned
    .
  5. Avoid using parent/child classes in private clouds. This includes the use of the
    AssetParentID
     field.
  6. Upload the file.

Uploading Generic Provider Data (GPD) files manually

Users authorized to set up providers and upload files can manually upload Generic Provider Data (GPD) files so that
Cost & Asset Management (CAM)
 can analyze any provider's files. The files must be in compressed format and must contain properly formatted Asset, Cost, Utilization, and Provider data. The formatting ensures that CAM accurately displays the data in the Asset, Cost, and Utilization maps and tabs.
To learn more about navigating to the different services from each tenant, refer to Landing page navigation or Kyndryl Bridge Landing page navigation.

Uploading GPD files

To upload GPD files, complete the following steps:
  1. Navigate to
    CAM > Settings
    .
  2. Select
    Manual Ingestion
     from the list.
  3. Select the
    Custom Provider
     tab.
  4. Select the provider from the list for which you want to upload a GPD file.
  5. Click the
    Main
     menu on the right-side of the provider row and select
    Upload GPD Files
    .
  6. Drag and drop your GPD file into the Upload GPD Files screen. The file size limit is 250 MB.
  7. From the
    Custom Provider
    tab, select the provider and then click
    Start Enrichment
    .
CAM recognizes data in a 60-day window. Older data remains in the system, but new data is ingested for only the last 60 days. CAM requires a minimum of 60 days of data for proper utilization readings.

Uploading GPD files through S3 bucket

Users authorized to set up providers and upload GPD files, can do it via S3 bucket. The files must be in compressed format and must contain properly formatted Asset, Cost, Utilization, and Provider data. The provider is created through an API, for more information on this API, go to: Supported APIs The formatting ensures that CAM accurately displays the data in the Asset, Cost, and Utilization maps and tabs.
To upload GPD files using S3 bucket
, complete the following steps:
  1. Go to
    CAM
    , on settings you see the provider already created.
  2. Upload your GPD file using the S3 bucket.
  3. From the
    Custom Provider
     tab, select the provider and click
    Start Enrichment
    .
CAM recognizes data in a 60-day window. Older data remains in the system, but new data is ingested for only the last 60 days. CAM requires a minimum of 60 days of data for proper utilization readings.
Do you have two minutes for a quick survey?
Take Survey