Oracle RAC with Dataguard
Overview
This document outlines the technical solution guide integration for integrating the Kyndryl Bridge AIOps-based Resiliency platform. It provides multiple IT applications and databases business continuity and Cyber Resiliency option to manage from a single platform as a service.
Kyndryl Bridge (AIOps) platform will automate various tasks and configurations, improving efficiency, scalability, and reliability. This document is focused on Oracle solution integration with the Kyndryl Bridge platform.
Objective
The primary objectives of Oracle Disaster Recovery (DR) solutions, such as Oracle RAC with DataGuard, are to ensure the availability, reliability, and continuity of critical databases in the event of a disaster or unexpected failure.
The key objectives include:
Business Continuity
- Minimize Downtime: Ensure minimal disruption to business operations by minimizing the downtime associated with database failures.
- Continuous Access: Provide continuous access to data and services even during unexpected incidents.
Data Protection
- Data Integrity: Maintain the integrity of data across primary and standby databases to prevent data loss or corruption.
- Data Consistency: Synchronize changes made to the primary database with standby databases to ensure consistent data across all sites.
Disaster Recovery
- Fast Recovery: Facilitate quick recovery in a disaster by enabling failover to standby databases.
- Redundancy: Establish redundant databases in geographically separated locations to protect against natural disasters, hardware failures or other catastrophic events.
High Availability
- Fault Tolerance: Enhance fault tolerance by configuring multiple standby databases and leveraging Oracle Real Application Clusters (RAC) for load balancing and failover capabilities.
- Load Distribution: Distribute database workload across multiple nodes to prevent overloading and ensure optimal performance.
Scalability
- Horizontal Scaling: Scale database resources horizontally by adding standby databases to the configuration, allowing for increased capacity and performance.
- Vertical Scaling: Utilize Oracle Real Application Clusters (RAC) to vertically scale by adding nodes to the cluster to handle increased demand.
Flexibility
- Snapshot Standby: Allow temporary read-write access to standby databases for reporting or testing purposes without affecting the primary database.
- Configuration Options: Offer flexibility in choosing configurations that suit the specific needs of the organization, such as synchronous or asynchronous replication.
Automated Management
- Data Guard Broker: Streamline the configuration, management, and monitoring of Data Guard environments through the use of the Data Guard Broker, allowing for easier failover and switchover operations.
- Monitoring Tools: Provide tools and utilities for monitoring the health and performance of both primary and standby databases.
Compliance and Security
- Compliance: Ensure compliance with regulatory requirements regarding data availability, integrity and security.
- Security: Implement secure communication and authentication mechanisms between primary and standby databases to protect against unauthorized access.
Architecture diagram, explanation with the required port number and network flow
Oracle RAC Archive logs with DataGuard DR Solution provide business continuity for the applications that run on the Oracle database. Data protection is done using DataGuard to copy log files and other DB-related files between Production and DR (remote) sites. Since this whole process is automated using Kyndryl Resiliency Orchestration, the management is simplified and the DR database is continuously updated.
The following diagram explains how Oracle RAC works with DataGuard to protect Oracle data in the Kyndryl Resiliency Orchestration environment.
Architecture network flow diagram
Supported versions
Supported Versions of the OS and Platforms | Supported Versions of Storage (Database/File System) | Supported Versions Protection Scheme (Replication) |
|---|---|---|
Windows 2016* RHEL 7.0/8.0/8.4/8.6/9.2 Solaris 10/11* AIX 7.1(not supported) | Oracle RAC Logs 19c/21c (64 bits) Dataguard 11g*/18c
| Oracle RAC Logs 19c/21c (64 bits) Dataguard 11g*/18c |
Prerequisites
The following are the prerequisites:
- Intelligence added where the playbook automation will automatically pick the active instance out of multiple instances and perform the recovery operations.
- OS user with sudo permissions to execute Oracle tools:
- lsnrctl
- srvctl
- dgmgrl
- Create an execution environment with SQLPLUS on CACF Tower. If not created, the customer must have a dedicated SQLPLUS client.
- The user must add the Oracle hosts to the inventory file for both PR and DR.
- Create credentials for each host credential type of the machine.
- On CACF Tower, a new credential type will be created to perform the automation. Follow the instructions given below:
- Login to CACF Tower.
- Navigate to Credential type and create a new with the below configuration:
- Credential Types :<custID>-resiliency-oracle
Input configuration
Fields
Field ID | Label | Type | Required | Secret |
|---|---|---|---|---|
username | DB Username | String | Yes | No |
password | DB Password | String | Yes | Yes |
os_username | OS Username | String | Yes | No |
os_password | OS Password | String | Yes | Yes |
Required Fields
- username
- password
- os_username
- os_password
Injector Configuration
Variable | Value |
|---|---|
oracle_user | {{username}} |
oracle_password | {{password}} |
os_username | {{os_username}} |
os_password | {{os_password}} |
Solution Level Privileges
- The user needs admin privileges for this solution.
- Both OS and DB authentication are supported for this solution.
Features of the Solution
The features of this solution are as follows:
- The following features of Discovery and Config Data Model are supported:
- Resource Type
- Data Collection
- BluePrint
- The following features of Insights are supported:
- RPO Calculation
- Datalag Calculation
- The following features of Metrics are supported:
- Oracle Oracle RAC Dataguard Status
- Oracle Log Apply status
- Oracle Oracle RAC DataGuard Mode
- Oracle Database Status
- The following features of Automation are supported:
- Failover
- Switchover
- Switchback
Benefits of the Solution
The benefits of this solution are as follows:
- Advanced prechecks on all automation so that automation gets triggered when the system is in the correct state.
- Advanced Post recovery checks of the database on the DR site.
- Automated enabling the Oracle RAC with DataGuard replication in a reverse way.
- Performs DR operations only when there is no gap w.r.t logs between the primary and the secondary sites.
- Real-time metrics is displayed on the dashboard along with the alerts.
Known Limitations
Solution Limitations
- DB user with sys equivalent with sysdba role is only supported.
- Since Oracle provides datalag in sequence. We can only display the sequence in dashbaord. Displaying datalag in MB is not possible.
- The following is the list of unsupported scenarios for the Oracle RAC Dataguard solution.
- Multi-Site DG environment.
- Fallback is not supported.
- Reverse Failover (Failover to PR post Switchover).
Technology limitations
- Recovery operations in Oracle can only be performed with the user which has the highest roles and privileges.
- Oracle provides datalag in sequence, which can only be displayed in the dashboard. Displaying datalag in MB is not possible.
Setting up of the Solution
Inputs for Onboarding
Configure: Resource type are the properties required to be configured to the existing resource_type.yml under the DNC-data-model.
Fields | Description | Sample Value | Fixed | Mandatory | Recommended Values |
|---|---|---|---|---|---|
Oracle sid name | Enter the Oracle sid name. | orcl | No | Yes | |
Automatic Target Host | Enter the automatic Target Host name. | 192.168.6.109 172.168.6.109 | No | Yes | |
Automatic Execution Environment | Enter the automatic Execution Environment details. | anisible-ee-sql | Yes | No | |
Configured RPO in minutes | Enter the time to Configure RPO in minutes. | 15 | No | Yes | |
Threshold for Acceptable RPO deviation | Enter the threshold for Acceptable RPO deviation. | 20 | No | Yes | |
Compute interval in minutes | Enter the time to compute interval in minutes. | 5 | No | Yes | |
Configured datalag deviation in MB | Enter the configured datalag deviation in MB. | 15 | No | Yes | |
Configured datalag deviation threshold percentage | Enter the configured datalag deviation threshold percentage. | 50 | No | Yes | |
Automation: The following table provides the inputs required to execute the automated features.
Failover: This automation performs the failover test for the solution.
Fields | Description | Sample Value | Fixed | Mandatory | Recommended Values |
|---|---|---|---|---|---|
Credential Label Name | Enter the Credential Label Name | ro1_cred_oracle_dg | No | Yes | |
Oracle sid name | Enter the Oracle sid name | orcl | No | Yes | |
Oracle DR Port Number | Enter the Oracle DR Port Number | 1521 | No | Yes | |
Oracle DR Server | Enter the Oracle DR Server | 172.168.6.109 | No | Yes | |
Oracle DB Role | Enter the Oracle DB Role | sysdba | No | Yes | |
Oracle PR Server | Enter the Oracle PR Server | 192.168.6.109 | No | Yes | |
Automatic Target Host | Enter the Automatic Target Host | 192.168.6.109 172.168.6.109 | No | Yes | |
Automatic Execution Environment | Enter the Automatic Execution Environment details | anisible-ee-sql | Yes | No | |
Error code and recommendation
Return Code Group | Return Code | Comments |
|---|---|---|
connection_issue | 8521,8524,8526 | Unable to connect to the host. Please check the user name and pwd |
misconfiguration | 8501 | dr_db_port is not configured |
misconfiguration | 8502 | oracle_dr_db_name is not configured. |
misconfiguration | 8503 | oracle_dr_db_role is not configured. |
misconfiguration | 8504 | oracle_db_role is not configured. |
misconfiguration | 8505 | Either PR or DR node group is missing or incorrect. |
misconfiguration | 8506 | All nodes are not reachable |
misconfiguration | 8507 | Nodes are reachable but none of them is active status. |
component_playbook | 8508 | ERROR: Oracle RAC database instance is not in running state |
component_playbook | 8509 | Technical issue in running the srvctl command. |
misconfiguration | 8510 | Reachable but not node server is not active. |
component_playbook | 8511 | no DataBase is available |
component_playbook | 8522 | DB execution error |
misconfiguration | 8523 | database validation failed |
component_playbook | 8525 | ERROR: database instance failed to stop. |
component_playbook | 8528 | ERROR: Failed the backup of the control files. |
component_playbook | 8529 | ERROR : Failed to get the resources of RAC. |
component_playbook | 8530 | ERROR: Failed to get the database instance state |
component_playbook | 8531 | ERROR : Failed to get the status of RAC DB |
component_playbook | 8532 | Role in not a expected RAC db role |
component_playbook | 8527 | ERROR: database instance failed to start. |